Convert Nested JSON to Pandas Column - json

Question

I have a Json file as below:
[{
'instrument_token': 12335618, 'last_price': 31584.6,
'ohlc': {'open': 31080.1, 'high': 31590.0, 'low': 31049.05, 'close': 31114.7},
'depth': {'buy': [{'quantity': 40, 'price': 31576.4, 'orders': 1}, {'quantity': 160, 'price': 31576.0, 'orders': 1}], 'sell': [{'quantity': 200, 'price': 31584.6, 'orders': 2}, {'quantity': 60, 'price': 31584.65, 'orders': 1}]}
}]
I have tried as below:
df = json_normalize(ticks)
print(df)
This gives me a result as:
instrument_token last_price ohlc.open ohlc.high ohlc.low ohlc.close depth.buy depth.sell
0 12335618 31584.6 31080.1 31590.0 31049.05 31114.7 [{'quantity': 40, 'price': 31576.4, 'orders': ... [{'quantity': 200, 'price': 31584.6, 'orders':...
I want to further normalize the data of depth.buy & depth.sell columns in separate columns with column name as:
depth.buy.quantity1, depth.buy.price1, depth.buy.orders1,
depth.buy.quantity2, depth.buy.price2, depth.buy.orders2,
depth.sell.quantity1, depth.sell.price1, depth.sell.orders1,
depth.sell.quantity2, depth.sell.price2, depth.sell.orders2,
Is it possible to normalise further?

Answer 1

for this example dataset,you can do it like this.
from pandas.io.json import json_normalize
data = [{
'instrument_token': 12335618, 'last_price': 31584.6,
'ohlc': {'open': 31080.1, 'high': 31590.0, 'low': 31049.05, 'close': 31114.7},
'depth': {'buy': [{'quantity': 40, 'price': 31576.4, 'orders': 1},
{'quantity': 160, 'price': 31576.0, 'orders': 1}],
'sell': [{'quantity': 200, 'price': 31584.6, 'orders': 2},
{'quantity': 60, 'price': 31584.65, 'orders': 1}]
}
}]
df = json_normalize(data)
cols = ["depth.buy","depth.sell"]
for c in cols:
postdf = pd.DataFrame()
tmp = df[c].values
for i,val in enumerate(tmp):
vals = list(val.values())
keys = [f"{c}.{k}{i+1}"for k in val.keys()]
tmpdf =pd.DataFrame([vals],columns=keys)
postdf = pd.concat([postdf,tmpdf],axis=1)
df = pd.concat([df,postdf],axis=1)
df = df.drop(columns=[c])
note that,
tmp = df[c].values take the 0th element of the list.if you have multiple elements you have to loop over the elements.i assumed that all data inside one list.
if you need to get column names list (["depth.buy","depth.sell"]) dynamically,you can do it by checking dtypes of the df and get object type column names.