Deserialize complex JSON with pandas - json

Question

I am trying to deserialize some complex JSON that is also inconsistent with pandas and struggling to get the parsing right
{
"STATUS":"REQUEST_OK",
"DATA":[
{
"companyID":"AABBCCDD",
"ITEMS":[
{
"ind":"12345",
"pt":"1231",
"code":"E333",
"name":"Pop ,",
"RES":[
{
"i":1,
"D":{
"e":123674,
"p":"",
"s":"",
"t":1000
},
"lot":"073",
"V":[
{
"t":6,
"v":0.1
}
],
"p":1
},
{
}
]
},
{
"ind":"423",
"pt":"571",
"code":"E1",
"name":"Dam ,",
"RES":[
{
"i":5,
"D":{
"e":120751,
"p":"",
"s":"",
"t":800
},
"lot":"9",
"V":[
{
"t":4543,
"v":1.33
}
],
"p":1
},
{
}
]
},
{
"ind":"0323",
"pt":"123221",
"code":"LS",
"name":"Paint ,",
"RES":[
{
"i":61,
"D":{
"e":946,
"p":"",
"s":"",
"t":11100
},
"lot":"8",
"V":[
{
"t":9,
"v":0.06
}
],
"p":1
},
{
}
]
}
]
}
]
}
The data here is supposed to build this table
|companyID | ind |pt |code |name |i |e |p |s |t |lot |t |v |p
|------------------------------------------------------------------------------------------------
|AABBCCDD |12345 |1231 |E333 |Pop |1 |123674 | | |1000 |073 |6 |0.1 |1
|------------------------------------------------------------------------------------------------
|----
And so on.
The biggest pain for me is that there could be only 1 level of this tag
{
"ind":"423",
"pt":"571",
"code":"E1",
"name":"Dam ,",
"RES":[
{
but inside it, multiple
{
"i":61,
"D":{
"e":946,
"p":"",
"s":"",
"t":11100
},
"lot":"8",
"V":[
{
"t":9,
"v":0.06
}
],
"p":1
},
Example
i:61
means there are 61 of those json tags inside of the first tag.
Any clues on how to parse this JSON?

Answer 1

Try it this way:
data = """your json above"""
import pandas as pd
key = []
value = []
new_dat = data.split(',')
for n in new_dat:
if '{' in n:
m = n.split('{')[1].strip()
else:
m = n.strip().replace('}','').replace('\n','')
if ':' in m:
key.append(m.split(':')[0])
value.append(m.split(':')[1])
pd.DataFrame([value],columns=key)
Output:
"STATUS" "companyID" "ind" "pt" "code" "name" "i" "e" "p" "s" ... "name" "i" "e" "p" "s" "t" "lot" "t" "v" "p"
0 "REQUEST_OK" "AABBCCDD" "12345" "1231" "E333" "Pop 1 123674 "" "" ... "Paint 61 946 "" "" 11100 "8" 9 0.06 ] 1
You can then use standard pandas methods to drop unnecessary columns, etc.