dataframe to JSON conversion - json

Question

I have a dataframe that i need to convert to JSON. The data currently looks like
text ids
0 add a car None
1 None 695f1
2 None a86b5c
3 add another car to my log None
4 None 1ba0
5 Concerts None
6 None a4f7
7 None fea
8 None 410
I need the JSON dict to look something like this and ignore the none
{
"text": "add a car",
"ids": [
"695f1",
"a86b5c"
]
}
The steps I have are:
Figured it out.
first set NaN Values where None
df1 = df1.fillna(value=np.nan)
Fill with NaN with previous known value
df1['text'] = df['text'].fillna(method='ffill')
Drop NaN
df1 = df1.dropna()
convert to json
df1.to_json('temp.json', orient='records', lines=True)
The problem I have is the format appears incorrect. I am seeing
{"text":"add a car","ids":" 695f1"}
{"text":"add a car","ids":"a86b5c"}
{"text":"add another car to my log","ids":"1ba0"}
{"text":"Concerts","ids":"a4f7"}
I want :
{
"text": "add a car",
"ids":
[
"695f1",
"a86b5c"
]
}
{
"text": "add another car to my log",
"ids":
[
"1ba0",
],
}
{
"text": "Concerts",
"ids":
[
"a4f7",
"fea",
"410",
]
}

Answer 1

I think you are close, need aggregate list first:
df['text'] = df['text'].ffill()
df = df.dropna()
df1 = df.groupby('text', sort=False).agg(list).reset_index()
print (df1)
text ids
0 add a car [695f1, a86b5c]
1 add another car to my log [1ba0]
2 Concerts [a4f7, fea, 410]
df1.to_json('temp.json', orient='records')