splitting JSON file using python - json

Question

I have below big json file
{
"sections": [
{
"facts": [
{
"name": "Server",
"value": "<https://xxxxxxx:18443/collector/pipeline/v1_allagents>"
},
{
"name": "Environment",
"value": "dev"
},
{
"name": "Issue",
"value": "Server is [EDITED]"
}
]
},
{
"facts": [
{
"name": "Server",
"value": "<https://xxxxx:18443/collector/pipeline/customer-characterstics-v1>"
},
{
"name": "Environment",
"value": "dev"
},
{
"name": "Issue",
"value": "Server is [STOPPED]"
}
]
}
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxx:18443/collector/pipeline/soap-post-v1_relations>'},
{'name': 'Environment', 'value': u'dev'}, {'name': 'Issue', 'value': u' status is [STOPPED]'}
]
},
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxxx.134:18443/collector/pipeline/characterstics-v1_allagents>'},
{'name': 'Environment', 'value': u'dev'}, {'name': 'Issue', 'value': u' status is [EDITED]'}
]
},
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxxx:18443/collector/pipeline/ab23-8128b7c9fcf2>'},
{'name': 'Environment', 'value': u'dev'}, {'name': 'Issue', 'value': u'status is [EDITED]'}
]
}
]
}
....
now I'm struggling to split above file as below and dump into another new files:
{
"text": "Status",
"themeColor": "#FF0000",
{
"sections": [
{
"facts": [
{
"name": "Server",
"value": "<https://xxxxxxx:18443/collector/pipeline/v1_allagents>"
},
{
"name": "Environment",
"value": "dev"
},
{
"name": "Issue",
"value": "Server is [EDITED]"
}
]
}
]
}
}
what I could able to achieve so far is print each tags under facts, but not the way I expect as above.
so, I'm having trouble adding those extra lines prior the final ones and then dump it to another file.
How should I approach this? not using JQ.
each splitted file should have same header and then exactly same pattern for key sections and facts .
edit:
As per Andrej's solution it works perfectly alright for one split at a time.
But how to split the file based on n size, let's say I want to split my original big file where 5 facts exists 2 facts per file.n = 2
so, it should create 3 json files , where first 2 contains 2 blocks of facts and last one should be only one since that's left.
Then final output should be:
{'text': ' Status', 'themeColor': '#FF0000', 'sections':
[
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxx:18443/collector/pipeline/soap-post-v1>'},
{'name': 'Environment', 'value': u'dev'},
{'name': 'Issue', 'value': u' status is [STOPPED]'}
]
},
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxx:18443/collector/pipeline/be9694085a70>'},
{'name': 'Environment', 'value': u'dev'},
{'name': 'Issue', 'value': u' status is [STOPPED]'}
]
}
]
}
and
{'text': ' Status', 'themeColor': '#FF0000', 'sections':
[
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxx:18443/collector/pipeline/soap-post-v1_relations>'},
{'name': 'Environment', 'value': u'dev'}, {'name': 'Issue', 'value': u' status is [STOPPED]'}
]
},
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxxx.134:18443/collector/pipeline/characterstics-v1_allagents>'},
{'name': 'Environment', 'value': u'dev'}, {'name': 'Issue', 'value': u' status is [EDITED]'}
]
}
]}
as per above one block of fact from original file, hence it will create it's own json
{'text': ' Status', 'themeColor': '#FF0000', 'sections':
[
{'facts':
[
{'name': 'Server', 'value': u'<https://xxxxxxx:18443/collector/pipeline/ab23-8128b7c9fcf2>'},
{'name': 'Environment', 'value': u'dev'}, {'name': 'Issue', 'value': u'status is [EDITED]'}
]
}
]}

Answer 1

You can load the big file json into dictionary using json module. Then treat the loaded data as classical Python dict.
If your file contains the string in question, then this example:
import json
with open('YOUR_JSON_FILE.json', 'r') as f_in:
data = json.load(f_in)
for i, fact in enumerate(data['sections'], 1):
with open('data_out_{}.json'.format(i), 'w') as f_out:
d = {}
d['text'] = 'Status'
d['themeColor'] = '#FF0000'
d['sections'] = fact
json.dump(d, f_out, indent=4)
This creates two files data_out_1.json and data_out_2.json containing:
{
"text": "Status",
"themeColor": "#FF0000",
"sections": {
"facts": [
{
"name": "Server",
"value": "<https://xxxxxxx:18443/collector/pipeline/v1_allagents>"
},
{
"name": "Environment",
"value": "dev"
},
{
"name": "Issue",
"value": "Server is [EDITED]"
}
]
}
}
and
{
"text": "Status",
"themeColor": "#FF0000",
"sections": {
"facts": [
{
"name": "Server",
"value": "<https://xxxxx:18443/collector/pipeline/customer-characterstics-v1>"
},
{
"name": "Environment",
"value": "dev"
},
{
"name": "Issue",
"value": "Server is [STOPPED]"
}
]
}
}
EDIT:
To chunk the JSON file, you can use this example:
import json
def chunk(lst, n):
for i in range(0, len(lst), n):
yield lst[i:i + n]
with open('YOUR_JSON_FILE.json', 'r') as f_in:
data = json.load(f_in)
for i, fact in enumerate(chunk(data['sections'], 2), 1): # <-- change 2 to your chunk size
with open('data_out_{}.json'.format(i), 'w') as f_out:
d = {}
d['text'] = 'Status'
d['themeColor'] = '#FF0000'
d['sections'] = fact
json.dump(d, f_out, indent=4)

Answer 2

import json
with open('/tmp/json_response_output.json') as datafile:
datastore = json.load(datafile)
for n, details in enumerate(datastore['sections']):
split_json = datastore.copy()
split_json['sections'] = [details]
with open(f'json_response_output_part{n}.json', 'w') as f:
json.dump(split_json, f, indent=4, ensure_ascii=False)