Problem parsing JSON file - fields not stored correctly - json

Question

I have a JSON file that I'm trying to parse. My script has a few print statements to check if I'm parsing the various fields correctly. It appears to be that way for ids and ip_metadata[ip] within the for loop. However, when I print out the entire dictionary at the end, the ids list is always the last ones. Only ids has this problem, the other fields were stored correctly. I can't see why ids isn't stored correctly.
JSON file:
{
"count": 4,
"data": [
{
"ip": "1.2.3.4",
"cty": {
"country": "US",
"organization": "ABC"
},
"info": {
"p": [
{
"id": 123,
"grp": "A"
},
{
"id": 234,
"grp": "B"
},
{
"id": 345,
"grp": "C"
},
{
"id": 456,
"grp": "D"
}
]
}
},
{
"ip": "2.3.4.5",
"cty": {
"country": "US",
"organization": "ABC"
},
"info": {
"p": [
{
"id": 111,
"grp": "A"
},
{
"id": 222,
"grp": "B"
},
{
"id": 333,
"grp": "C"
},
{
"id": 444,
"grp": "D"
}
]
}
},
{
"ip": "1.2.3.1",
"cty": {
"country": "AU",
"organization": "ABC"
},
"info": {
"p": [
{
"id": 222,
"grp": "A"
},
{
"id": 333,
"grp": "B"
},
{
"id": 444,
"grp": "C"
}
]
}
},
{
"ip": "10.2.3.4",
"cty": {
"country": "US",
"organization": "DDD"
},
"info": {
"p": [
{
"id": 555,
"grp": "A"
},
{
"id": 666,
"grp": "B"
},
{
"id": 777,
"grp": "C"
},
{
"id": 888,
"grp": "D"
}
]
}
}
],
"status": "ok"
}
My python script is
import json
import glob
from collections import defaultdict
ip_metadata = defaultdict(list)
def main():
for json_file in glob.glob("test_input/test_json.json"):
with open(json_file, "r") as fin:
ids = []
json_data = json.load(fin)
if json_data["count"] > 0:
for data in json_data['data']:
ip = data['ip']
country = data['cty']['country']
organization = data['cty']['organization']
ids[:] = []
ids_2 = data['info']['p']
for idss in ids_2:
id = idss['id']
grp = idss['grp']
ids.append((id,grp))
print(ids)
ip_metadata[ip].append((country,organization,ids))
print(ip_metadata[ip])
print("=============================")
for k, v in ip_metadata.items():
print(k,v)
if __name__ == '__main__':
main()
The output is
[(123, 'A'), (234, 'B'), (345, 'C'), (456, 'D')]
[('US', 'ABC', [(123, 'A'), (234, 'B'), (345, 'C'), (456, 'D')])]
[(111, 'A'), (222, 'B'), (333, 'C'), (444, 'D')]
[('US', 'ABC', [(111, 'A'), (222, 'B'), (333, 'C'), (444, 'D')])]
[(222, 'A'), (333, 'B'), (444, 'C')]
[('AU', 'ABC', [(222, 'A'), (333, 'B'), (444, 'C')])]
[(555, 'A'), (666, 'B'), (777, 'C'), (888, 'D')]
[('US', 'DDD', [(555, 'A'), (666, 'B'), (777, 'C'), (888, 'D')])]
=============================
1.2.3.4 [('US', 'ABC', [(555, 'A'), (666, 'B'), (777, 'C'), (888, 'D')])]
2.3.4.5 [('US', 'ABC', [(555, 'A'), (666, 'B'), (777, 'C'), (888, 'D')])]
1.2.3.1 [('AU', 'ABC', [(555, 'A'), (666, 'B'), (777, 'C'), (888, 'D')])]
10.2.3.4 [('US', 'DDD', [(555, 'A'), (666, 'B'), (777, 'C'), (888, 'D')])]

Answer 1

I managed to resolve this by removing the ids = [] line under the open() line, and replace ids[:] = [] with ids = [] instead. I don't quite understand why this fixes things, because I don't know why the value of ids would replace previously stored values of ids of different keys. But it works now.