Trying to merge two json using python dictionary - json

Question

I am trying to merge two json stored in python dictionaries. Here, the first dictionary is the parent into which the second dictionary gets merged into. In reality, the second dictionary represents a second line in a TSV file that holds the next record of a json array.
My program is trying to read the TSV file line by line and merging them into one single nested json.
Let us consider the two dictionaries:
Parent dictionary dict1: {"CA": [{"Marin": [{"zip":1}], "population":10000}]}
and,
dict2: {"CA": {"Marin": {"zip":2}}}
Note: dict1 is the source-of-truth with regards to the correct json structure.
Here, as you can see, I would like to append the zip: 2 into the Marin county of California state.
Here is my merge code:
class MyClass:
# merges two lines containing json arrays inside nested json
def merge_lines(dict1: dict, dict2: dict) -> dict:
for key in dict1:
if key in dict2:
if isinstance(dict1[key], dict):
MyClass.merge_lines(dict1[key], dict2[key])
elif isinstance(dict1[key], list):
to_be_merged_list = [dict2[key]]
dict1[key].extend(to_be_merged_list)
return dict1
Below is how I am trying to test:
def test_nested_json_arrays(self):
d1 = {"CA": [{"Marin": [{"zip":1}], "population":10000}]}
d2 = {"CA": {"Marin": {"zip":2}}}
expected_result = {"CA": [{"Marin": [{"zip":1}, {"zip":2}], "population":1000}]}
actual = MyClass.merge_lines(d1, d2)
assert expected_result == actual
However, I am getting the below result:
E AssertionError: assert {'CA': [{'Mar...tion': 1000}]} == {'CA': [{'Mari... {'zip': 2}}]}
E Differing items:
E {'CA': [{'Marin': [{'zip': 1}, {'zip': 2}], 'population': 1000}]} != {'CA': [{'Marin': [{'zip': 1}], 'population': 10000}, {'Marin': {'zip': 2}}]}
E Full diff:
E - {'CA': [{'Marin': [{'zip': 1}, {'zip': 2}], 'population': 1000}]}
E + {'CA': [{'Marin': [{'zip': 1}], 'population': 10000}, {'Marin': {'zip': 2}}]}
Can someone help me figure out the changes required in the code to fix this?
Note: the field names are not constant and this can apply to any combination of country, state, county, zip and other nested attributes.

Answer 1

dict1 = {"CA": [{"Marin": [{"zip":1}], "population":10000}]}
dict2 = {"CA": {"Marin": {"zip":2}}}
Looking at dict2:
You are given dict2, containing key k (in this case k = "CA")
dict2[k] itself is a dictionary, that contains one (or more) key (c = "Marin") - value (z) pair(s)
z now is the dictionary that you care about.
Looking at dict1:
For each element county_info in dict1[k], you care about the one that has a key c.
The value at this key (county_info[c]) is a list, to which you want to append z
So let's do that:
def merge_lines(dict1, dict2):
for k, v in dict2.items():
for c, z in v.items():
# Find the element of dict1[k] that has the key c:
for county_info in dict1[k]:
if c in county_info:
county_info[c].append(z)
break
Since the function modifies dict1 in-place, running merge_lines(dict1, dict2) gives us a modified dict1 that looks like what you expect:
{'CA': [{'Marin': [{'zip': 1}, {'zip': 2}], 'population': 10000}]}

Answer 2

This is a more generic approach, where d1 is updated with changes contained in d2.
Of course, you should provide more examples of updates from d2 to see if my solution works in all cases.
#!/usr/bin/env python3
d1 = {"CA": [{"Marin": [{"zip": 1}], "population": 10000}]}
d2 = {"CA": {"Marin": {"zip": 2}}}
# read update data from d2 and append new values to d1
def update(d1, d2):
for key, value in d2.items():
if key in d1:
for key1, value1 in value.items():
if key1 in d1[key][0]:
d1[key][0][key1].append(value1)
else:
d1[key][0][key1] = [value1]
else:
d1[key] = [value]
return d1
print(update(d1, d2))