Could someone help me getting the nested JSON to normalized JSON objects, I have deep nested JSON structure. I have seen examples using the flatten structure, However having one flatten structure doesnt help me in my case.
Sample dictionary:
emp = {
'name': "Paul",
'Age': '25',
'Location': "USA",
'Addresses':[
{
"longitude": "987",
"latitude": "765",
"postal_code": "90266" ,
"area" : [{"state": "CA"}]
},
{
"longitude": "123",
"latitude": "456",
"postal_ode": "1234" ,
"area" : [{"state": "NY"}]
}
]
}
expected Normalized objects
emp = ['name': "Paul", 'Age': '25','Location': "USA"}
emp_address = [
{ "longitude": "987", "latitude": "765","postal_code": "90266" },
{ "longitude": "123", "latitude": "456","postal_code": "1234" }
]
emp_address_area = [{"state": "CA"}, {"state": "NY"}]
I tried recursive functions to solve this problem, but no luck.
def normalized_json(y):
out = {}
def flatten(x, name=''):
if type(x) is dict:
for k, v in x.items():
if not isinstance (v, (dict, list)):
pass
flatten(v, name + k + '#')
elif type(x) is list:
i = 0
for a in x:
flatten(a, name + str(i) + '#')
i += 1
else:
out[name[:-1]] = x
flatten(y)
return out
print(normalized_json(emp))
Related
I am trying to parse a nested JSON and trying to collect data into a list under some condition.
Input JSON as below:
[
{
"name": "Thomas",
"place": "USA",
"items": [
{"item_name":"This is a book shelf", "level":1},
{"item_name":"Introduction", "level":1},
{"item_name":"parts", "level":2},
{"item_name":"market place", "level":3},
{"item_name":"books", "level":1},
{"item_name":"pens", "level":1},
{"item_name":"pencils", "level":1}
],
"descriptions": [
{"item_name": "Books"}
]
},
{
"name": "Samy",
"place": "UK",
"items": [
{"item_name":"This is a cat house", "level":1},
{"item_name":"Introduction", "level":1},
{"item_name":"dog house", "level":3},
{"item_name":"cat house", "level":1},
{"item_name":"cat food", "level":2},
{"item_name":"cat name", "level":1},
{"item_name":"Samy", "level":2}
],
"descriptions": [
{"item_name": "cat"}
]
}
]
I am reading json as below:
with open('test.json', 'r', encoding='utf8') as fp:
data = json.load(fp)
for i in data:
if i['name'] == "Thomas":
#collect "item_name", values in a list (my_list) if "level":1
#my_list = []
Expected output:
my_list = ["This is a book shelf", "Introduction", "books", "pens", "pencils"]
Since it's a nested complex JSON, I am not able to collect the data into a list as I mentioned above. Please let me know no how to collect the data from the nested JSON.
Try:
import json
with open("test.json", "r", encoding="utf8") as fp:
data = json.load(fp)
my_list = [
i["item_name"]
for d in data
for i in d["items"]
if d["name"] == "Thomas" and i["level"] == 1
]
print(my_list)
This prints:
['This is a book shelf', 'Introduction', 'books', 'pens', 'pencils']
Or without list comprehension:
my_list = []
for d in data:
if d["name"] != "Thomas":
continue
for i in d["items"]:
if i["level"] == 1:
my_list.append(i["item_name"])
print(my_list)
Once we have the data we iterate over the outermost list of objects.
We check if the object has the name equals to "Thomas" if true then we apply filter method with a lambda function on items list with a condition of level == 1
This gives us a list of item objects who have level = 1
In order to extract the item_name we use a comprehension so the final value in the final_list will be as you have expected.
["This is a book shelf", "Introduction", "books", "pens", "pencils"]
import json
def get_final_list():
with open('test.json', 'r', encoding='utf8') as fp:
data = json.load(fp)
final_list = []
for obj in data:
if obj.get("name") == "Thomas":
x = list(filter(lambda item: item['level'] == 1, obj.get("items")))
final_list = final_list + x
final_list = [i.get("item_name") for i in final_list]
return final_list
JSON Response:
[
{
"actNum": "12345678",
"prodType": "Test",
"period": {
"January": [
{
"name": "Jake",
"rRar": 12.34,
"lRar": 340.45,
"address": "New York"
},
{
"name": "Jorge",
"rRar": 28.78,
"lRar": 250.49,
"address": "Chicago"
}
]
}
}
]
I have to verify that numeric fields in the above response rRar and lRar should have value till two decimal points like 12.78,32.56.
Could anyone please help me if this validation can be done using Karate API?
Here you go:
* def nums = $response..rRar
* def temp = $response..lRar
* karate.appendTo(nums, temp)
* def strs = karate.map(nums, function(x){ return x + '' })
* match each strs == '#regex [0-9].+\\.[0-9]{2}'
I want to read a JSON value using a dynamic key using Python. For example, I have a JSON in the following format
{
"Personal": {
"Address": {
"City": "Newyork",
"Country": "USA"
}
},
"number": 123,
"object": {
"a": "b",
"c": "d",
"e": "f"
}
}
If I enter a string as "Personal.Address.City", I need to get city value using Python.
It seems like you have to split the key and look up level-by-level manually. I give two implementations below:
data = {
"Personal": {
"Address": {
"City": "Newyork",
"Country": "USA"
}
},
"number": 123,
"object": {
"a": "b",
"c": "d",
"e": "f"
}
}
key = 'Personal.Address.City'
def lookup_dot_separated_key(data, key):
value = data
for k in key.split('.'):
value = value[k]
return value
print(lookup_dot_separated_key(data, key))
def lookup_key_list(data, keys):
if keys:
return lookup_key_list(data[keys[0]], keys[1:])
else:
return data
print(lookup_key_list(data, key.split('.')))
I want to merge several lists into one JSON array.
These are my two lists:
address = ['address1','address2']
temp = ['temp1','temp2']
I combine both lists by the following call and create a JSON .
new_list = list(map(list, zip(address, temp)))
jsonify({
'data': new_list
})
This is my result for the call:
{
"data": [
[
"address1",
"temp1"
],
[
"address2",
"temp2"
]
]
}
However, I would like to receive the following issue. How do I do that and how can I insert the identifier address and hello.
{
"data": [
{
"address": "address1",
"temp": "temp1"
},
{
"address": "address2",
"temp": "temp2"
}
]
}
You can use a list-comprehension:
import json
address = ['address1','address2']
temp = ['temp1','temp2']
d = {'data': [{'address': a, 'temp': t} for a, t in zip(address, temp)]}
print( json.dumps(d, indent=4) )
Prints:
{
"data": [
{
"address": "address1",
"temp": "temp1"
},
{
"address": "address2",
"temp": "temp2"
}
]
}
You can just change your existing code like this. That lambda function will do the trick of converting it into a dict.
address = ['address1','address2']
temp = ['temp1','temp2']
new_list = list(map(lambda x : {'address': x[0], 'temp': x[1]}, zip(address, temp)))
jsonify({
'data': new_list
})
Python noob here, again. I'm trying to create a python script to auto-generate a JSON with multiple item but records multiple times using a for loop to generate them, the JSON message is structured and cardinality are as follows:
messageHeader[1]
-item [1-*]
--itemAttributesA [0-1]
--itemAttributesB [0-1]
--itemAttributesC [0-1]
--itemLocaton [1]
--itemRelationships [0-1]
I've had some really good help before for looping through the same object but for one record for example just the itemRelationships record. However as soon as I try to create one message with many items (i.e. 5) and a single instance of an itemAttribute, itemLocation and itemRelationships it does not work as I keep getting a key error. I've tried to define what a keyError is in relation to what I am trying to do but cannot link what I am doing wrong to the examples else where.
Here's my code as it stands:
import json
import random
data = {'messageID': random.randint(0, 2147483647), 'messageType': 'messageType'}
data['item'] = list()
itemAttributeType = input("Please selct what type of Attribute item has, either 'A', 'B' or 'C' :")
for x in range(0, 5):
data['item'].append({
'itemId': "I",
'itemType': "T"})
if itemAttributeType == "A":
data['item'][0]['itemAttributesA']
data['item'][0]['itemAttributesA'].append({
'attributeA': "ITA"})
elif itemAttributeType == "B":
data['item'][0]['itemAttributesB']
data['item'][0]['itemAttributesB'].append({
'attributeC': "ITB"})
else:
data['item'][0]['itemAttributesC']
data['item'][0]['itemAttributesC'].append({
'attributeC': "ITC"})
pass
data['item'][0]['itemLocation'] = {
'itemDetail': "ITC"}
itemRelation = input("Does the item have a relation: ")
if itemRelation > '':
data['item'][0]['itemRelations'] = {
'itemDetail': "relation"}
else:
pass
print(json.dumps(data, indent=4))
I have tried also tried this code which gives me better results:
import json
import random
data = {'messageID': random.randint(0, 2147483647), 'messageType': 'messageType'}
data['item'] = list()
itemAttributeType = input("Please selct what type of Attribute item has, either 'A', 'B' or 'C' :")
for x in range(0, 5):
data['item'].append({
'itemId': "I",
'itemType': "T"})
if itemAttributeType == "A":
data['item'][0]['itemAttributesA'] = {
'attributeA': "ITA"}
elif itemAttributeType == "B":
data['item'][0]['itemAttributesB'] = {
'attributeB': "ITB"}
else:
data['item'][0]['itemAttributesC'] = {
'attributeC': "ITC"}
pass
data['item'][0]['itemLocation'] = {
'itemDetail': "ITC"}
itemRelation = input("Does the item have a relation: ")
if itemRelation > '':
data['item'][0]['itemRelations'] = {
'itemDetail': "relation"}
else:
pass
print(json.dumps(data, indent=4))
This actually gives me a result but gives me messageHeader, item, itemAttributeA, itemLocation, itemRelations, and then four items records at the end as follows:
{
"messageID": 1926708779,
"messageType": "messageType",
"item": [
{
"itemId": "I",
"itemType": "T",
"itemAttributesA": {
"itemLocationType": "ITA"
},
"itemLocation": {
"itemDetail": "location"
},
"itemRelations": {
"itemDetail": "relation"
}
},
{
"itemId": "I",
"itemType": "T"
},
{
"itemId": "I",
"itemType": "T"
},
{
"itemId": "I",
"itemType": "T"
},
{
"itemId": "I",
"itemType": "T"
}
]
}
What I am trying to achieve is this output:
{
"messageID": 2018369867,
"messageType": "messageType",
"item": [{
"itemId": "I",
"itemType": "T",
"itemAttributesA": {
"attributeA": "ITA"
},
"itemLocation": {
"itemDetail": "Location"
},
"itemRelation": [{
"itemDetail": "D"
}]
}, {
"item": [{
"itemId": "I",
"itemType": "T",
"itemAttributesB": {
"attributeA": "ITB"
},
"itemLocation": {
"itemDetail": "Location"
},
"itemRelation": [{
"itemDetail": "D"
}]
}, {
"item": [{
"itemId": "I",
"itemType": "T",
"itemAttributesC": {
"attributeA": "ITC"
},
"itemLocation": {
"itemDetail": "Location"
},
"itemRelation": [{
"itemDetail": "D"
}]
}, {
"item": [{
"itemId": "I",
"itemType": "T",
"itemAttributesA": {
"attributeA": "ITA"
},
"itemLocation": {
"itemDetail": "Location"
},
"itemRelation": [{
"itemDetail": "D"
}]
},
{
"item": [{
"itemId": "I",
"itemType": "T",
"itemAttributesB": {
"attributeA": "ITB"
},
"itemLocation": {
"itemDetail": "Location"
},
"itemRelation": [{
"itemDetail": "D"
}]
}]
}
]
}]
}]
}]
}
I've been at this for the best part of a whole day trying to get it to work, butchering away at code, where am I going wrong, any help would be greatly appreciated
Your close. I think the part your are missing is adding the dict to your current dict and indentation with your for loop.
import json
import random
data = {'messageID': random.randint(0, 2147483647), 'messageType': 'messageType'}
data['item'] = list()
itemAttributeType = input("Please selct what type of Attribute item has, either 'A', 'B' or 'C' :")
for x in range(0, 5):
data['item'].append({
'itemId': "I",
'itemType': "T"})
if itemAttributeType == "A":
# First you need to add `itemAttributesA` to your dict:
data['item'][x]['itemAttributesA'] = dict()
# You could also do data['item'][x] = {'itemAttributesA': = dict()}
data['item'][x]['itemAttributesA']['attributeA'] = "ITA"
elif itemAttributeType == "B":
data['item'][x]['itemAttributesB'] = dict()
data['item'][x]['itemAttributesB']['attributeC'] = "ITB"
else:
data['item'][x]['itemAttributesC'] = dict()
data['item'][x]['itemAttributesC']['attributeC'] = "ITC"
data['item'][x]['itemLocation'] = {'itemDetail': "ITC"}
itemRelation = input("Does the item have a relation: ")
if itemRelation > '':
data['item'][x]['itemRelations'] = {'itemDetail': "relation"}
else:
pass
print(json.dumps(data, indent=4))
This code can also be shortened considerably if your example is close to what you truly desire:
import json
import random
data = {'messageID': random.randint(0, 2147483647), 'messageType': 'messageType'}
data['item'] = list()
itemAttributeType = input("Please selct what type of Attribute item has, either 'A', 'B' or 'C' :")
for x in range(0, 5):
new_item = {
'itemId': "I",
'itemType': "T",
'itemAttributes' + str(itemAttributeType): {
'attribute' + str(itemAttributeType): "IT" + str(itemAttributeType)
},
'itemLocation': {'itemDetail': "ITC"}
}
itemRelation = input("Does the item have a relation: ")
if itemRelation > '':
new_item['itemRelations'] = {'itemDetail': itemRelation}
data['item'].append(new_item)
print(json.dumps(data, indent=4))
Another note: If you want messageID to be truly unique than you should probably look into a UUID; otherwise you may have message ids that match.
import uuid
unique_id = str(uuid.uuid4())
print(unique_id)