I have file that is list of JSON objects. It looks like this :
[
{
"id": 748,
"location": {
"slug": "istanbul",
"parent": {
"id": 442,
"slug": "turkey"
}
},
"rank": 110
},
{
"id": 769,
"location": {
"slug": "dubai",
"parent": {
"id": 473,
"slug": "uae"
}
},
"rank": 24
}
]
I want to create a list of hotel parent names, so i write this code to do this, I read the JSON file and assigned it to a variable, that part is correct. But look at this code :
with open('hotels.json', 'r', encoding="utf8") as hotels_data:
hotels = json.load(hotels_data)
parents_list = []
for item in hotels:
if item["location"]["parent"]["slug"] not in parents_list:
parents_list.append(item["location"]["parent"])
when i run this code, i give this error :
if item["location"]["parent"]["slug"] not in parents_list:
TypeError: 'NoneType' object is not subscriptable
This code does not work, so I tried to print the JSON objects so I wrote this in the loop:
print(item["location"]["parent"]["slug"])
This code prints the values I want, but also give me the exact same error.
thank you for any help.
I tried running the code and it seems to be working fine with your dataset.
However, instead of opening the file to read the data, I just assigned hotels with your dataset, hotels = [...].
The result I got was this:
[{'id': 442, 'slug': 'turkey'}, {'id': 473, 'slug': 'uae'}]
What is your result if you print hotels, is it the same as you shown here?
If you actually have a lot more data in your dataset, then I can presume that some of the dictionaries don't contain item["location"]["parent"]["slug"]. If that is the case, you should skip those by checking if that element exists in each item first before reading off from the parents_list.
For example:
try:
item["location"]["parent"]["slug"]
except (KeyError, TypeError) as e:
pass
else:
if item["location"]["parent"]["slug"] not in parents_list:
parents_list.append(item["location"]["parent"])
I cannot replicate the same error as you. The only thing that I can think of is that the last item in each object in the JSON shouldn't have a comma after it. See if that fixes your error
Related
I'm getting a feed from the web and I need to process it, however I'm getting an error and not sure how to process a list inside a list. I'm sure it's something simple I'm overlooking.
The JSON file is like this
{"alerts":[{"country":"AS","nThumbsUp":0,"city":"Albion, Vic","reportRating":3,"confidence":0,"reliability":5,"type":"JAM","uuid":"19c56810-3b8b-31a1-a658-c779f99b9388","magvar":279,"subtype":"JAM_STAND_STILL_TRAFFIC","street":"Polish Club Driveway","location":{"x":144.807815,"y":-37.771797},"pubMillis":1559688120073},{"country":"AS","nThumbsUp":0,"city":"Calder Park","reportRating":2,"confidence":0,"reliability":5,"type":"WEATHERHAZARD","uuid":"283a1bb4-6c0e-3f84-a4ff-cf187aa97dbd","roadType":2,"magvar":221,"subtype":"HAZARD_ON_SHOULDER_CAR_STOPPED","street":"Calder Park Dr","location":{"x":144.761619,"y":-37.679113},"pubMillis":1559689265092},
url = urllib.request.urlopen(turl)
output = url.read().decode('utf-8')
raw_api_dict = json.loads(output)
for x in json.loads(output)['alerts']:
print(x['country'])
print(x['nThumbsUp'])
print(x['reportRating'])
print(x['confidence'])
print(x['reliability'])
print(x['type'])
print(x['uuid'])
print(x['roadType'])
print(x['magvar'])
print(x['subtype'])
print(x['street'])
print(x['location_x'])
print(x['location_y'])
print(x['pubMillis'])
***This is the error **
Traceback (most recent call last):
File "waze.py", line 58, in
print(x['location_x'][0])
KeyError: 'location_x'
Its mainly because the JSON which you are using is invalid. Use the below JSON.
{
"country": "AS",
"nThumbsUp": 0,
"city": "Taylors Hill",
"reportRating": 1,
"confidence": 0,
"reliability": 5,
"type": "JAM",
"uuid": "a0241505-b0f8-3e83-a9c9-678f3c9039c5",
"roadType": 2,
"magvar": 103,
"subtype": "JAM_STAND_STILL_TRAFFIC",
"street": "Taylors Rd",
"location_x": 144.764866,
"location_y": -37.725576,
"pubMillis": 1559626611999 }
Python compiler considers ' as a special character.
use JSON Validator to validate your JSON always before running in code. I hope my answer helps. See the above comment. I guess json.dumps() could help you in this case.
import json
person_dict = {'name': 'Bob',
'age': 12,
'children': None
}
person_json = json.dumps(person_dict)
# Output: {"name": "Bob", "age": 12, "children": null}
print(person_json)
person_dict = json.loads(person_json)
print( person_dict)
print(person_dict['age'])
use json.dumps to solve the problem if it works.
I got a JSON text which I should parse, but for some reason I can't parse it because it has another array inside. My JSON looks like that:
{
"statementId": "1",
"movements": [
{
"id": 65,
"date": "2019-02-05",
"number": 32,
"balance": -4.62,
"purpose": "1"
},
{
"id": 1,
"date": "2019-02-05",
"number": 22,
"balance": -3,
"purpose": "23"
},
{
"id": 32,
"date": "2019-02-05",
"number": 12,
"balance": -11,
"purpose": "2"
}
],
"startPointer": "1122",
"endPointer": "3333"
}
I am using JsonSlurper. I want to know if it is possible to catch all the data inside "movements", I have tried to use this script:
JsonSlurper slurper = new JsonSlurper()
Map parsedJson = slurper.parseText(bodyContent)
String parsed_movements = parsedJson["movements"]
I have no problem with parsing single strings, like statementId or startPointer, but when I try to parse movements with my script it gives me result as null. I have also tried parsedJson["movements"][0] to catch first movement but it also gives me an error.
I have found a lot of things about json parsers on internet and also on stackoverflow but nothing what I seek. I really don't think that it is a duplicate question.
EDIT: I tried for statement also to put each object in array like that:
def movements_array = []
for(def i = 0; i < parsedJson.movements.size(); i++) {
movements_array << parsedJson.movements[i].id
println(movements_array)
}
But it gives me an error: Cannot invoke method size() on null object, because parsedJson.movements is null.
When you do:
String parsed_movements = parsedJson["movements"]
You're sticking a map into a String, which isn't what you want.
Given the json in your question, you can just do
def movementIds = new JsonSlurper().parseText(bodyContents).movements.id
To get a list of [65, 1, 32]
If you're getting NPEs I assume the json isn't what you show in the question
What started as my personal initiative, ended up being a quiet interesting ( may I say, challenging to some degree) project. My company decided to phase out one product and replace it with new one, which instead of storing data in mdb files, uses JSON files. So I took the initiative to create a converter that will read already created mdb files and convert them into the new format JSON.
However, now I'm at wits-ends with this one:
I can read mdb files, run query to extract specific data.
By placing the targetobj inside the FOR LOOP, I managed to extract data for each row and fed into a dict(targetobj)
for val in rows:
targetobj={"connection_props": {"port": 7800, "service": "", "host": val.Hostname, "pwd": "", "username": ""},
"group_list": val.Groups, "cpu_core_cnt": 2, "target_name": "somename", "target_type": "somethingsamething",
"os": val.OS, "rule_list": [], "user_list": val.Users}
if I print targetobj to console, I can clearly get all extracted values for each row.
Now, my quest is to have the obtained results ( for each row), inserted into the main_dict under the key targets:[]. ( Please see sample of JSON file for illustration)
main_dict = {"changed_time": 0, "year": 0, "description": 'blahblahblah', 'targets':[RESULTS FROM TARGETOBJ SHOULD BE ADDED HERE],"enabled": False}
so for example my Json file should have structure such as:
{"changed_time":1234556,
"year":0,
"description":"blahblahblah",
"targets":[
{"group_list":["QA"],
"cpu_core_cnt":1,
"target_name":"NewTarget",
"os":"unix",
"target_type":"",
"rule_list":[],
"user_list":[""],"connection_props":"port":someport,"service":"","host":"host1","pwd":"","username":""}
},
{"group_list":[],
"cpu_core_cnt":2,
"target_name":"",
"os":"unix",
"target_type":"",
"rule_list":[],
"user_list":["Web2user"],
"connection_props":{"port":anotherport,"service":"","host":"host2","pwd":"","username":""}}
],
"enabled":false}
So far I've been tweaking here and there, to have the results written as intended, however each time,I'm getting only the last row values written.
ie.: putting the targetobj as a variable inside the targets:[]
{"changed_time": 0, "year": 0, "description": 'ConvertedConfigFile', 'targets':[targetobj],
I know I'm missing something, I just need to find what and where.
Any help would be highly appreciated.
thank you
Just create your main_dict first and append to it in your loop, i.e.:
main_dict = {"changed_time": 0,
"year": 0,
"description": "blahblahblah",
"targets": [], # a new list for the target objects
"enabled": False}
for val in rows:
main_dict["targets"].append({ # append this dict to the targets list of main_dict
"connection_props": {
"port": 7800,
"service": "",
"host": val.Hostname,
"pwd": "",
"username": ""},
"group_list": val.Groups,
"cpu_core_cnt": 2,
"target_name": "somename",
"target_type": "somethingsamething",
"os": val.OS,
"rule_list": [],
"user_list": val.Users
})
I have a JSON Object graph that looks like this (note there is only 1 array in the object called lineItems:
{
"salesOrderUid": 52,
"orderNumber": "1428002206349",
"billToCity": "Golden",
"billToFirstName": "Duke",
"billToLastName": "Developer",
"shipToStreetNumber": "12345",
"shipToUnitNumber": null,
"shipToZipCode": 80401,
"promoCode": "Test",
"lineItems": [
{
"salesOrderLineUid": 59,
"salesOrderUid": 52,
"extendedPrice": 50,
"itemQuantity": 10,
"itemPrice": 5,
"catalogItem": {
"catalogItemUid": 1,
"itemPrice": 5,
"catalog": {
"catalogUid": 1,
"validFrom": 1420095600000,
"validThrough": 1435644000000
},
"item": {
"itemUid": 1,
"productCategoryUid": 1,
"productDescription": "Product used for testing",
"productName": "Test"
}
},
"shipmentUid": null
}
]
}
I iterate over lineItems like so:
<tr ng-repeat="salesOrderLineItem in salesOrder.lineItems">
<td>{{salesOrderLineItem.catalogItem.catalog.productName}}</td>
<td>{{salesOrderLineItem.itemQuantity}}</td>
<td>{{salesOrderLineItem.itemPrice | currency}}</td>
<td>{{salesOrderLineItem.extendedPrice | currency}}</td>
</tr>
The "first level" properties are displayed just fine. (itemQuantity, itemPrice and extendedPrice) But nothing gets displayed for the nested property called catalogItem.catalog.productName.
The JSON object reflected above came directly out of the Developer Tools console so it's clear that the contents are there. And the catalogItem property is not an array so I should be able to chain object property references, shouldn't I?
I've seen many questions posted related to accessing nested JSON but they seem to all have nested arrays in the JSON...which is not the case here.
Thanks in advance
catalog does not contain a field called productName. Did you mean to use item instead of catalog?
Looks to me that 'productName' is nested under 'item', not 'catalog'.
Change the line to be
<td>{{salesOrderLineItem.catalogItem.item.productName}}</td>
And that should be what you are looking for?
I have a large JSON file that looks similar to the code below. Is there anyway I can iterate through each object, look for the field "element_type" (it is not present in all objects in the file if that matters) and extract or write each object with the same element type to a file? For example each user would end up in a file called user.json and each book in a file called book.json?
I thought about using javascript but to my knowledge js can't write to files, I also tried to do it using linux command line tools by removing all new lines, then inserting a new line after each "}," and then iterating through each line to find the element type and write it to a file. This worked for most of the data; however, where there were objects like the "problem_type" below, it inserted a new line in the middle of the data due to the nested json in the "times" element. I've run out of ideas at this point.
{
"data": [
{
"element_type": "user",
"first": "John",
"last": "Doe"
},
{
"element_type": "user",
"first": "Lucy",
"last": "Ball"
},
{
"element_type": "book",
"name": "someBook",
"barcode": "111111"
},
{
"element_type": "book",
"name": "bookTwo",
"barcode": "111111"
},
{
"element_type": "problem_type",
"name": "problem object",
"times": "[{\"start\": \"1230\", \"end\": \"1345\", \"day\": \"T\"}, {\"start\": \"1230\", \"end\": \"1345\", \"day\": \"R\"}]"
}
]
}
I would recommend Java for this purpose. It sounds like you're running on Linux so it should be a good fit.
You'll have no problems writing to files. And you can use a library like this - http://json-lib.sourceforge.net/ - to gain access to things like JSONArray and JSONObject. Which you can easily use to iterate through the data in your JSON request, and check what's in "element_type" and write to a file accordingly.