I'm acceessing an open JSON API like this
require 'net/http'
require 'rubygems'
require 'json'
require 'uri'
require 'pp'
url = "http://api.turfgame.com/v4/users"
uri = URI.parse(url)
data = [{"name" => "tbone"}]
headers = {"Content-Type" => "application/json"}
http = Net::HTTP.new(uri.host,uri.port)
response = http.post(uri.path,data.to_json,headers)
This gives a JSON ouput like this
[{"region"=>{"id"=>141, "name"=>"Stockholm"}, "medals"=>[34, 53, 12, 5, 46], "pointsPerHour"=>95, "blocktime"=>24, "zones"=>[275, 42460, 35956, 31926, 24247, 31722, 1097, 26104, 6072, 24283, 289, 325, 22199, 37740, 22198, 37743, 37074, 22845, 22201, 22846, 7477, 7310], "country"=>"se", "id"=>95195, "rank"=>24, "name"=>"tbone", "uniqueZonesTaken"=>178, "taken"=>1170, "points"=>41693, "place"=>377, "totalPoints"=>176654}]
What I want to do is to grab some of the tags:
name (not in the region block but "tbone")
blocktime
totalPoints
all the IDs from the zone-array
and insert into a mysql table. But I don't get how to iterate the JSON object and get the stuff I want.
doing
puts data["name"]
gives an error like
./headerTest.rb:28:in `[]': can't convert String into Integer (TypeError)
from ./headerTest.rb:28:in `<main>'
And I get that it's because there's two name tags but at different depth but i don't get how to accees either one specifically.
Please?
So you have:
result = [{"region"=>{"id"=>141, "name"=>"Stockholm"}, "medals"=>[34, 53, 12, 5, 46], "pointsPerHour"=>95, "blocktime"=>24, "zones"=>[275, 42460, 35956, 31926, 24247, 31722, 1097, 26104, 6072, 24283, 289, 325, 22199, 37740, 22198, 37743, 37074, 22845, 22201, 22846, 7477, 7310], "country"=>"se", "id"=>95195, "rank"=>24, "name"=>"tbone", "uniqueZonesTaken"=>178, "taken"=>1170, "points"=>41693, "place"=>377, "totalPoints"=>176654}]
This is an array with one value. To obtain those values you desire do:
result[0].slice("name", "blocktime", "totalPoints", "zones")
# this returns => {"name"=>"tbone", "blocktime"=>24, "totalPoints"=>176654, "zones"=>[275, 42460, 35956, 31926, 24247, 31722, 1097, 26104, 6072, 24283, 289, 325, 22199, 37740, 22198, 37743, 37074, 22845, 22201, 22846, 7477, 7310]}
Related
I have made a new table with three columns
customer_id,media_urls,survey_taste
in a db in pgadmin with attributes as
int,text[],jsonb
respectively.
I have a csv that I was trying to import into this table using pgadmin and
the contents of that file are like this
1,"{'http://example.com','http://example.com'}","{'taste':[1,2,3,4]}"
but I am getting this error
ERROR: invalid input syntax for type json
DETAIL: Token "'" is invalid.
CONTEXT: JSON data, line 1: '...
COPY survey_taste, line 2, column survey_taste: "{'taste': [-0.19101654669350904, 0.08575981750112513, 0.07133783942655376, -0.10579014363010293, 0.0..." ```
To address your comments in reverse order. To have this entered in one field you would need to have it as:
'[{"http":"abc","http":"abc"},{"taste":[1,2,3,4]}]'
Per:
select '[{"http":"abc","http":"abc"},{"taste":[1,2,3,4]}]'::json;
json
---------------------------------------------------
[{"http":"abc","http":"abc"},{"taste":[1,2,3,4]}]
As to the quoting issue:
When you pass a dict to csv you will get:
d = {"taste":[1,2,3,4]}
print(d)
{'taste': [1, 2, 3, 4]
What you need is:
import json
json.dumps(d)
'{"test": [1, 2, 3, 4]}'
Using json.dumps will turn the dict into a proper JSON string representation.
Putting it all together:
# Create list of dicts
l = [{'http': 'abc', 'http': 'abc'}, {'taste': [1,2,3,4]}]
# Create JSON string representattion
json.dumps(l)
'[{"http": "abc"}, {"taste": [1, 2, 3, 4]}]'
I have a json file that I am trying to pull specific attribute data from. The Json data is essentially a dictionary. Before the data is turned into a file, it is first held in a variable like this:
params = {'f': 'json', 'where': '1=1', 'geometryType': 'esriGeometryPolygon', 'spatialRel': 'esriSpatialRelIntersects','outFields': '*', 'returnGeometry': 'true'}
r = requests.get('https://hazards.fema.gov/gis/nfhl/rest/services/CSLF/Prelim_CSLF/MapServer/3/query', params)
cslfJson = r.json()
and then written into a file like this:
path = r"C:/Workspace/Sandbox/ScratchTests/cslf.json"
with open(path, 'w') as f:
json.dump(cslfJson, f, indent=2)
within this json data is an attribute called DFIRM_ID. I want to create an empty list called dfirm_id = [], get all of the values for DFIRM_ID and for that value, append it to the list like this dfirm_id.append(value). I am thinking I need to somehow read through the json variable data or the actual file, but I am not sure how to do it. Any suggestions on an easy method to accomplish this?
dfirm_id = []
for k, v in cslf:
if cslf[k] == 'DFIRM_ID':
dfirm.append(cslf[v])
As requested, here is what print(cslfJson) looks like:
It actually prints a huge dictionary that looks like this:
{'displayFieldName': 'CSLF_ID', 'fieldAliases': {'OBJECTID':
'OBJECTID', 'CSLF_ID': 'CSLF_ID', 'Area_SF': 'Area_SF', 'Pre_Zone':
'Pre_Zone', 'Pre_ZoneST': 'Pre_ZoneST', 'PRE_SRCCIT': 'PRE_SRCCIT',
'NEW_ZONE': 'NEW_ZONE', 'NEW_ZONEST': .... {'attributes': {'OBJECTID':
26, 'CSLF_ID': '13245C_26', 'Area_SF': 5.855231804165408e-05,
'Pre_Zone': 'X', 'Pre_ZoneST': '0.2 PCT ANNUAL CHANCE FLOOD HAZARD',
'PRE_SRCCIT': '13245C_STUDY1', 'NEW_ZONE': 'A', 'NEW_ZONEST': None,
'NEW_SRCCIT': '13245C_STUDY2', 'CHHACHG': 'None (Zero)', 'SFHACHG':
'Increase', 'FLDWYCHG': 'None (Zero)', 'NONSFHACHG': 'Decrease',
'STRUCTURES': None, 'POPULATION': None, 'HUC8_CODE': None, 'CASE_NO':
None, 'VERSION_ID': '2.3.3.3', 'SOURCE_CIT': '13245C_STUDY2', 'CID':
'13245C', 'Pre_BFE': -9999, 'Pre_BFE_LEN_UNIT': None, 'New_BFE':
-9999, 'New_BFE_LEN_UNIT': None, 'BFECHG': 'False', 'ZONECHG': 'True', 'ZONESTCHG': 'True', 'DFIRM_ID': '13245C', 'SHAPE_Length':
0.009178426056888393, 'SHAPE_Area': 4.711699932249018e-07, 'UID': 'f0125a91-2331-4318-9a50-d77d042a48c3'}}, {'attributes': .....}
If your json data is already a dictionary, then take advantage of that. The beauty of a dictionary / hashmap is that it provides an average time complexity of O(1).
Based on your comment, I believe this will solve your problem:
dfirm_id = []
for feature in cslf['features']:
dfirm_id.append(feature['attributes']['DFIRM_ID'])
I am trying to use rest client in ruby to parse json but I am stuck how to validate the response (extract values from the response). Also I am trying to validate the valid response code(200) response.code does not work.
Following is the JSON response and Code that uses rest client to get it:
def self.call_legacy_transactions
get_response=RestClient::Request.execute(method: :get, url: 'URL', timeout: 15)
puts(get_response.body)
json_response = JSON.parse(get_response)
//Dont know how to get the values from response hash. Please suggest
end
JSON Response:
[
{
"value": "12345678912345",
"events": [
{
"transaction_id": 205,
"package_name": "",
"event_codes": [
465,
469,
471,
474,
410,
490,
1040
]
},
{
"transaction_id": 204,
"package_name": "",
"event_codes": [
465,
469,
474,
490
]
},
{
"transaction_id": 207,
"package_name": "",
"event_codes": [
465,
469,
471,
474,
490
]
}
]
}
]
I want the event code for each transaction for each value.
If you just want a flat list of event code integers, you can use:
json_response.flat_map do |value_data|
value_data[:events].flat_map do |event_data|
event_data[:event_codes]
end
end
UPDATE based on the comment "I want to extract event codes only where transaction id is 205":
If there could only ever be one item with that transaction ID:
json_response.flat_map do |value_data|
value_data[:events].find do |event_data|
event_data[:transaction_id] == 205
end[:event_codes]
end
If there could be many items with that transaction ID:
json_response.flat_map do |value_data|
value_data[:events].select do |event_data|
event_data[:transaction_id] == 205
end.flat_map do |event_data|
event_data[:event_codes]
end
end
You can call methods on the response to see the body, the response code, etc. More info in the README
Add to your code:
def self.call_legacy_transactions(tx_id = 205)
get_response=RestClient::Request.execute(method: :get, url: 'URL', timeout: 15)
puts(get_response.body)
# check if the response was successful
if get_response.code == 200
# need to parse the body
json_response = JSON.parse(get_response.body)
# the json response is an array of json objects
# we need to iterate over them and grab the value
# in the `events` key and iterate over those and
# select just the ones with the desired transaction id and
# get the value in each of the `event_codes` keys and
# then flatten all the sub arrays into one
event_codes = json_response.flat_map do |data|
data['events'].
select { |event| event['transaction_id'] == tx_id }.
flat_map { |event| event['event_codes'] }
end
event_codes # is now a list of just the event codes
end
end
With the above you can pass the transaction id into the method to get the event codes for any transaction e.g.
call_legacy_transactions 205
the python program
http://sourceforge.net/projects/cppheaderparser/
can parse a c++ header file and store the info (about classes etc) in a python dictionary.
Using the included example program readSampleClass.py and
data_string = ( repr(cppHeader) )
with open('data.txt', 'w') as outfile:
json.dumps(data_string,outfile)
it saved the output but it is not valid json as
it uses single, not double quotes and key part is not quoted.
sample of output: (reduced)
{'enums': [], 'variables': [], 'classes':
{'SampleClass':
{'inherits': [], 'line_number': 8, 'declaration_method': 'class', 'typedefs':
{'public': [], 'private': [], 'protected': []
}, 'abstract': False, 'parent': None,'parent': None, 'reference': 0, 'constant': 0, 'aliases': [], 'raw_type': 'void', 'typedef': None, 'mutable': False
}], 'virtual': False, 'rtnType': 'int', 'returns_class': False, 'name': 'anotherFreeFunction', 'constructor': False, 'inline': False, 'returns_pointer': 0, 'defined': False
}]
}
so the question is:
How can I make it use double quotes and not single and how can I also make it quote the value part. Like False in sample.
I assume is possible as the creator of cppheaderparser wrote
about json.dumps(repr(cppHeader))
https://twitter.com/senexcanis/status/559444754166198272
Why use the json lib if its not valid jason?
That said I have never used python before and it might just not work as i think.
-update-
After some json doc reading, i gave up on json.dump as it seems to do nothing to the output in this case.
I ended up doing
data_string = ( repr(cppHeader) )
data_string = string.replace(data_string,'\'', '\"')
data_string = string.replace(data_string,'False', '\"False\"')
data_string = string.replace(data_string,'True', '\"True\"')
data_string = string.replace(data_string,'None', '\"None\"')
data_string = string.replace(data_string,'...', '')
with open('data.txt', 'w') as outfile:
outfile.write (data_string)
which give valid json - at least for my test c++ headers.
-update 2-
The creator of cppheaderparse just released a new 2.6 version where its possible to write CppHeaderParser.CppHeader("yourHeader.h").toJSON() to save as json.
I am trying to create JSON data to pass to InfluxDB. I create it using strings but I get errors. What am I doing wrong. I am using json.dumps as has been suggested in various posts.
Here is basic Python code:
json_body = "[{'points':["
json_body += "['appx', 1, 10, 0]"
json_body += "], 'name': 'WS1', 'columns': ['RName', 'RIn', 'SIn', 'OIn']}]"
print("Write points: {0}".format(json_body))
client.write_points(json.dumps(json_body))
The output I get is
Write points: [{'points':[['appx', 1, 10, 0]], 'name': 'WS1', 'columns': ['RName', 'RIn', 'SIn', 'OIn']}]
Traceback (most recent call last):
line 127, in main
client.write_points(json.dumps(json_body))
File "/usr/local/lib/python2.7/dist-packages/influxdb/client.py", line 173, in write_points
return self.write_points_with_precision(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/influxdb/client.py", line 197, in write_points_with_precision
status_code=200
File "/usr/local/lib/python2.7/dist-packages/influxdb/client.py", line 127, in request
raise error
influxdb.client.InfluxDBClientError
I have tried with double quotes too but get the same error. This is stub code (to minimize the solution), I realize in the example the points list contains just one list object but in reality it contains multiple. I am generating the JSON code reading through outputs of various API calls.
json_body = '[{\"points\":['
json_body += '[\"appx\", 1, 10, 0]'
json_body += '], \"name\": \"WS1\", \"columns\": [\"RName\", \"RIn\", \"SIn\", \"OIn\"]}]'
print("Write points: {0}".format(json_body))
client.write_points(json.dumps(json_body))
I understand if I used the below things would work:
json_body = [{ "points": [["appx", 1, 10, 0]], "name": "WS1", "columns": ["Rname", "RIn", "SIn", "OIn"]}]
You don't need to create JSON manually. Just pass an appropriate Python structure into write_points function. Try something like that:
data = [{'points':[['appx', 1, 10, 0]],
'name': 'WS1',
'columns': ['RName', 'RIn', 'SIn', 'OIn']}]
client.write_points(data)
Please visit JSON.org for proper JSON structure. I can see several errors with your self-generated JSON:
The outer-most item can be an unordered object, enclosed by curly braces {}, or an ordered array, enclosed by brackets []. Don't use both. Since your data is structured like a dict, the curly braces are appropriate.
All strings need to be enclosed in double quotes, not single. "This is valid JSON". 'This is not valid'.
Your 'points' value array is surrounded by double brackets, which is unnecessary. Only use a single set.
Please check out the documentation of the json module for details on how to use it. Basically, you can feed json.dumps() your Python data structure, and it will output it as valid JSON.
In [1]: my_data = {'points': ["appx", 1, 10, 0], 'name': "WS1", 'columns': ["RName", "RIn", "SIn", "OIn"]}
In [2]: my_data
Out[2]: {'points': ['appx', 1, 10, 0], 'name': 'WS1', 'columns': ['RName', 'RIn', 'SIn', 'OIn']}
In [3]: import json
In [4]: json.dumps(my_data)
Out[4]: '{"points": ["appx", 1, 10, 0], "name": "WS1", "columns": ["RName", "RIn", "SIn", "OIn"]}'
You'll notice the value of using a Python data structure first: because it's Python, you don't need to worry about single vs. double quotes, json.dumps() will automatically convert them. However, building a string with embedded single quotes leads to this:
In [5]: op_json = "[{'points':[['appx', 1, 10, 0]], 'name': 'WS1', 'columns': ['RName', 'RIn', 'SIn', 'OIn']}]"
In [6]: json.dumps(op_json)
Out[6]: '"[{\'points\':[[\'appx\', 1, 10, 0]], \'name\': \'WS1\', \'columns\': [\'RName\', \'RIn\', \'SIn\', \'OIn\']}]"'
since you fed the string to json.dumps(), not the data structure.
So next time, don't attempt to build JSON yourself, rely on the dedicated module to do it.