I have a WS using Flask/python 2.7. I have 1 JSON object passed to the WS. I have been successful in capturing the object and returning the whole JSON.
I have looked all over for examples (many use print of test dataset in python) and have tried json.dumps, json.loads, json.dump, json.load, for loops, etc.
What I would like to do seems simple and I know it is me, but I get errors no matter what I try. I am trying to parse the JSON, put the values in to variables, and do "stuff".
This works:
#app.route('/v1/test', methods = ['POST'])
def api_message():
if request.headers['Content-Type'] == 'application/json':
return "JSON Message: " + json.dumps(request.json, separators=(',',':'))
else:
return "415 Unsupported Media Type"
This does not (and many variations of this using different things):
jsonobject = json.dumps(request.json)
pstring = json.loads(jsonobject)
for key, value in pstring.iteritems():
return value
What I want to do (pseudo code):
for each JSON
get the name value pairs in to a place where I can do something like this (which was done on a flat file)
input_data = pd.read_csv(sio, delimiter=',', names=columns)
probs = model.predict_proba(input_data)
I am sure I didn't make this as clear as I could but it is a challenge because I get errors like below (examples -- not all at once of course) with all the different things I try:
AttributeError: 'dict' object has no attribute 'translate'
TypeError: 'dict' object is not callable
AttributeError: 'str' object has no attribute 'iteritems'
So after all that, what is the right way to do this?
Related
I get the response of a server:
r = requests.get(my_url)
print(r.text)
json_response = json.loads(r.text)
print(type(json_response))
I am using json.loads() to convert the string into json. However, the response has this format:
[{"element_info":{"data":201863539001,......]
I want to access element_info.data. I tried accessing it with json_response.element_info.data but i got AttributeError: 'list' object has no attribute 'bkdn_elem_info'.
I also tried content = r.read() and i got AttributeError: 'Response' object has no attribute 'read'.
You cannot access the elements of a dictionary in python with '.' operator as in javascript. You have to use the square bracket notation as below to access the elements
json_response[0]["element_info"]["data"]
The type of json_response is list. So, you can iterate on it and access each element in it. Each element is a dictionary. So, you can use the keys:
for each_element in json_response:
print(each_element["element_info"]["data"])
If you are not sure that these keys are available in all elements, you can use the get() method with a default value to avoid errors when the key does not exist. Here, the first operand is the key that you want to read, and the second operand can be used to define a default value:
for each_element in json_response:
print(each_element.get("element_info", {}).get("data", ""))
In access to data from a list, you must specify the element with index first and then get the property of the object
objectOfList =json_response[0];
value = objectOfList["element_info"]["data"];
I am trying to get one object returned from a list of objects;
list = [{"name":"Joe","id":1},{"name":"Fred","id":2}]
I want to output the object exactly like the format below;
{"name":"Joe","id":1}
It throws errors when I want to access the object in its entirety;
${list?first} -> Error: Expected a string or something automatically...
Testing this on https://try.freemarker.apache.org/ but I can't seem to be able to extract the full first object, whilst accessing a property from that object works fine..?
I understand that this is somewhat of an odd use case to display the object like that. I need it for a business program that uses freemarker and I want to assign the object back into a variable, which accepts this format.
You can show values that can be converted to String as error (below) suggest
<#assign item = list[0]>
${item.name} ${item.id}
For "${...}" content: Expected a string or something automatically convertible to
string (number, date or boolean), or "template output" , but this has evaluated to a
sequence (ArrayList wrapped into f.t.DefaultListAdapter):
==> list [in nameless template at line 1, column 3]
You can check freemarker parse a JSON answers, for executing similar as:
<#assign array = '[{"name":"Joe","id":1},{"name":"Fred","id":2}]'>
<#assign object = array?eval[0]>
<#list object?keys as key>"${key}":"${object[key]}"
</#list>
A small question: i'm using JsonConverter from Github.
(https://github.com/VBA-tools/VBA-JSON/blob/master/JsonConverter.bas)
Code is working on most of the "GET" request.
Only when the same 'column' is accuring multiple times in the 'ResponseText' then it's not. (like "imei" in the example)
So I need a way to handle a long 'Responsetext' to fill multiple rows in an access db.
Dim Json As Object
Set Json = JsonConverter.ParseJson(xmlhttp.ResponseText)
MsgBox (Json("imei")) 'temp
Error 5: Invalid Procedure or Call Argument.
Any ideas?
Many thanks,
I can only provide a part of the answer, because I cannot recreate it, without the full JSON Response Text:
The response with mutliple values in JSON returns an Object of the type Collection. Hence you have to use a loop to iterate through all responses. Like this:
Dim Json As Object
Set Json = JsonConverter.ParseJson(xmlhttp.ResponseText)
For Each singleJsonItem In Json
'What object type is singleJsonItem? To find out, maybe use:
'MsgBox singleJsonItem("imei")
Next singleJsonItem
You have to find out the object type of the collectionentries to extract the JSON Entry.
Solved like this:
used : https://github.com/VBA-tools/VBA-JSON
and named the module: 'mdl_JsonConverter'
Set Json = mdl_JsonConverter.ParseJson(xmlhttp.ResponseText)
For Each item In Json
input_1 = item("input_1")
input_2 = item("input_2")
'THEN DO SOMETHING WITH VALUES F.E. ADDING THEM IN A TABLE
Next
If you responseText contains a searchRecords list then filter by searchRecords before for each
Set Json = JsonConverter.ParseJson(strResponse)
For Each singleJsonItem In Json("searchRecords")
'What object type is singleJsonItem? To find out, maybe use:
Msgbox singleJsonItem("Name")
Next singleJsonItem
This question is multipart-
(1a) JSON is fundamental to JavaScript, so why is there no JSON type? A JSON type would be a string that is formatted as JSON. It would be marked as parsed/stringified until the data was altered. As soon as the data was altered it would not be marked as JSON and would need to be re-parsed/re-stringified.
(1b) In some software systems, isn't it possible to (accidentally) attempt to send a plain JS object over the network instead of a serialized JS object? Why not make an attempt to avoid that?
(1c) Why can't we call JSON.parse on a straight up JavaScript object without stringifying it first?
var json = { //JS object in properJSON format
"baz":{
"1":1,
"2":true,
"3":{}
}
};
var json0 = JSON.parse(json); //will throw a parse error...bad...it should not throw an error if json var is actually proper JSON.
So we have no choice but to do this:
var json0= JSON.parse(JSON.stringify(json));
However, there are some inconsistencies, for example:
JSON.parse(true); //works
JSON.parse(null); //works
JSON.parse({}); //throws error
(2) If we keep calling JSON.parse on the same object, eventually it will throw an error. For example:
var json = { //same object as above
"baz":{
"1":1,
"2":true,
"3":{}
}
};
var json1 = JSON.parse(JSON.stringify(json));
var json2 = JSON.parse(json1); //throws an error...why
(3) Why does JSON.stringify infinitely add more and more slashes to the input? It is not only hard to read the result for debugging, but it actually puts you in dangerous state because one JSON.parse call won't give you back a plain JS object, you have to call JSON.parse several times to get back the plain JS object. This is bad and means it is quite dangerous to call JSON.stringify more than once on a given JS object.
var json = {
"baz":{
"1":1,
"2":true,
"3":{}
}
};
var json2 = JSON.stringify(json);
console.log(json2);
var json3 = JSON.stringify(json2);
console.log(json3);
var json4 = JSON.stringify(json3);
console.log(json4);
var json5 = JSON.stringify(json4);
console.log(json5);
(4) What is the name for a function that we should be able to call over and over without changing the result (IMO how JSON.parse and JSON.stringify should behave)? The best term for this seems to be "idempotent" as you can see in the comments.
(5) Considering JSON is a serialization format that can be used for networked objects, it seems totally insane that you can't call JSON.parse or JSON.stringify twice or even once in some cases without incurring some problems. Why is this the case?
If you are someone who is inventing the next serialization format for Java, JavaScript or whatever language, please consider this problem.
IMO there should be two states for a given object. A serialized state and a deserialized state. In software languages with stronger type systems, this isn't usually a problem. But with JSON in JavaScript, if call JSON.parse twice on the same object, we run into fatal exceptions. Likewise, if we call JSON.stringify twice on the same object, we can get into an unrecoverable state. Like I said there should be two states and two states only, plain JS object and serialized JS object.
1) JSON.parse expects a string, you are feeding it a Javascript object.
2) Similar issue to the first one. You feed a string to a function that needs an object.
3) Stringfy actually expects a string, but you are feeding it a String object. Therefore, it applies the same measures to escape the quotes and slashes as it would for the first string. So that the language can understand the quotes, other special characters inside the string.
4) You can write your own function for this.
5) Because you are trying to do a conversion that is illegal. This is related to the first and second question. As long as the correct object types are fed, you can call it as many times as you want. The only problem is the extra slashes but it is in fact the standard.
We'll start with this nightmare of your creation: string input and integer output.
IJSON.parse(IJSON.stringify("5")); //=> 5
The built-in JSON functions would not fail us this way: string input and string output.
JSON.parse(JSON.stringify("5")); //=> "5"
JSON must preserve your original data types
Think of JSON.stringify as a function that wraps your data up in a box, and JSON.parse as the function that takes it out of a box.
Consider the following:
var a = JSON.stringify;
var b = JSON.parse;
var data = "whatever";
b(a(data)) === data; // true
b(b(a(a(data)))) === data; // true
b(b(b(a(a(a(data)))))) === data; // true
That is, if we put the data in 3 boxes, we have to take it out of 3 boxes. Right?
If I put my data in 2 boxes and take it out of 1, I'm not holding my data yet, I'm holding a box that contains my data. Right?
b(a(a(data))) === data; // false
Seems sane to me...
JSON.parse unboxes your data. If it is not boxed, it cannot unbox it. JSON.parse expects a string input and you're giving it a JavaScript object literal
The first valid call to JSON.parse would return an object. Calling JSON.parse again on this object output would result in the same failure as #1
repeated calls to JSON.stringify will "box" our data multiple times. So of course you have to use repeated calls to JSON.parse then to get your data out of each "box"
Idempotence
No, this is perfectly sane. You can't triple-stamp a double-stamp.
You'd never make a mistake like this, would you?
var json = IJSON.stringify("hi");
IJSON.parse(json);
//=> "hi"
OK, that's idempotent, but what about
var json = IJSON.stringify("5");
IJSON.parse(json);
//=> 5
UH OH! We gave it a string each time, but the second example returns an integer. The input data type has been lost!
Would the JSON functions have failed us here?
var json = JSON.stringify("hi");
JSON.parse(json);
//=> "hi"
All good. And what about the "5" ?
var json = JSON.stringify("5");
JSON.parse(json));
//=> "5"
Yay, the types have been preseved! JSON works, IJSON does not.
Maybe a more real-life example:
OK, so you have a busy app with a lot of developers working on it. It makes
reckless assumptions about the types of your underlying data. Let's say it's a chat app that makes several transformations on messages as they move from point to point.
Along the way you'll have:
IJSON.stringify
data moves across a network
IJSON.parse
Another IJSON.parse because who cares? It's idempotent, right?
String.prototype.toUpperCase — because this is a formatting choice
Let's see the messages
bob: 'hi'
// 1) '"hi"', 2) <network>, 3) "hi", 4) "hi", 5) "HI"
Bob's message looks fine. Let's see Alice's.
alice: '5'
// 1) '5'
// 2) <network>
// 3) 5
// 4) 5
// 5) Uncaught TypeError: message.toUpperCase is not a function
Oh no! The server just crashed. You'll notice it's not even the repeated calling of IJSON.parse that failed here. It would've failed even if you called it once.
Seems like you were doomed from the start... Damned reckless devs and their careless data handling!
It would fail if Alice used any input that happened to also be valid JSON
alice: '{"lol":"pwnd"}'
// 1) '{"lol":"pwnd"}'
// 2) <network>
// 3) {lol:"pwnd"}
// 4) {lol:"pwnd"}
// 5) Uncaught TypeError: message.toUpperCase is not a function
OK, unfair example maybe, right? You're thinking, "I'm not that reckless, I
wouldn't call IJSON.stringify or IJSON.parse on user input like that!"
It doesn't matter. You've fundamentally broken JSON because the original
types can no longer be extracted.
If I box up a string using IJSON, and then unbox it, who knows what I will get back? Certainly not you, and certainly not the developer using your reckless function.
"Will I get a string type back?"
"Will I get an integer?"
"Maybe I'll get an object?"
"Maybe I will get cake. I hope it's cake"
It's impossible to tell!
You're in a whole new world of pain because you've been careless with your data types from the start. Your types are important so start handling them with care.
JSON.stringify expects an object type and JSON.parse expects a string type.
Now do you see the light?
I'll try to give you one reason why JSON.parse cannot be called multiple time on the same data without us having a problem.
you might not know it but a JSON document does not have to be an object.
this is a valid JSON document:
"some text"
lets store the representation of this document inside a javascript variable:
var JSONDocumentAsString = '"some text"';
and work on it:
var JSONdocument = JSON.parse(JSONDocumentAsString);
JSONdocument === 'some text';
this will cause an error because this string is not the representation of a JSON document
JSON.parse(JSONdocument);
// SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
in this case how could have JSON.parse guessed that JSONdocument (being a string) was a JSON document and that it should have returned it untouched ?
I ma new to writing python unit tests. I have a method in a class returning a Json response from an API. The JSON response contains attributes such as data, token, object name and status. The method hits API and returns response with different values each time, so I can't hard code a response and test it. How can I write a unit test for this type of method.
One thing, I thought of is to check whether the response is not null. Is there any other type of checks I can do here.
Each time it returns a different token, date(timestamp). The status will be same.
def Json_get_status(self):
get_url = "xxxx" #URL to hit API
r = requests.get(get_url)
self.get_json = json.loads(r.text)
self.get_token=self.get_json["token"]
self.get_date=self.get_json["date"]
self.get_status=self.get_json["status"]
return self.get_json
If your method under test is supposed to "read the status correctly", then you might want to specifically test that.
So assuming your app is something like
def read_status(response):
parsed = json.loads(response)
# does other stuff
return something
Then in your test_unit.py
def test_read_status(self):
mock_input_val = {'something': 'good val'}
expected_positive_return_val = something
self.assertEqual(read_status(json.dumps(mock_input_val)),
expected_positive_return_val)
Also good to do a negative test for the condition where read_status either fails to parse the json object or finds an error in the response.