When I try to use "Get Value From Json" on a Json with specific JsonPath , I always have KeyError
even web simple example doesn't work for me...
When I try that code :
Library JSONLibrary
*** Test Cases ***
Example
${tmp} = Convert String To JSON {"foo": {"bar": [1,2,3]}}
Log to console tmp: ${tmp}
${values_test}= Get Value From Json ${tmp} $.foo.bar
Log to console values_test: ${values_test}
I always have this kind of Errors and log :
tmp: {'foo': {'bar': [1, 2, 3]}}
...
Resolving variable '${tmp['$.foo.bar']}' failed: KeyError: '$.foo.bar'
Can somebody Help me please ?
it is really basic example by the way and community always says that it works like that in comments..
My application generates the below mentioned mulogs, which is in a nested json logs. While am trying to parse it in kibana, the json parse fails. Below is my log sample,
2022-08-04T12:43:03.977Z {"tags":
{"caller":"sphe",
"job-id":"1",
"mulog/duration":"3180930",
"mulog/namespace":"tool.utilities.db",
"mulog/outcome":":ok",
"user-name":"Pol",
"type":":execute!",
"app-name":"kan",
"mulog/parent-trace":"_YznrMCc",
"user-id":"52-7d4128fb7cb7",
"sql":"SELECT data FROM kan.material_estimate_history WHERE job_id = '167aa1cc' ",
"result":"[]",
"within-tx":"false",
"mulog/root-trace":"S0yn8jclLsmNVyKpH",
"mulog/timestamp":"1659616983977",
"uri":"/api/kan/material-estimates/job/f14b167aa1cc",
"mulog/trace-id":"kI4grnAMe4bGmFc_aX",
"request-method":":get",
"mulog/event-name":":kan=.source.db.material-estimates/find-history-by-job-id"},
"localEndpoint":{"serviceName":"kan"},
"name":"kan.source.db.material-estimates/find-history-by-job-id",
"traceId":"112721d07ecc9be",
"duration":3180,"id":"c90a259a2",
"kind":"SERVER","timestamp":1659616983977000,
"parentId":"dd7368"}
I have a request which tend to upload a file, if a file with the same name already exists it throws a message that the file already exists. This can be considered as expected result and even though the error I would the test to pass as it is.
This is the code I am using:
Create Session mysession ${test_env}
&{headers} Create Dictionary Content-Type=application/json; charset=utf-8 Authorization=${token}
${json}= Catenate { "FileName": "File.txt", "Content": "PD94bWwg..", "UserId": "email.com" }
${value} Set Variable 2
${value} Convert To Integer ${value}
${json}= Evaluate json.loads('''${json}''') json
#Set To Dictionary ${json["FileName"]}
${json}= Evaluate json.dumps(${json}) json
${resp} POST url=${test_env}/api/nt data=${json} headers=${headers}
${log}= Log To Console ${resp.status_code} 400
Log To Console ${resp.content}
Status Should Be expected_status=any
The test stops at the POST request and does not want to read the expected_status=any and consider the test as pass.
I would appreciate any hints on how to make it pass.
Below code will verify the 400 error and will continue further execution
Run Keyword And Expect Error HTTPError: 400* POST url=${test_env}/api/nt data=${json} headers=${headers}
I am trying to parse a json payload sent via a POST request to a NGINX/Openresty location. To do so, I combined Openresty's content_by_lua_block with its cjson module like this:
# other locations above
location /test {
content_by_lua_block {
ngx.req.read_body()
local data_string = ngx.req.get_body_data()
local cjson = require "cjson.safe"
local json = cjson.decode(data_string)
local endpoint_name = json['endpoint']['name']
local payload = json['payload']
local source_address = json['source_address']
local submit_date = json['submit_date']
ngx.say('Parsed')
}
}
Parsing sample data containing all required fields works as expected. A correct JSON object could look like this:
{
"payload": "the payload here",
"submit_date": "2018-08-17 16:31:51",
},
"endpoint": {
"name": "name of the endpoint here"
},
"source_address": "source address here",
}
However, a user might POST a differently formatted JSON object to the location. Assume a simple JSON document like
{
"username": "JohnDoe",
"password": "password123"
}
not containing the desired fields/keys.
According to the cjson module docs, using cjson (without its safe mode) will raise an error if invalid data is encountered. To prevent any errors being raised, I decided to use its safe mode by importing cjson.safe. This should return nil for invalid data and provide the error message instead of raising the error:
The cjson module will throw an error during JSON conversion if any invalid data is encountered. [...]
The cjson.safe module behaves identically to the cjson module, except when errors are encountered during JSON conversion. On error, the cjson_safe.encode and cjson_safe.decode functions will return nil followed by the error message.
However, I do not encounter any different error handling behavior in my case and the following traceback is shown in Openresty's error.log file:
2021/04/30 20:33:16 [error] 6176#6176: *176 lua entry thread aborted: runtime error: content_by_lua(samplesite:50):16: attempt to index field 'endpoint' (a nil value)
Which in turn results in an Internal Server Error:
<html>
<head><title>500 Internal Server Error</title></head>
<body>
<center><h1>500 Internal Server Error</h1></center>
<hr><center>openresty</center>
</body>
</html>
I think a workaround might be writing a dedicated function for parsing the JSON data and calling it with pcall() to catch any errors. However, this would make the safe mode kind of useless. What am I missing here?
Your “simple JSON document” is a valid JSON document. The error you are facing is not related to cjson, it's a standard Lua error:
resty -e 'local t = {foo = 1}; print(t["foo"]); print(t["foo"]["bar"])'
1
ERROR: (command line -e):1: attempt to index field 'foo' (a number value)
stack traceback:
...
“Safeness” of cjson.safe is about parsing of malformed documents:
cjson module raises an error:
resty -e 'print(require("cjson").decode("[1, 2, 3"))'
ERROR: (command line -e):1: Expected comma or array end but found T_END at character 9
stack traceback:
...
cjson.safe returns nil and an error message:
resty -e 'print(require("cjson.safe").decode("[1, 2, 3"))'
nilExpected comma or array end but found T_END at character 9
Trying to load the json file which is having null values in it by using elephant-bird JsonLoader.
sample.json
{"created_at": "Mon Aug 22 10:48:23 +0000 2016","id": 767674772662607873,"id_str": "767674772662607873","text": "KPIT Image Result for https:\/\/t.co\/Nas2ZnF1zZ... https:\/\/t.co\/9TnelwtIvm","source": "\u003ca href=\"http:\/\/twitter.com\" rel=\"nofollow\"\u003eTwitter Web Client\u003c\/a\u003e","truncated": false,"in_reply_to_status_id": 123,"in_reply_to_status_id_str": null,"in_reply_to_user_id": null,"in_reply_to_user_id_str": null,"in_reply_to_screen_name": null,"geo": null,"coordinates": null,"place": null,"contributors": null,"is_quote_status": false,"retweet_count": 0,"favorite_count": 0,"entities": {"hashtags": [],"urls": [{"url": "https:\/\/t.co\/Nas2ZnF1zZ","expanded_url": "http:\/\/miltonious.com\/","display_url": "miltonious.com","indices": [24, 47]}],"user_mentions": [],"symbols": []},"favorited": false,"retweeted": false,"possibly_sensitive": false,"filter_level": "low","lang": "en","timestamp_ms": "1471862903167"}
script:
REGISTER piggybank.jar
REGISTER json-simple-1.1.1.jar
REGISTER elephant-bird-pig-4.3.jar
REGISTER elephant-bird-core-4.1.jar
REGISTER elephant-bird-hadoop-compat-4.3.jar
json = LOAD 'sample.json' USING JsonLoader('created_at:chararray, id:chararray, id_str:chararray, text:chararray, source:chararray, in_reply_to_status_id:chararray, in_reply_to_status_id_str:chararray, in_reply_to_user_id:chararray, in_reply_to_user_id_str:chararray, in_reply_to_screen_name:chararray, geo:chararray, coordinates:chararray, place:chararray, contributors:chararray, is_quote_status:bytearray, retweet_count:long, favorite_count:chararray, entities:map[], favorited:bytearray, retweeted:bytearray, possibly_sensitive:bytearray, lang:chararray');
describe json;
dump json;
When I dump json,I am getting the following output and the worning
(Mon Aug 22 10:48:23 +0000 2016,767674772662607873,767674772662607873,google Image Result for Twitter Web Client,false,1234,12345,3214,43215,,,,,,,,,,,,,,)
WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - org.apache.pig.builtin.JsonLoader(UDF_WARNING_1): Bad record, returning null for {complete json}
By warning i guess it is getting NULL values.
So how can we load a Json which is having null values in it.
And I have tried in another way i.e
json = LOAD 'sample.json' USING com.twitter.elephantbird.pig.load.JsonLoader('created_at:chararray, id:chararray, id_str:chararray, text:chararray, source:chararray, in_reply_to_status_id:chararray, in_reply_to_status_id_str:chararray, in_reply_to_user_id:chararray, in_reply_to_user_id_str:chararray, in_reply_to_screen_name:chararray, geo:chararray, coordinates:chararray, place:chararray, contributors:chararray, is_quote_status:bytearray, retweet_count:long, favorite_count:chararray, entities:map[], favorited:bytearray, retweeted:bytearray, possibly_sensitive:bytearray, lang:chararray');
describe json;
Output
Schema for json unknown.
Please suggest me.
Thanks.
You can try something like this,
MY_JSON = LOAD 'sample.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad');
dump MY_JSON;