I am calling a R script from Scala by using scala.sys.process. This script takes a command line argument in JSON format and processes it. I am using rjson's fromJSON function to store the JSON in a list.
This works very fine when I execute the R script from the command line:
$ ./dfChargerFlink.R '{"Id":"1","value":"ABC"}'
But when I call it from scala, I get the following error:
Error in fromJSON(commandArgs(1)) : unexpected character '''
Execution halted
This is the code I am using:
val shellCommand = "./dfChargerFlink.R '"+arg+"'"
return shellCommand !!
where arg is the JSON string.
You can notice that I have appended " ' " to both the sides of the JSON string as if I don't, I get this error:
Error in fromJSON(commandArgs(1)) : unclosed string
Execution halted
How can this be solved? Is it some bug?
Related
I am trying to parse a json payload sent via a POST request to a NGINX/Openresty location. To do so, I combined Openresty's content_by_lua_block with its cjson module like this:
# other locations above
location /test {
content_by_lua_block {
ngx.req.read_body()
local data_string = ngx.req.get_body_data()
local cjson = require "cjson.safe"
local json = cjson.decode(data_string)
local endpoint_name = json['endpoint']['name']
local payload = json['payload']
local source_address = json['source_address']
local submit_date = json['submit_date']
ngx.say('Parsed')
}
}
Parsing sample data containing all required fields works as expected. A correct JSON object could look like this:
{
"payload": "the payload here",
"submit_date": "2018-08-17 16:31:51",
},
"endpoint": {
"name": "name of the endpoint here"
},
"source_address": "source address here",
}
However, a user might POST a differently formatted JSON object to the location. Assume a simple JSON document like
{
"username": "JohnDoe",
"password": "password123"
}
not containing the desired fields/keys.
According to the cjson module docs, using cjson (without its safe mode) will raise an error if invalid data is encountered. To prevent any errors being raised, I decided to use its safe mode by importing cjson.safe. This should return nil for invalid data and provide the error message instead of raising the error:
The cjson module will throw an error during JSON conversion if any invalid data is encountered. [...]
The cjson.safe module behaves identically to the cjson module, except when errors are encountered during JSON conversion. On error, the cjson_safe.encode and cjson_safe.decode functions will return nil followed by the error message.
However, I do not encounter any different error handling behavior in my case and the following traceback is shown in Openresty's error.log file:
2021/04/30 20:33:16 [error] 6176#6176: *176 lua entry thread aborted: runtime error: content_by_lua(samplesite:50):16: attempt to index field 'endpoint' (a nil value)
Which in turn results in an Internal Server Error:
<html>
<head><title>500 Internal Server Error</title></head>
<body>
<center><h1>500 Internal Server Error</h1></center>
<hr><center>openresty</center>
</body>
</html>
I think a workaround might be writing a dedicated function for parsing the JSON data and calling it with pcall() to catch any errors. However, this would make the safe mode kind of useless. What am I missing here?
Your “simple JSON document” is a valid JSON document. The error you are facing is not related to cjson, it's a standard Lua error:
resty -e 'local t = {foo = 1}; print(t["foo"]); print(t["foo"]["bar"])'
1
ERROR: (command line -e):1: attempt to index field 'foo' (a number value)
stack traceback:
...
“Safeness” of cjson.safe is about parsing of malformed documents:
cjson module raises an error:
resty -e 'print(require("cjson").decode("[1, 2, 3"))'
ERROR: (command line -e):1: Expected comma or array end but found T_END at character 9
stack traceback:
...
cjson.safe returns nil and an error message:
resty -e 'print(require("cjson.safe").decode("[1, 2, 3"))'
nilExpected comma or array end but found T_END at character 9
I am trying to retrieve an xml file from my Mongo DB with mongofiles. I get a JSON parsing error. Here is an excerpt my terminal:
$ mongofiles -d anhalytics get_id 'ObjectId("5e7f56d30800611b17fc66b1")'
2020-09-15T16:55:33.205+0200 connected to: mongodb://localhost/
2020-09-15T16:55:33.205+0200 Failed: error parsing id as Extended JSON: invalid JSON number. Position: 18
I am using a MongoDB server version: 4.2.9
Here is the record of the target file
{
"_id" : ObjectId("5e7f56d30800611b17fc66b1"),
"filename" : "5e7f56d30800611b17fc66b0.tei.xml",
"aliases" : null,
"chunkSize" : NumberLong(261120),
"uploadDate" : ISODate("2020-03-28T13:53:23.708Z"),
"length" : NumberLong(35405),
"contentType" : null,
"md5" : "eeafae907c44b207071ccb6036148808"
}
Any idea why I am getting this error? Thanks!
The message error parsing id as Extended JSON indicates that the mongofiles tool had trouble parsing the id string that was provided on the command line.
That is done in parseOrCreateId function here: https://github.com/mongodb/mongo-tools/blob/master/mongofiles/mongofiles.go#L330
That function wraps the value from the command line in another string like {"_id":"%s"}, so the value actually passed to the bson.UnmarshalExtJSON function would have been
"{\"_id\":\"ObjectId(\"5e7f56d30800611b17fc66b1\")\"}"
Position 18 of that string, as called out in the error message is the quotation mark immediately preceding the hex string.
I am making use of non-static import
JsonPath jp = response.jsonPath();
System.out.println(jp.get("data?(#.id>14).employee_name").toString());
For a JSON as shown below:
{"status":"success","data":[{"id":"1","employee_name":"Tiger Nixon","employee_salary":"320800","employee_age":"61","profile_image":""},{"id":"2","employee_name":"Garrett Winters","employee_salary":"170750","employee_age":"63","profile_image":""}]}
When i am trying to run it , i am getting below error:
java.lang.IllegalArgumentException: Invalid JSON expression:
Script1.groovy: 1: expecting EOF, found '[' # line 1, column 31.
data[?(#.id>14)].employee_name
^
1 error
Can someone guide me why is this error being thrown ?
I doubt if that syntax is right, nonetheless, you should be using the below
Also note that the id is a string in your response so you will have to include it in quotes
js.get("data.find {it.id > '14'}.employee_name").toString();
I can't seem to catch an exception when using json.loads even though I specifically call it out. I largely see this when trying to stress my server with lots of client connection sending data very quickly. See my error below:
(I've replaced my IP address with X's in the error code)
EX: Unterminated string starting at: line 1 column 49 (char 48) Data:
'{"ap-hdop":0.55,"rtcmin":"38","ap-latdec":3.134,"a' error: uncaptured
python exception, closing channel
(:Unterminated string
starting at: line 1 column 49 (char 48)
[//faraday_server_handler.py|collect_incoming_data|34]
[/usr/lib/python2.7/json/init.py|loads|338]
[/usr/lib/python2.7/json/decoder.py|decode|366]
[/usr/lib/python2.7/json/decoder.py|raw_decode|382])
I understand this that the code fails because I simply miss a double quotes on the line:
'{"ap-hdop":0.55,"rtcmin":"38","ap-latdec":3.134,"a'
This line is usually a LOT longer so that "a.... was supposed to keep going and complete it's quotes.
Here's my relevant code:
def collect_incoming_data(self, data):
"""Read an incoming message from the client, place JSON message data into buffer"""
#self.logger.debug('collect_fing_data() -> (%d bytes)\n"""%s"""', len(data), data)
try:
loaded_data = json.loads(data)
except ValueError, ex:
self.handle_error()
type,value,traceback = sys.exc_info()
print type
#print "Collect Incoming Data: " . time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime())
print "EX: ", ex
print "Data: ",repr(data)
Any ideas? I scoured the internet to see if I can find this issue, but I appear to be setting up to capture the exception which everyone else having this issue with loads seems to suggest to do.
EDIT 3/1/2016 - Evening
Commenting out my exception handle_error() let me see more of the error:
except ValueError, ex:
self.handle_error()
type,value,traceback = sys.exc_info()
print type
#print "Collect Incoming Data: " . time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime())
print "EX: ", ex
print "Data: ",repr(data)
Below is my new error, I've commented out personal data. It's apparent that the issue I really have now is in-fact the unterminated string
EX: Unterminated string starting at:
line 1 column 49 (char 48) Data:
'{"ap-hdop":0.55,"rtcmin":"31","ap-latdec":XX.XXX,"a' EX: No JSON object could be decoded Data:
'p-latdeg":34,"adc6":2006,"adc7":2007,"adc4":2004,"adc5":2005,"adc2":2002,"adc3":2003,"adc0":2000,"adc1":2001,"gpio-0":30,"gpio-1":50,"gpio-2":70,"speed":5.0,"adc8":2008,"rtcday":"01","longdeg":118,"longdec":XX.XXX,"altitude":31.0,"ap-speed":0.0,"ap-pdop":0.77,"lat-dir":"N","long-dir":"W","hdop":0.01,"ap-rf":0,"alt-units":"M","rtcdow":"2","callsign":"XXXXX","ap-callsign":"XXXXX","id":1,"ap-id":1,"rtcyear":"2016","rtcmon":"03","ap-vdop":0.66,"ap-lat-dir":"N","vdop":0.02,"rtchour":"22","latdec":XX.XXX,"latdeg":34,"ap-longdeg":118,"ap-longdec":XX.XXX,"rtcsec":"15","ap-altitude":86.0,"ap-long-dir":"W","pdop":0.01,"ap-alt-units":"M","faraday-port":0}'
OK my original question was answered which was "Why am I not catching the ValueError exception even though I am providing code to do just that?"
#tadhg McDonald-jensen was correct with his comment to remove my call to handle_error().
I still have some other issues but they are a different question. Thanks!
I am using the R package rjson to download weather data from Wunderground.com. Often I leave the program to run and there are no problems, with the data being collected fine. However, often the program stops running and I get the following error message:
Error in fromJSON(paste(raw.data, collapse = "")) : unclosed string
In addition: Warning message:
In readLines(conn, n = -1L, ok = TRUE) :
incomplete final line found on 'http://api.wunderground.com/api/[my_API_code]/history_20121214pws:1/q/pws:IBIRMING7.json'
Does anyone know what this means, and how I can avoid it since it stops my program from collecting data as I would like?
Many thanks,
Ben
I can recreate your error message using the rjson package.
Here's an example that works.
rjson::fromJSON('{"x":"a string"}')
# $x
# [1] "a string"
If we omit a double quote from the value of x, then we get the error message.
rjson::fromJSON('{"x":"a string}')
# Error in rjson::fromJSON("{\"x\":\"a string}") : unclosed string
The RJSONIO package behaves slightly differently. Rather than throwing an error, it silently returns a NULL value.
RJSONIO::fromJSON('{"x":"a string}')
# $x
# NULL