i got some problems with my code. After i read a JSON received via socket by a Qt client in a python server, i want to get all the fields of that JSON so i can use it, but i got an error like this: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0). This is the code that cause the exception to raise.
data = connection.recv(1024)
temp = data.decode("utf-8")
jdata = json.loads(temp)
The exception is raised by json.loads(temp). I tried to be sure to have the right type for the loads function, i tried to copy the same string that i get from the socket into another str type and the function works well. Does anyone knows if there is something i overlooked?
update: I just found out that the JSON i get from the socket have a size that differs from a string with the same characters
the exception "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)" usually indicates that the data you're trying to load is not valid JSON.
See Python JSON Docs
"If the data being deserialized is not a valid JSON document, a JSONDecodeError will be raised." (Scroll down to json.loads function)
Can you print out your temp variable? You'll be able to see the value you're trying to JSONDecode.
Related
I am trying to load a JSON file into BigQuery using the bq load command
bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON project_abd:ds.online_data gs://online_data/file.json
One of the key:value pair in the JSON file looks like -
"taxIdentifier":"T"
The bq load fails with the message - Error while reading data, error message: JSON parsing error in row
starting at position 713452: Could not convert value to boolean.
Field: taxIdentifier; Value: T (The JSON is really huge, hence cant paste it here)
I am really confused as to why the autodetect is treating the value T as boolean. I have tried all combinations of creating the table with STRING datatype and then load the table, but due to autodetect, it errors out mentioning - changed type from STRING to BOOLEAN, if I do not use the autodetect the load succeeds.
I have to use the "autodetect" feature, since the JSON is a result of an API call and the columns may increase or decrease.
Any idea why the value T is behaving weird, and how to get around this ?
I am trying to send the following JSON object and I get the error shown below,I checked #http://jsonviewer.stack.hu ,the JSON format seems to be correct,what am I missing and how to fix this?
{"component":{"name":"Company tech (New Bugs)", "version":"B"},"assignee":1234456,"milestone":"chiHW","priority":2,"state":"Analyze","substate":"Nominate","title":"[please ignore]CS\:4355C1\,4364B2\:WDI\:DHD\:HLK\(16299\)\-\>\"DF\ \-\ Sleep\ Tests\"\-\>Assert\-\>bcmpciedhd63\.sys\(dhd\_os\_ioctl\_resp\_wait\)\-\>dhd\_ndis\.c\#4449"}
Error:-
{"message":"An invalid JSON Object was passed in the request body. Please pass a valid JSON object.","help":"View documentation at http://bugs.company.com/","title":"Invalid Request","status":"400 Bad Request"}
You could have pasted your expression into https://jsonlint.com. It tells you where the problem is, and you can experiment until the JSON is no longer invalid. In your case, the problem is those backslashes in the last dictionary value (the one that starts "please ignore").
You could use jq to get a more specific parse error
watson:~$ cat >j
{"component":{"name":"Company tech (New Bugs)", "version":"B"},"assignee":1234456,"milestone":"chiHW","priority":2,"state":"Analyze","substate":"Nominate","title":"[please ignore]CS\:4355C1\,4364B2\:WDI\:DHD\:HLK\(16299\)\-\>\"DF\ \-\ Sleep\ Tests\"\-\>Assert\-\>bcmpciedhd63\.sys\(dhd\_os\_ioctl\_resp\_wait\)\-\>dhd\_ndis\.c\#4449"}
^D
watson:~$ jq <j
parse error: Invalid escape at line 1, column 333
I am using the tweetscores package of R to get 'tweets list from twitter. The tweets are stored in json format. While converting it to a data frame I get a lexical error
' Error: lexical error: inside a string, '\' occurs before a character which it may not.".
Any solution to the mentioned error.
A part of the json file text
":[{"text":["MUFC"],"indices":[[83],[88]]}],"symbols":[],"user_mentions":[],"urls":[]},"metadata":{"iso_language_code":["en"],"result_type":["recent"]},"source":["http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>"],"in_reply_to_status_id":null,"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"in_reply_to_user_id_str":null,"in_reply_to_screen_name":null,"user":{"id":[7.32108114527322e+017],"id_str":["732108114527322112"],"name":["wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww(^o^)/"],"screen_name":["SukiSukinal"],"location":["+6222"],"description":["Alliansi osaosi ngevote kagak. katanya sih fans a.k.a + "],"url":null,"entities":{"description":{"urls":[]}},"protected":[false],"followers_count":[163],"friends_count":[107],"listed_count":[4],"created_at":["Mon May 16 07:19:11 +0000
Json format does not allow backslashes so you need to escape them. replace any '\' character found with '\\'. Refer [here][1]
[1]: http://www.json.org/ for more info
You likely have an incomplete json string, which may be caused by the package or by an interrupted connection to Twitter's API. A complete json string returned from Twitter should look something like the following:
which I got using rtweet's stream_tweets() function. With a complete string returned by Twitter's REST or stream API, you should be able to convert the data using basically any json parser (e.g., jsonlite::fromJSON()).
I am trying to insert JSON payload into memsql JSON type column but it is failing due to the following reason.
My JSON content having '?' character.
I tried to escape '?' by using the following ways, but it doesn't worked for me.
The Exception i am getting is:
Root Exception stack trace:
java.lang.IndexOutOfBoundsException: Index: 0
Ex payload: "question mark content?"
1. #[org.mule.util.StringUtils.replace(payload,"?","\\?")]
Result: "question mark content\?"
2. #[org.mule.util.StringUtils.replace(payload,"?","\?")]
Result: not allowed to use the above expression
If i use the payload "question mark content" then it is inserted successfully.
Please help me how can I escape '?' in my JSON content while saving it into memsql?
'\?' itself is an escape sequence, so achive this you have to use "\\?" which produce "\?" which should work with memsql.
#[org.mule.util.StringUtils.replace(payload,"?","\\\\?")]
Hope this helps.
From the looks of your exception it looks like you are calling for it to replace the payload, but you're not assigning it to anything.
Going off of the documentation at:
http://grepcode.com/file/repo1.maven.org/maven2/commons-lang/commons-lang/2.4/org/apache/commons/lang/StringUtils.java#3457
It basically says that it's trying to replace the items in a string, and the method itself returns a string. Based on what I can tell in the stack trace, it seems as though you are passing a null or uninitialized variable to something that's trying to parse str[0], which is returning an array out of bounds error.
The way to correct this would be to do something like:
payload = org.mule.util.StringUtils.replace(payload,"?","\\\\?")
Which should replace any instance of ? with \? and re-write it to the payload variable. That said, it sounds like payload may actually be null when you're evaluating it later in your program, which could be indicative of a larger issue.
I'm using the jsonlite library to try to get some JSON data and turn it into a data frame that I could use, but so far it hasn't worked. I've tried three methods so far:
testData = fromJSON("<json file location>")
Which outputs:
Error in feed_push_parser(readBin(con, raw(), n), reset = TRUE) :
parse error: trailing garbage
n_reply_to_status_id":null} {"contributors":null,"text":"Te
(right here) ------^
So I figured that if the quotes caused error, I'd just have to remove them:
singleString <- paste(readLines("<json file location>"), collapse=" ")
singleString = gsub('"', "",singleString)
singleString = fromJSON(singleString)
Which outputs:
Error: lexical error: invalid char in json text.
{contributors:null,text:A colour
(right here) ------^
It seems to be pointing to an 'o' in the text. If I delete that 'o' it just points to the 'n' that comes after it.
My last attempt I read on a forum post of someone having a similar problem that it should solve it:
stream = stream_in(file("<json file location>"))
I was pretty happy to see that this did not return an error and that a data frame named "stream" had been added to memory, but when I tried to view that data frame...
> View(stream)
Error in View : 'names' attribute [1] must be the same length as the vector [0]
I'm not exactly sure what this means or what I could do about it in this situation.
If you have an idea of I could get this data loaded correctly, I would deeply appreciate it. Thanks!
EDIT: Here is the JSON data in question: https://gist.github.com/geocachecs/f68d769aeed8e019a26cc230559bbf7f