i got some problems with my code. After i read a JSON received via socket by a Qt client in a python server, i want to get all the fields of that JSON so i can use it, but i got an error like this: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0). This is the code that cause the exception to raise.
data = connection.recv(1024)
temp = data.decode("utf-8")
jdata = json.loads(temp)
The exception is raised by json.loads(temp). I tried to be sure to have the right type for the loads function, i tried to copy the same string that i get from the socket into another str type and the function works well. Does anyone knows if there is something i overlooked?
update: I just found out that the JSON i get from the socket have a size that differs from a string with the same characters
the exception "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)" usually indicates that the data you're trying to load is not valid JSON.
See Python JSON Docs
"If the data being deserialized is not a valid JSON document, a JSONDecodeError will be raised." (Scroll down to json.loads function)
Can you print out your temp variable? You'll be able to see the value you're trying to JSONDecode.
for prefix, event, value in parser:
print(prefix)
I get the following error after executing the above code , I dont understand what the error is.
ijson.common.IncompleteJSONError: parse error: trailing garbage
nt":19485,"verified":false}} {"user_id":1408970940,"id":4496
(right here) ------^
Seems you are trying to parse a JSON document with more than one top-level object, which is not standard. ijson allows for this though if you give the multiple_values=True flag to the method you are using.
I have to load the JSON into Bigquery. As per the Bigquery documentation, I made my JSON in the correct formation i.e. Newline delimited with one JSON object each row.
Now, JSON file has around 10 million rows and while loading I get the below error:
Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 2165; errors: 1. Please look into the error stream for more details.
When I found the line 2165, it looks like as follows:
{"deviceId":"13231fd01222a28e","dow":"Wednesday","downloadFlag":"N","email":"clstone898#gmail.com","emailSha256":"1bdf11821f867799bde022ccb57a2e899f827c988b4275571ffd60279c863272","event":"streamStop","firebaseUID":"UDVC3hyQpBWLCnlhXhjAQBeI95Q2","halfHourFull":"08h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"google","minutesSinceMidnight":497,"quarterHourFull":"08q2","stationName":"Fox hit 101.9","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T08:017:04.679+11:00","timestampUTC":"2018-02-06T21:17:04.679Z"}
When I load this single line then it gets loaded successfully. Kindly guide/suggest what is incorrect here.
I'm loading this json from Bigquery UI using schema Auto-Detect option.
Sample records are as follows:
{"deviceId":"3c7a345dafcff93f","dow":"Tuesday","downloadFlag":"N","email":"psloper.ps#gmail.com","emailSha256":"1cebae8c35db32edcd35e746863fc65a04ac68f2f5b3350f2df477a86bfaa07d","event":"streamStop","firebaseUID":"AMFYYjsvZjauhCktJ5lUzZj0d3D2","halfHourFull":"21h2","liveFlag":"Y","localDate":"2018-02-06","localHalfHour":2,"login":"google","minutesSinceMidnight":1311,"quarterHourFull":"21q4","stationName":"hit 105","streamListenMethod":"Headphones","timestampLocal":"2018-02-06T21:51:40.216+10:00","timestampUTC":"2018-02-06T11:51:40.216Z"}
{"deviceId":"2f1a8c84c738b752","dow":"Wednesday","downloadFlag":"N","email":"kory.maxwell#icloud.com","emailSha256":"13348786c15bff95e4afb4968a9bdbe883b70206a737c02c89fc8215f2a4e101","event":"streamStop","facebookId":"1784054201892593","firebaseUID":"Tx1bHjP6dhaDB2nl2c7yi2KZHsq2","halfHourFull":"06h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"facebook","minutesSinceMidnight":384,"quarterHourFull":"06q2","stationName":"hit 105","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T06:24:44.533+10:00","timestampUTC":"2018-02-06T20:24:44.533Z"}
{"deviceId":"AA1D685F-6BF6-B0DC-0000-000000000000","dow":"Wednesday","email":"lozza073#bigpond.com","emailSha256":"525db286e9a35c9f9f55db0ce338762eee02c51955ede6b35afb7e808581664f","event":"streamStart","facebookId":"10215879897177171","firebaseUID":"f2efT61sW5gHTfgEbtNfyaUKWaF3","halfHourFull":"7h2","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":463,"quarterHourFull":"7q3","stationName":"Fox hit 101.9","streamListenMethod":"Speaker","timestampLocal":"2018-02-07T07:43:00.39+11:00","timestampUTC":"2018-02-06T20:43:00.39Z"}
{"deviceId":"AEFD39FC-B116-4063-0000-000000000000","dow":"Wednesday","event":"catchUpPause","facebookId":"379907925802180","firebaseUID":"vQPh9tbO3Yge88fpMyNUFzJO7dl1","halfHourFull":"7h2","liveFlag":"N","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":465,"quarterHourFull":"7q4","stationName":"Fox hit 101.9","streamListenMethod":"USBAudio","timestampLocal":"2018-02-07T07:45:08.524+11:00","timestampUTC":"2018-02-06T20:45:08.524Z"}
{"deviceId":"AA1D685F-6BF6-B0DC-0000-000000000000","dow":"Wednesday","email":"lozza073#bigpond.com","emailSha256":"525db286e9a35c9f9f55db0ce338762eee02c51955ede6b35afb7e808581664f","event":"streamStop","facebookId":"10215879897177171","firebaseUID":"f2efT61sW5gHTfgEbtNfyaUKWaF3","halfHourFull":"7h2","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":475,"quarterHourFull":"7q4","stationName":"Fox hit 101.9","streamListenMethod":"Speaker","timestampLocal":"2018-02-07T07:55:35.788+11:00","timestampUTC":"2018-02-06T20:55:35.788Z"}
{"deviceId":"AA1D685F-6BF6-B0DC-0000-000000000000","dow":"Wednesday","email":"lozza073#bigpond.com","emailSha256":"525db286e9a35c9f9f55db0ce338762eee02c51955ede6b35afb7e808581664f","event":"streamStart","facebookId":"10215879897177171","firebaseUID":"f2efT61sW5gHTfgEbtNfyaUKWaF3","halfHourFull":"7h2","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":477,"quarterHourFull":"7q4","stationName":"Fox hit 101.9","streamListenMethod":"Speaker","timestampLocal":"2018-02-07T07:57:42.343+11:00","timestampUTC":"2018-02-06T20:57:42.343Z"}
{"deviceId":"13231fd01222a28e","dow":"Wednesday","downloadFlag":"N","email":"clstone898#gmail.com","emailSha256":"1bdf11821f867799bde022ccb57a2e899f827c988b4275571ffd60279c863272","event":"streamStop","firebaseUID":"UDVC3hyQpBWLCnlhXhjAQBeI95Q2","halfHourFull":"08h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"google","minutesSinceMidnight":497,"quarterHourFull":"08q2","stationName":"Fox hit 101.9","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T08:017:04.679+11:00","timestampUTC":"2018-02-06T21:17:04.679Z"}
Any help is greatly appreciated.
Well, look to that specific 2165 line:
{"deviceId":"13231fd01222a28e","dow":"Wednesday","downloadFlag":"N","email":"clstone898#gmail.com","emailSha256":"1bdf11821f867799bde022ccb57a2e899f827c988b4275571ffd60279c863272","event":"streamStop","firebaseUID":"UDVC3hyQpBWLCnlhXhjAQBeI95Q2","halfHourFull":"08h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"google","minutesSinceMidnight":497,"quarterHourFull":"08q2","stationName":"Fox hit 101.9","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T08:017:04.679+11:00","timestampUTC":"2018-02-06T21:17:04.679Z"}
And specifically to:
"timestampLocal":"2018-02-07T08:017:04.679+11:00"
And the error message:
Couldn't convert value to timestamp: Could not parse
'2018-02-07T08:017:04.679+11:00' as a timestamp. Required format is
YYYY-MM-DD HH:MM[:SS[.SSSSSS]]
So, if you change "T08:017:04.679" to "T08:17:04.679" (17 minutes instead of 017) then it works. :)
I am trying to send the following JSON object and I get the error shown below,I checked #http://jsonviewer.stack.hu ,the JSON format seems to be correct,what am I missing and how to fix this?
{"component":{"name":"Company tech (New Bugs)", "version":"B"},"assignee":1234456,"milestone":"chiHW","priority":2,"state":"Analyze","substate":"Nominate","title":"[please ignore]CS\:4355C1\,4364B2\:WDI\:DHD\:HLK\(16299\)\-\>\"DF\ \-\ Sleep\ Tests\"\-\>Assert\-\>bcmpciedhd63\.sys\(dhd\_os\_ioctl\_resp\_wait\)\-\>dhd\_ndis\.c\#4449"}
Error:-
{"message":"An invalid JSON Object was passed in the request body. Please pass a valid JSON object.","help":"View documentation at http://bugs.company.com/","title":"Invalid Request","status":"400 Bad Request"}
You could have pasted your expression into https://jsonlint.com. It tells you where the problem is, and you can experiment until the JSON is no longer invalid. In your case, the problem is those backslashes in the last dictionary value (the one that starts "please ignore").
You could use jq to get a more specific parse error
watson:~$ cat >j
{"component":{"name":"Company tech (New Bugs)", "version":"B"},"assignee":1234456,"milestone":"chiHW","priority":2,"state":"Analyze","substate":"Nominate","title":"[please ignore]CS\:4355C1\,4364B2\:WDI\:DHD\:HLK\(16299\)\-\>\"DF\ \-\ Sleep\ Tests\"\-\>Assert\-\>bcmpciedhd63\.sys\(dhd\_os\_ioctl\_resp\_wait\)\-\>dhd\_ndis\.c\#4449"}
^D
watson:~$ jq <j
parse error: Invalid escape at line 1, column 333