Python JSON Decoding With Asynchat Cannot Catch ValueError Exception - json

I can't seem to catch an exception when using json.loads even though I specifically call it out. I largely see this when trying to stress my server with lots of client connection sending data very quickly. See my error below:
(I've replaced my IP address with X's in the error code)
EX: Unterminated string starting at: line 1 column 49 (char 48) Data:
'{"ap-hdop":0.55,"rtcmin":"38","ap-latdec":3.134,"a' error: uncaptured
python exception, closing channel
(:Unterminated string
starting at: line 1 column 49 (char 48)
[//faraday_server_handler.py|collect_incoming_data|34]
[/usr/lib/python2.7/json/init.py|loads|338]
[/usr/lib/python2.7/json/decoder.py|decode|366]
[/usr/lib/python2.7/json/decoder.py|raw_decode|382])
I understand this that the code fails because I simply miss a double quotes on the line:
'{"ap-hdop":0.55,"rtcmin":"38","ap-latdec":3.134,"a'
This line is usually a LOT longer so that "a.... was supposed to keep going and complete it's quotes.
Here's my relevant code:
def collect_incoming_data(self, data):
"""Read an incoming message from the client, place JSON message data into buffer"""
#self.logger.debug('collect_fing_data() -> (%d bytes)\n"""%s"""', len(data), data)
try:
loaded_data = json.loads(data)
except ValueError, ex:
self.handle_error()
type,value,traceback = sys.exc_info()
print type
#print "Collect Incoming Data: " . time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime())
print "EX: ", ex
print "Data: ",repr(data)
Any ideas? I scoured the internet to see if I can find this issue, but I appear to be setting up to capture the exception which everyone else having this issue with loads seems to suggest to do.
EDIT 3/1/2016 - Evening
Commenting out my exception handle_error() let me see more of the error:
except ValueError, ex:
self.handle_error()
type,value,traceback = sys.exc_info()
print type
#print "Collect Incoming Data: " . time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime())
print "EX: ", ex
print "Data: ",repr(data)
Below is my new error, I've commented out personal data. It's apparent that the issue I really have now is in-fact the unterminated string
EX: Unterminated string starting at:
line 1 column 49 (char 48) Data:
'{"ap-hdop":0.55,"rtcmin":"31","ap-latdec":XX.XXX,"a' EX: No JSON object could be decoded Data:
'p-latdeg":34,"adc6":2006,"adc7":2007,"adc4":2004,"adc5":2005,"adc2":2002,"adc3":2003,"adc0":2000,"adc1":2001,"gpio-0":30,"gpio-1":50,"gpio-2":70,"speed":5.0,"adc8":2008,"rtcday":"01","longdeg":118,"longdec":XX.XXX,"altitude":31.0,"ap-speed":0.0,"ap-pdop":0.77,"lat-dir":"N","long-dir":"W","hdop":0.01,"ap-rf":0,"alt-units":"M","rtcdow":"2","callsign":"XXXXX","ap-callsign":"XXXXX","id":1,"ap-id":1,"rtcyear":"2016","rtcmon":"03","ap-vdop":0.66,"ap-lat-dir":"N","vdop":0.02,"rtchour":"22","latdec":XX.XXX,"latdeg":34,"ap-longdeg":118,"ap-longdec":XX.XXX,"rtcsec":"15","ap-altitude":86.0,"ap-long-dir":"W","pdop":0.01,"ap-alt-units":"M","faraday-port":0}'

OK my original question was answered which was "Why am I not catching the ValueError exception even though I am providing code to do just that?"
#tadhg McDonald-jensen was correct with his comment to remove my call to handle_error().
I still have some other issues but they are a different question. Thanks!

Related

Cannot send base64 String to PubNub

I am using PyCamera module of Raspberry Pi to capture an image and store as .jpg.
first encoding the image using base64.encodestring(). but while sending encoded string to PubNub server, I get error on my_publish_callback as
('ERROR: ', 'Expecting value: line 1 column 1 (char 0)')
('ERROR: ', JSONDecodeError('Expecting value: line 1 column 1 (char 0)',))
I have tried using base64.b64encode() but still get the same errors. I have tried the script in python 2 and 3;
def my_publish_callback(envelope, status):
if not status.is_error():
pass # Message successfully published to specified channel.
else:
#print("recv: ", envelope)
print("ERROR: ", status.error_data.information)
print("ERROR: ", status.error_data.exception)
def publish(channel, msg):
pubnub.publish().channel(channel).message(msg).async(my_publish_callback)
def captureAndSendImage():
camera.start_preview()
time.sleep(2)
camera.capture("/home/pi/Desktop/image.jpg")
camera.stop_preview()
with open("/home/pi/Desktop/image.jpg", "rb") as f:
encoded = base64.encodestring(f.read())
publish(myChannel, str(encoded))
I am not able to find or print full error traceback so that I can get some more clues about where the error is occurring. But it looks like PubNub is trying to parse the data in JSON, and its failing.
I realized the .jpg file size is 154KB, whereas PubNub max packet size is 32KB, so that should clearly say it all. PubNub recommends to send large messages by splitting them and re-arranging them in subscriber-end. Thanks #Craig for referring to that link, Its useful though support.pubnub.com/support/discussions/topics/14000006326

LuaLaTex using fontspec package and luacode reading JSON file

I'm using Latex since years but I'm new to embedded luacode (with Lualatex). Below you can see a simplified example:
\begin{filecontents*}{data.json}
[
{"firstName":"Max", "lastName":"Möller"},
{"firstName":"Anna", "lastName":"Smith"}
];
\end{filecontents*}
\documentclass[11pt]{article}
\usepackage{fontspec}
%\setmainfont{Carlito}
\usepackage{tikz}
\usepackage{luacode}
\begin{document}
\begin{luacode}
require("lualibs.lua")
local file = io.open('data.json','rb')
local jsonstring = file:read('*a')
file.close()
local jsondata = utilities.json.tolua(jsonstring)
tex.print('\\begin{tabular}{cc}')
for key, value in pairs(jsondata) do
tex.print(value["firstName"] .. ' & ' .. value["lastName"] .. '\\\\')
end
tex.print('\\hline\\end{tabular}')
\end{luacode}
\end{document}
When executing Lualatex following error occurs:
LuaTeX error [\directlua]:6: attempt to index field 'json' (a nil value) [\directlua]:6: in main chunk. \end{luacode}
When commenting the line \usepackage{fontspec} the output will be produced. Alternatively, the error can be avoided by commenting utilities.json.tolua(jsonstring) and all following lua-code lines.
So the question is: How can I use both "fontspec" package and json-data without generating an error message? Apart from this I have another question: How to enable german umlauts in output of luacode (see first "lastName" in example: Möller)?
Ah, I'm using TeX Live 2015/Debian on Ubuntu 16.04.
Thank you,
Jerome

Error in fromJSON(commandArgs(1)) : unexpected character '''

I am calling a R script from Scala by using scala.sys.process. This script takes a command line argument in JSON format and processes it. I am using rjson's fromJSON function to store the JSON in a list.
This works very fine when I execute the R script from the command line:
$ ./dfChargerFlink.R '{"Id":"1","value":"ABC"}'
But when I call it from scala, I get the following error:
Error in fromJSON(commandArgs(1)) : unexpected character '''
Execution halted
This is the code I am using:
val shellCommand = "./dfChargerFlink.R '"+arg+"'"
return shellCommand !!
where arg is the JSON string.
You can notice that I have appended " ' " to both the sides of the JSON string as if I don't, I get this error:
Error in fromJSON(commandArgs(1)) : unclosed string
Execution halted
How can this be solved? Is it some bug?

World of tanks Python list comparison from json

ok I am trying to create a definition which will read a list of IDS from an external Json file, Which it is doing. Its even putting the data into the database on load of the program, my issue is this. I cant seem to match the list IDs to a comparison. Here is my current code:
def check(account):
global ID_account
import json, httplib
if not hasattr(BigWorld, 'iddata'):
UID_DB = account['databaseID']
UID = ID_account
try:
conn = httplib.HTTPConnection('URL')
conn.request('GET', '/ids.json')
conn.sock.settimeout(2)
resp = conn.getresponse()
qresp = resp.read()
BigWorld.iddata = json.loads(qresp)
LOG_NOTE('[ABRO] Request of URL data successful.')
conn.close()
except:
LOG_NOTE('[ABRO] Http request to URL problem. Loading local data.')
if UID_DB is not None:
list = BigWorld.iddata["ids"]
#print (len(list) - 1)
for n in range(0, (len(list) - 1)):
#print UID_DB
#print list[n]
if UID_DB == list[n]:
#print '[ABRO] userid located:'
#print UID_DB
UID = UID_DB
else:
LOG_NOTE('[ABRO] userid not set.')
if 'databaseID' in account and account['databaseID'] != UID:
print '[ABRO] Account not active in database, game closing...... '
BigWorld.quit()
now my json file looks like this:
{
"ids":[
"1001583757",
"500687699",
"000000000"
]
}
now when I run this with all the commented out prints it seems to execute perfectly fine up till it tries to do the match inside the for loop. Even when the print shows UID_DB and list[n] being the same values, it does not set my variable, it doesn't post any errors, its just simply acting as if there was no match. am I possibly missing a loop break? here is the python log starting with the print of the length of the table print:
INFO: 2
INFO: 1001583757
INFO: 1001583757
INFO: 1001583757
INFO: 500687699
INFO: [ABRO] Account not active, game closing......
as you can see from the log, its never printing the User located print, so it is not matching them. its just continuing with the loop and using the default ID I defined above the definition. Anyone with an idea would definitely help me out as ive been poking and prodding this thing for 3 days now.
the answer to this was found by #VikasNehaOjha it was missing simply a conversion to match types before the match comparison I did this by adding in
list[n] = int(list[n])
that resolved my issue and it finally matched comparisons.

Error in fromJSON(paste(raw.data, collapse = "")) : unclosed string

I am using the R package rjson to download weather data from Wunderground.com. Often I leave the program to run and there are no problems, with the data being collected fine. However, often the program stops running and I get the following error message:
Error in fromJSON(paste(raw.data, collapse = "")) : unclosed string
In addition: Warning message:
In readLines(conn, n = -1L, ok = TRUE) :
incomplete final line found on 'http://api.wunderground.com/api/[my_API_code]/history_20121214pws:1/q/pws:IBIRMING7.json'
Does anyone know what this means, and how I can avoid it since it stops my program from collecting data as I would like?
Many thanks,
Ben
I can recreate your error message using the rjson package.
Here's an example that works.
rjson::fromJSON('{"x":"a string"}')
# $x
# [1] "a string"
If we omit a double quote from the value of x, then we get the error message.
rjson::fromJSON('{"x":"a string}')
# Error in rjson::fromJSON("{\"x\":\"a string}") : unclosed string
The RJSONIO package behaves slightly differently. Rather than throwing an error, it silently returns a NULL value.
RJSONIO::fromJSON('{"x":"a string}')
# $x
# NULL