In Python, how would one retrieve information from a incomplete or "overcomplete" JSON string?
Example (incomplete):
msg = '{"MESSAGE": {"MESSAGE_SIZE": "2230", "IMAGE_NUM'
Example (overcomplete):
msg = '{"MESSAGE": {"MESSAGE_SIZE": "2230", "IMAGE_NUMBER": "16227"}}{"MESSA'
Specifying the key MESSAGE_SIZE, I want to retrieve the integer 2230.
The position of the key inside the string is mutable.
One working solution I found is the following (ugly) piece of code. I'm sure there are better solutions though:
key = "\"MESSAGE_SIZE\":"
len_start_idx = 0
len_str = ""
len_int = 0
# Find position right after key
start = msg.find(key)
msg = msg[start+len(key):]
# Find the first value digit after the key
for c in msg:
if not c.isdigit():
len_start_idx += 1
else:
break
msg = msg[len_start_idx:]
# Append value digits until no more digits are found
for c in msg:
if c.isdigit():
len_str += c
else:
break
len_int = int(len_str)
Interestingly, there is a npm-module to untruncate json files but I couldn't find anything similar for python.
It shouldn't be too complicated to implement one though. This should work in most cases (also closing lists):
import json
import re
closing_chars = {
'{': '}',
'[': ']'
}
def close_structs(json_string):
stack = []
for char in json_string:
if char in ['{', '[']:
stack.append(char)
elif char in ['}', ']']:
stack.pop()
for open_struct in stack[::-1]:
json_string += closing_chars[open_struct]
return json_string
def untruncate_json(json_string):
while True:
try:
d = json.loads(json_string)
return d
except Exception as e:
if "Expecting ':' delimiter" in repr(e):
json_string += ": null"
elif "Unterminated string starting at" in repr(e):
json_string += '"'
elif "Expecting property name enclosed in double quotes" in repr(e) \
or "Expecting value" in repr(e):
json_string = re.sub(r'[:,]\s*$', '', json_string)
elif "Expecting ',' delimiter" in repr(e):
json_string = close_structs(json_string)
else:
print(json_string)
raise e
msg = '{"MESSAGE": {"MESSAGE_SIZE": "2230", "IMAGE_NUMBER": "16227", "TE'
d = untruncate_json(msg)
# getting the desired value:
print(int(d["MESSAGE"]["MESSAGE_SIZE"]))
Edit: handling cases "Expecting property name enclosed in double quotes" and "Expecting value" + stripping all kinds of whitespace chars with re.sub (in order to include \n)
need your kind help!
I have defined a new Exception called 'UnderAge' (just checking if a person is under an age limit).
The strange thing is that when i use 'raise UnderAge as e....' it does not work under the 'try' segment.
only the exception is working.
Here is the code :
class UnderAge(Exception):
def __init__(self, name, age):
self._name = name
self._age = age
def __str__(self):
return "the age %s is under 18 yo" % self._age
def get_age(self):
return self._age
def send_invitation(name, age):
try:
if int(age) < 18:
raise UnderAge(name, age)
else:
print("You should send an invitation to " + name)
except UnderAge as e:
print("Age must be above 18 but now age is %s." % e.get_age())
def main():
send_invitation("miri", 16)
if __name__ == "__main__":
main()
-----When running it i only get :
C:\Python\Python38-32\python.exe "C:/Users/mikie/Documents/Pyton Programs/Next.py Course/Targil_3.3.2a.py"
Age must be above 18 but now age is 16.
Process finished with exit code 0
----- i should see also an output indicating the UnderAge and its relevant str which i have changed.
Please help me guys:-)
thanks,
Mike
false alarm - this is a normal behaviour of the raise.
It should pop the customized exception only if there is no exception defined in the code after the 'try'.
Also, the string which overrides the__str__ can be called by e.str.
this is the code in Python, I really don't know how to do this, I am just a beginner and someone can understand my question and help me
def get_float(prompt, low, high):
while True:
prompt = input("Enter monthly investment:")
number= float(input(prompt))
if number > low or number <= high:
is_valid = True
return number
else:
print("Entry must be greater than {low}and less than or equal to {high}")
def main():
get_float(prompt,low,high)
if __name__ == "__main__":
main()
In main, you are passing in the prompt variable into get_float. However, prompt is not defined in main, therefore you are attempting to pass an undefined variable which is not allowed.
In fact, given that get_float reads the prompt from input (and not the value passed in), you do not need to pass prompt into get_float, and prompt can be removed from the function signature.
You cannot pass prompt as a Function argument cause you are reading the value inside the function
def get_float(low, high):
while True:
prompt = input("Enter monthly investment:")
number= float(input(prompt))
if number > low or number <= high:
is_valid = True
return number
else:
print("Entry must be greater than {low}and less than or equal to {high}")
def main():
get_float(200,1000)
if __name__== "__main__":
main()
def usersdecision(n):
if option == n:
calculation.format(n)()
Is there a way to get calculation to become calculation + the value of n then (),
so if n=1 calculation would become calculation1().
Here is an inelegant way of doing it. Pass in a list, see how long it is and use the length to call different routines
import sys
def userd(*n):
if (len(n) == 1):
return calc1(n[0])
if (len(n) == 2):
return calc2(n[0],n[1])
if (len(n) > 2):
warn("bad input",n)
sys.exit(1)
def calc1(x):
return x*2
def calc2(x,x2):
return x**x2
assert userd(1) == 2
assert userd(2,3) == 8
print "it works"
This is my attempt so far to trap all exceptions in Jython code. The most difficult thing, I find, is trapping exceptions when you override a method from a Java class: with the "vigil" decorator below (which also tests whether the EDT/Event Despatch Thread status is correct) you can find out the first line where the code is thrown... so you can identify the method itself. But not the line.
Furthermore, tracing stack frames back through Python and Java stacks is completely beyond me. Obviously there seem to be these layers and layers of "proxy" classes, a no doubt inevitable part of Jython mechanics. It'd be great if someone much cleverer than me were to take an interest in this question!
NB this is an example of how to use the "vigil" decorator:
class ColumnCellRenderer( javax.swing.table.DefaultTableCellRenderer ):
#vigil( True ) # means a check is done that the thread is the EDT, as well as intercepting Python Exceptions and Java Throwables...
def getTableCellRendererComponent( self, table, value, isSelected, hasFocus, row, column):
super_comp = self.super__getTableCellRendererComponent( table, value, isSelected, hasFocus, row, column)
super_comp.foreground = java.awt.Color.black
super_comp.font = main_frame_self.m_plain_font
...
... and these are three functions I use in an attempt to trap stuff...
def custom_hook(type_of_e, e, tb ):
"""
Method to catch Python-style BaseExceptions, using the command sys.excepthook = custom_hook.
The first thing this method needs to do in Jython is to determine whether this is a Java
java.lang.Throwable or not. If it is this JThrowable
must be handled by the code which caters for B{uncaught Java Throwables}.
"""
try:
if 'tb' not in locals():
tb = None
logger.error("Python custom_hook called...\ntype of e: %s\ne: %s\ntb: %s" % ( unicode( type_of_e ), unicode( e ),
unicode( tb ) ))
msg = ''.join( traceback.format_exception(type_of_e, e, tb ))
logger.error( "traceback:\n" + msg )
except BaseException, e:
logger.error( "exception in Python custom_hook!:\n%s" % e )
raise e
sys.excepthook = custom_hook
class JavaUncaughtExceptHandler( java.lang.Thread.UncaughtExceptionHandler ):
"""
java.lang.Class to catch any Java Throwables thrown by the app.
"""
def uncaughtException( self, thread, throwable ):
try:
'''
NB getting the Java stack trace like this seems to produce a very different trace
from throwable.printStackTrace()... why?
'''
# we want a single log message
exception_msg = "\n*** uncaught Java Exception being logged in %s:\n" % __file__
baos = java.io.ByteArrayOutputStream()
ps = java.io.PrintStream(baos)
throwable.printStackTrace( ps )
# remove multiple lines from Java stack trace message
java_stack_trace_lines = unicode( baos.toString( "ISO-8859-1" )).splitlines()
java_stack_trace_lines = filter( None, java_stack_trace_lines )
normalised_java_stack_trace = '\n'.join( java_stack_trace_lines )
exception_msg += normalised_java_stack_trace + '\n'
python_traceback_string = traceback.format_exc()
exception_msg += "Python traceback:\n%s" % python_traceback_string
logger.error( exception_msg )
except (BaseException, java.lang.Throwable ), e:
logger.error( "*** exception in Java exception handler:\ntype %s\n%s" % ( type( e ), unicode( e ) ) )
raise e
# NB printStackTrace causes the custom_hook to be invoked... (but doesn't print anything)
java.lang.Thread.setDefaultUncaughtExceptionHandler( JavaUncaughtExceptHandler() )
def vigil( *args ):
"""
Decorator with two functions.
1. to check that a method is being run in the EDT or a non-EDT thread;
2. to catch any Java Throwables which otherwise would not be properly caught and documented: in particular,
with normal Java error-trapping in Jython it seems impossible to determine the line number at which an
Exception was thrown. This at least records the line at which a Java java.lang.Throwable
was thrown.
"""
if len( args ) != 1:
raise Exception( "vigil: wrong number of args (should be 1, value: None/True/False): %s" % str( args ))
req_edt = args[ 0 ]
if req_edt and type( req_edt ) is not bool:
raise Exception( "vigil: edt_status is wrong type: %s, type %s" % ( req_edt, type( req_edt )) )
def real_decorator( function ):
if not hasattr( function, '__call__' ):
raise Exception( "vigil: function %s does not have __call__ attr, type %s"
% ( function, type( function )) )
# NB info about decorator location can't be got when wrapper called, so record it at this point
penultimate_frame = traceback.extract_stack()[ -2 ]
decorator_file = penultimate_frame[ 0 ]
decorator_line_no = penultimate_frame[ 1 ]
def wrapper( *args, **kvargs ):
try:
# TODO is it possible to get the Python and/or Java stack trace at this point?
if req_edt and javax.swing.SwingUtilities.isEventDispatchThread() != req_edt:
logger.error( "*** vigil: wrong EDT value, should be %s\nfile %s, line no %s, function: %s\n" %
( "EDT" if req_edt else "non-EDT", decorator_file, decorator_line_no, function ))
return function( *args, **kvargs )
except ( BaseException, java.lang.Throwable ), e:
''' NB All sorts of problems if a vigil-protected function throws an exception:
1) just raising e means you get a very short stack trace...
2) if you list the stack trace elements here you get a line (seemingly inside the function where the
exception occurred) but often the wrong line!
3) Python/Java stack frames: how the hell does it all work???
4) want a single error message to be logged
'''
msg = "*** exception %s caught by vigil in file %s\nin function starting line %d" % ( e, decorator_file, decorator_line_no )
logger.error( msg )
frame = inspect.currentframe()
# the following doesn't seem to work... why not?
python_stack_trace = traceback.format_stack(frame)
python_stack_string = "Python stack trace:\n"
for el in python_stack_trace[ : -1 ]:
python_stack_string += el
logger.error( python_stack_string )
if isinstance( e, java.lang.Throwable ):
# NB problems with this stack trace: although it appears to show the
# correct Java calling pathway, it seems that the line number of every file and method
# is always shown as being the last line, wherever the exception was actually raised.
# Possibly try and get hold of the actual Pyxxx objects ... (?)
java_stack_trace = e.stackTrace
java_stack_string = "Java stack trace:\n"
for el in java_stack_trace:
java_stack_string += " %s\n" % unicode( el )
logger.error( java_stack_string )
raise e
return wrapper
return real_decorator
PS it is of course possible to top-and-tail every overridden Java method with try ... except... but where's the fun in that? Seriously, even doing that I'm unable to find the line at which the exception is thrown...
Here's an example of a decorator used in the socket module in Jython to map Java exceptions to Python exceptions. I didn't read your vigil decorator too closely since it's doing a lot of work, but I'm putting it here in case it might help:
def raises_java_exception(method_or_function):
"""Maps java socket exceptions to the equivalent python exception.
Also sets _last_error on socket objects so as to support SO_ERROR.
"""
#wraps(method_or_function)
def handle_exception(*args, **kwargs):
is_socket = len(args) > 0 and isinstance(args[0], _realsocket)
try:
try:
return method_or_function(*args, **kwargs)
except java.lang.Exception, jlx:
raise _map_exception(jlx)
except error, e:
if is_socket:
args[0]._last_error = e[0]
raise
else:
if is_socket:
args[0]._last_error = 0
return handle_exception
Mostly what we are seeing here is that we are dispatching on whether it is a Java exception (java.lang.Exception) or not. I suppose this could be generalized to java.lang.Throwable, although it's unclear what can be done in the case of java.lang.Error in any event. Certainly nothing corresponding to socket errors!
The above decorator in turn uses the _map_exception function to unwrap the Java exception. It's quite application specific here as you can see:
def _map_exception(java_exception):
if isinstance(java_exception, NettyChannelException):
java_exception = java_exception.cause # unwrap
if isinstance(java_exception, SSLException) or isinstance(java_exception, CertificateException):
cause = java_exception.cause
if cause:
msg = "%s (%s)" % (java_exception.message, cause)
else:
msg = java_exception.message
py_exception = SSLError(SSL_ERROR_SSL, msg)
else:
mapped_exception = _exception_map.get(java_exception.__class__)
if mapped_exception:
py_exception = mapped_exception(java_exception)
else:
py_exception = error(-1, 'Unmapped exception: %s' % java_exception)
py_exception.java_exception = java_exception
return _add_exception_attrs(py_exception)
There is some clunkiness in the code, and I'm sure room for improvement, but overall it certainly makes any decorated code much easier to follow. Example:
#raises_java_exception
def gethostname():
return str(InetAddress.getLocalHost().getHostName())
Jim Baker's answer is interesting... but what I wanted is sthg comprehensive which documents the maximum possible amount of stack trace info when an exception of any kind is raised. CPython not being multi-thread, its stack trace doesn't have to cope with Runnables. I'm not enough of a Jython/Python expert to know whether you can always get the whole stack in "pure Python" code (i.e. no use of Java classes).
But one of the things I wanted to get was the activity which had led up to the running of a Runnable in Jython. And the activity which had led up to the Runnable having run that Runnable, etc., right back to the very first thread. My solution below, taking inspiration from Jim's answer but also from doublep's comment, creates a new Jython class, TraceableRunnable, which will store the list of stack traces on creation.
When an exception, either Java or Python style, is raised, this logs everything right back to the start of the run (if you systematically use TraceableRunnable instead of Runnable).
Each run() code of a TraceableRunner subclass also has to do this call at some point:
self.record_thread()
... hopefully this is not too irksome an imposition...
(NB in a truly "grown up" implementation you'd want to check that this call had been made... I'm sure that this could be done by some suitably sophisticated Pythonic technique, failing which by unit testing or something. Also you might want to require a call at the very end of the run() code to delete this dictionary entry...)
This is the catching and logging code:
class TraceableRunnable( java.lang.Runnable ):
thread_to_tr_dic = {}
def __init__( self ):
# no need to call super's __init__: Runnable is a Java *interface*
caller_thread = java.lang.Thread.currentThread()
self.frame_stack_list = []
if hasattr( caller_thread, 'frame_stack_list' ):
self.frame_stack_list = copy.deepcopy( caller_thread.frame_stack_list )
self.frame_stack_list.append( traceback.extract_stack() )
def record_thread( self ):
TraceableRunnable.thread_to_tr_dic[ java.lang.Thread.currentThread() ] = self
class EDTException( Exception ):
pass
def vigil( *args ):
"""
Decorator with two functions.
1. to check that a method is being run in the EDT or a non-EDT thread
2. to catch any exceptions
"""
if len( args ) != 1:
raise Exception( "vigil: wrong number of args (should be 1, value: None/True/False): %s" % str( args ))
req_edt = args[ 0 ]
if req_edt and type( req_edt ) is not bool:
raise Exception( "vigil: edt_status is wrong type: %s, type %s" % ( req_edt, type( req_edt )) )
def process_exception( exc, python = True ):
tb_obj = sys.exc_info()[ 2 ]
msg = "Exception thrown message %s\nfamily %s, type: %s\n" % ( str( exc ), "Python" if python else "Java", type( exc ))
msg += "traceback object part:\n"
ex_tb = traceback.extract_tb( tb_obj )
# first is frame in vigil
ex_tb = ex_tb[ 1 : ]
if not ex_tb:
msg += " none\n"
else:
tb_strings = traceback.format_list( ex_tb )
for tb_string in tb_strings:
msg += tb_string
curr_thread = java.lang.Thread.currentThread()
if curr_thread in TraceableRunnable.thread_to_tr_dic:
runnable = TraceableRunnable.thread_to_tr_dic[ curr_thread ]
# duck-typing, obviously... although redundant test, as only TraceableRunnables should be in the dictionary...
if hasattr( runnable, 'frame_stack_list' ):
msg += "\nOLDER STACKS:\n"
for frame_stack in runnable.frame_stack_list:
msg += "\nframe stack id: %d\n" % id( frame_stack )
frame_stack = frame_stack[ : -1 ]
if not frame_stack:
msg += " no frames\n"
else:
# most recent call first: reverse array...
stack_strings = traceback.format_list( reversed( frame_stack ))
for stack_string in stack_strings:
msg += stack_string
logger.error( msg )
def real_decorator( function ):
if not hasattr( function, '__call__' ):
raise Exception( "vigil: function %s does not have __call__ attr, type %s"
% ( function, type( function )) )
# NB info about decorator location can't be got when wrapper called, so record it at this point
penultimate_frame = traceback.extract_stack()[ -2 ]
decorator_file = penultimate_frame[ 0 ]
decorator_line_no = penultimate_frame[ 1 ]
def wrapper( *args, **kvargs ):
try:
if req_edt is not None and javax.swing.SwingUtilities.isEventDispatchThread() != req_edt:
msg = \
"vigil: wrong EDT value, should be %s\nfile %s\nline no %s, function: %s" % \
( "EDT" if req_edt else "non-EDT", decorator_file, decorator_line_no, function )
raise EDTException( msg )
return function( *args, **kvargs )
except BaseException, e:
# we cannot know how calling code will want to deal with an EDTException
if type( e ) is EDTException:
raise e
process_exception( e )
except java.lang.Throwable, t:
process_exception( t, False )
return wrapper
return real_decorator
Would welcome improvements from proper programmers...!