Recently, when running my tests in Pytest, I started to get a strange warning at the end of the test results; many many iterations of the following:
Exception ignored in: <function _ConnectionRecord.checkout.<locals>.<lambda> at 0x10eea07a0>
Traceback (most recent call last):
File "/Users/username/appdev/scattr-api/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 503, in <lambda>
File "/Users/username/appdev/scattr-api/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 710, in _finalize_fairy
File "/Users/username/appdev/scattr-api/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 528, in checkin
File "/Users/username/appdev/scattr-api/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 387, in _return_conn
File "/Users/username/appdev/scattr-api/venv/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 106, in _do_return_conn
TypeError: catching classes that do not inherit from BaseException is not allowed
This started happening recently, regardless of whether tests pass or not. I don't understand why this warning started showing up, or what it means that it's "ignored," so I'm not sure what I should be doing about it, if anything. I haven't noticed any problems when running my actual application, it's just something that's happened when running the tests.
The fact that the last function in traceback is "_do_return_conn" makes me think it's something about the connection pool, but I'm still not sure how to make sense of it...
I determined that the issue arises from not explicitly closing a connection in pytest tests.
I'd been using custom connection to look up values in the database in the tests (to make sure correct values were inserted) and it's if this wasn't explicitly closed at the end that this cryptic error message appeared.
SQLAlchemy has garbage collection which will eventually return connections to the pool even if they're not explicitly closed. However, in the context of running pytest, this doesn't have time to happen during the duration of the test, resulting in the cryptic error message.
Related
I am working on topic inference that will require to load a previously saved model.
However, I got a pickle error that says
Traceback (most recent call last):
File "topic_inference.py", line 35, in <module>
model_for_inference = gensim.models.LdaModel.load(model_name, mmap = 'r')
File "topic_modeling/env/lib/python3.8/site-packages/gensim/models/ldamodel.py", line 1663, in load
result = super(LdaModel, cls).load(fname, *args, **kwargs)
File "topic_modeling/env/lib/python3.8/site-packages/gensim/utils.py", line 486, in load
obj = unpickle(fname)
File "topic_modeling/env/lib/python3.8/site-packages/gensim/utils.py", line 1461, in unpickle
return _pickle.load(f, encoding='latin1') # needed because loading from S3 doesn't support readline()
TypeError: __randomstate_ctor() takes from 0 to 1 positional arguments but 2 were given
The code I use to load the model is simply
gensim.models.LdaModel.load(model_name, mmap = 'r')
Here is the code that I use to create and save the model
model = gensim.models.ldamulticore.LdaMulticore(
corpus=comment_corpus,
id2word=key_word_dict, ## This is now a gensim.corpora.Dictionary Object, previously it was the .id2token attribute
chunksize=chunksize,
alpha='symmetric',
eta='auto',
iterations=iterations,
num_topics=num_topics,
passes=epochs,
eval_every=eval_every,
workers = 15,
minimum_probability= 0.0)
model.save(output_model)
where output_model doesn't have an extension like .model or .pkl
In the past, I tried the similar approach with the exception that I passed in a .id2token attribute under the gensim.corpora.Dictionary object instead of the full gensim.corpora.Dictionary to the id2word parameter when I created the model, and the method loads the model fine back then. I wonder if passing in a corpora.Dictionary will make a difference in the loading output...? Back that time, I was using regular python, but now I am using anaconda. However, all the versions of the packages are the same.
Another report of an error about __randomstate_ctor (at https://github.com/numpy/numpy/issues/14210) suggests the problem may be related to numpy object pickling.
Is there a chance that the configuration where your load is failing is using a later version of numpy than when the save occurred? Could you try, at least temporarily, rolling back to some older numpy (that's still sufficient for whatever Gensim you're using) to see if it helps?
If you find any load that works, even in a suboptimal config, you might be able to null-out whatever random-related objects are causing the problem and re-save, then having a saved version that loads better in your truly-desired configuration. Then, if the random-related objects truly needed after reload, it may be possible to manually re-constitute them. (I haven't looked into this yet, but if you find any workaround allowing a load, but then aren't sure what to manually null/rebuild, I could take a closer look.)
I'm using Windows UIAutomation quite heavily and it works mostly well. But it's throwing some exceptions that I haven't been able to figure out.
The first of these is:
Exception thrown at 0x762D46D2 (KernelBase.dll) in MyApp.exe: 0x80040155: Interface not registered.
The offending section of code is the last line of:
IUIAutomationElementPtr element; //This is a valid element
IUIAutomationCacheRequestPtr cachePtr; //This is a valid cache pointer
IUIAutomationElement **first;
HRESULT hr = myControlWalker->GetFirstChildElementBuildCache(element, cachePtr, firstChild);
The second of these is:
Exception thrown at 0x762D46D2 (KernelBase.dll) in MyApp.exe: 0x40080202: WinRT transform error (parameters: 0x80040155, 0x80004002, 0x0000001D, 0x0565DB4C).
which happens at the same line of code as the previous.
And the last is:
Exception thrown at 0x762D46D2 (KernelBase.dll) in MyApp.exe: 0x8001010D: An outgoing call cannot be made since the application is dispatching an input-synchronous call.
once again, from the same line of code.
The program runs fine and it does what it needs to do, it just bothers me to have zillions of these exceptions being thrown left and right. Any suggestions?
I'm trying to get the MNIST example for TPUs in GCE running (as shown at https://cloud.google.com/tpu/docs/tutorials/mnist) but I've run into a couple of bumps. First, I had to set my PYTHONPATH to pick up the models directory which isn't listed as a step in the walk-through (perhaps it's obvious to daily python programmers, but it's not stated if it's not). After that I'm now hitting the following error that I'm not sure how to work around:
frival#tpu-demo-vm:~$ python /usr/share/models/official/mnist/mnist_tpu.py --tpu_name=$TPU_NAME --data_dir=${STORAGE_BUCKET}/data --model_dir=${STORAGE_BUCKET}/output --use_tpu=True --iterations=500 --train_steps=1000 --train_file=${STORAGE_BUCKET}/data/train.tfrecords
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
Traceback (most recent call last):
File "/usr/share/models/official/mnist/mnist_tpu.py", line 163, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "/usr/share/models/official/mnist/mnist_tpu.py", line 135, in main
FLAGS.tpu, zone=FLAGS.tpu_zone, project=FLAGS.gcp_project)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/cluster_resolver/python/training/tpu_cluster_resolver.py", line 128, in __init__
self._tpu = compat.as_bytes(tpu) # self._tpu is always bytes
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/compat.py", line 68, in as_bytes
(bytes_or_text,))
TypeError: Expected binary or unicode string, got None
I've verified that TPU_NAME and STORAGE_BUCKET are set properly, and I've also verified that I see the TPU in the READY state from this VM although I don't think either of those would have caused this error. Does anyone know what I'm missing?
Probably your tensorflow version is newer than your mnist_tpu.py.
You may try this newer version of mnist_tpu.py here, instead use the --tpu flag as tayo mentioned above.
https://github.com/tensorflow/models/blob/master/official/mnist/mnist_tpu.py
Please change the --tpu_name=$TPU_NAME flag to --tpu=$TPU_NAME.
Apologies for the error, as this was a recent internal change that did not make it to the walk-through documentation. It is being corrected.
Good luck in TPU land!
I stumbled upon a very weird issue in ColdFusion which might very well be a bug, but I wanted to post it here in case I was missing something.
It seems that cfindex is throwing an uncatchable exception when trying to index invalid files.
I'm not entirely sure yet what consists of an invalid file, but the issue occurs with a valid PDF file, but that doesn't have a PDF extension. Obviously, the problem can easily be fixed by adding a valid extension, but that's not the scope of the question.
<cftry>
<cfindex
collection="some_collection"
action="update"
key="//someserver/some_file_without_extension">
After index
<cfcatch type="any">
Exception caught
</cfcatch>
<cffinally>
Finally block
</cffinally>
</cftry>
Completed
When running the above code, the only thing that is output is Finally block, therefore it's like if there is an exception thrown, but which cannot be caught.
I even tried with type="java.lang.Exception", type="java.lang.Throwable" and type="searchengine" and nothing works.
The only way I found to detect such exceptions is to check a boolean flag in the finally block, but gracefully recovering form those errors is very cumbersome.
Another very weird thing that occurs is that right after the issue is encountered, if I refresh the page in the browser, I'll get the following error:
HTTP Error 503.0 - Server Error
The service is unavailable.
Module IsapiModule
Notification ExecuteRequestHandler
Handler JWildCardHandler
Error Code 0x00000000
Then after refreshing again, I get Finally block and if I keep refreshing the same behavior occurs over and over (Finally block then 503 error).
I'm looking for a way to catch the actual exception thrown by BizTalk 2006 R2 when a receive port can't parse a message it picks up.
For example, I have a csv file that was not properly created, say it's missing a comma, so when BizTalk tries to determine the message type, it errors out. Without using failed message routing, there are 2 entries in the application event log. One has a pretty generic error description, something along the lines of "The Messaging Engine encountered an error during the processing of one or more inbound messages." The second entry has what I'm looking for. It will contain the details of where the missing comma is, something along the lines of "Unexpected data found while looking for:
','
The current definition being parsed is POSTrailer. The stream offset where the error occured is 44443. The line number where the error occured is 244. The column where the error occured is 1."
I've tried using the ESB toolkit Exception Handler for BizTalk 2006 R2. Using failed message routing and their exception handing framework, the error description it catches is the first, generic, error description.
I do not see any way to obtain the second, more detailed error description. I'm assuming somewhere in the bowels of the biztalk messaging engine this exception is getting thrown, which is caught by something a little bit higher in the process, which throws it's own generic biztalk exception. I would also assume that the more specific error description is contained as the inner exception of the exception that gets routed to the ESB toolkit, however by the time I can get a hold of it, as far as I can tell, the actual exception object has been replaced by the message context, which is nothing more than a bunch of name/value pairs. The error description is now the generic one that is not helpful, and anything resembling an inner exception is just a hexidecmal code, which means nothing to my end users.
Everything I've found regarding obtaining a specific error message deals with exceptions thrown from an orchestration. I am not dealing with an orchestration, though, I'm dealing with a receive port. The closest thing I can find is here. But even if I could create a custom pipeline component using that, that means I either need to add that component into every exising custom pipline, or create a new one to validate XML and use that instead of the XMLRecieve or XMLSend pipelines that come with BizTalk... That also means trapping errors generated from EDI is out of the question. Does anyone know of a way to get at this more detailed error description from the BizTalk generated error message?
I'm not sure of a tool that will do exactly what you're looking for, but I do have a couple of options.
1) I'm pretty sure SCOM can grab those errors and alert whoever needs to be alerted. I've had some problems with this in the past, though, so it may not be your best bet.
2) For basic Failure handling, you can write code to scrape the EventLog for any warning or error from BizTalk and pass that down the line (my current client does this- it's not super elegant, but it works well).
3) What I would encourage- if you don't want orchestrations- is to set up custom pipelines which simply implement your error handling/monitoring in addition to your normal operations. Then, for non-critical messages, you can still use the standard pipelines (reducing overhead/latency) and for critical messages you can use your "GuaranteedDeliveryPassThruPipeline" or your "GuaranteedDeliveryXMLTransmitPipeline" or whatever.