Stanford Tagger in nltk not working due to JVM parameters - nltk

I am having a wired error while running following example code snippet
st = StanfordTagger('bidirectional-distsim-wsj-0-18.tagger')
st.tag('What is the airspeed of an unladen swallow ?'.split())
The first line worked properly but second line is giving following error.
Could not create the Java virtual machine.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.6/dist-packages/nltk-2.0.1rc1- py2.6.egg/nltk/tag/stanford.py", line 51, in tag
return self.batch_tag([tokens])[0]
File "/usr/local/lib/python2.6/dist-packages/nltk-2.0.1rc1-py2.6.egg/nltk/tag/stanford.py", line 77, in batch_tag
stdout=PIPE, stderr=PIPE)
File "/usr/local/lib/python2.6/dist-packages/nltk-2.0.1rc1-py2.6.egg/nltk/internals.py", line 166, in java
raise OSError('Java command failed!')
OSError: Java command failed!
I have tried adding .usr/lib/jvm into path but still not working

It wasn't working for me either. So I tried the following and its working perfectly.
st = POSTagger('path-to/stanford-postagger-full-2012-07-09/models/wsj-0-18-left3words.tagger','path-to/stanford-postagger-full-2012-07-09/stanford-postagger.jar')
and use nltk's tokenize method instead of Python's split()
taggedSentence= st.tag(nltk.word_tokenize(sentence))

I see that question is very outdated, but this days I got same error for unknown reason. It gives me a lot of headache. But I found solution.
First, I installed Oracle Java (here is instructions: How To Manually Install Oracle Java on a Debian or Ubuntu VPS)
Now, my python script told me more information on error. It outputs something like:
Forking JVM: error=12, Cannot allocate memory or error=12, Not enough space
Here you can read more about such problem: Forking the JVM
And to avoid that annoying error I need to edit /etc/sysctl.conf and add the following:
vm.overcommit_memory = 1
Then restart system for the change to take effect.

Related

Java heap space error in TypeDB when importing CSV files

I tried to import some CSV files into my TypeDB schema using Python and got the following error.
Traceback (most recent call last):
File "C:\Users\panxx006\Desktop\test\migrate.py", line 65, in <module>
build_test_graph(inputs=inputs)
File "C:\Users\panxx006\Desktop\test\migrate.py", line 9, in build_test_graph
load_data_into_typedb(input, session)
File "C:\Users\panxx006\Desktop\test\migrate.py", line 21, in load_data_into_typedb
transaction.commit()
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\connection\transaction.py", line 95, in commit
self.execute(transaction_commit_req())
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\connection\transaction.py", line 80, in execute
return self.run_query(request, batch).get()
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\stream\bidirectional_stream.py", line 136, in get
return self._stream.fetch(self._request_id)
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\stream\bidirectional_stream.py", line 84, in fetch
raise error
typedb.common.exception.TypeDBClientException: Java heap space
Please check server logs for the stack trace.
Could you please tell me the way to fix this error? I'm using TypeDB 2.6.3
Upgrading your version of TypeDB to the latest version will help solve this error. As of writing, TypeDB 2.13.0 is the newest version, which includes a number of improvements to memory usage so you should see fewer errors of this kind in the future.
You can download the latest version of TypeDB from the releases page on GitHub: https://github.com/vaticle/typedb/releases
Full disclosure: I work for Vaticle, the creators of TypeDB.

libvirt.libvirtError: An error occurred, but the cause is unknow

I am using XEN hypervisor. For managing virtual Machine I am using virt-manager whenever I want to start to Virtual Machine at last when everything is ready and I click the create Button I get the following error
Unable to complete install: 'An error occurred, but the cause is unknown'
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/createvm.py", line 2089, in _do_async_install
guest.installer_instance.start_install(guest, meter=meter)
File "/usr/share/virt-manager/virtinst/install/installer.py", line 542, in start_install
domain = self._create_guest(
File "/usr/share/virt-manager/virtinst/install/installer.py", line 491, in _create_guest
domain = self.conn.createXML(install_xml or final_xml, 0)
File "/usr/lib/python3/dist-packages/libvirt.py", line 4034, in createXML
if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirt.libvirtError: An error occurred, but the cause is unknow
For the Xen driver, you may have more infos in /var/log/libvirt/libxl/libxl-driver.log
Same issue here.
When I check the /var/log/syslog, there is a log from libvirtd: "unsupported configuration: emulator '/usr/lib/xen-4.11/bin/qemu-system-i386' not found".
It may be caused by this bug. So we can create a symbolic link sudo ln -s /usr/bin/qemu-system-i386 /usr/lib/xen-4.11/bin/qemu-system-i386, and I hope it works for you too.
Check if you have installed
qemu-system-x86-xen
if not - install
apt install qemu-system-x86-xen
that helped me

Windows 10 Rtree installation successful from .whl file, but error when running code

I am running Python 3.7, 64bit on Windows 10 and trying desperately to get Rtree running. I use the package Rtree-0.9.1-cp37-cp37m-win_amd64.whl from Christoph Gohlke (https://www.lfd.uci.edu/~gohlke/pythonlibs/).
I have tried for very long to get it to work, but keep on getting the following error message when running a script that uses geopandas.
Traceback (most recent call last):
File "C:\Python37\lib\site-packages\rtree\core.py", line 90, in <module>
rt = ctypes.CDLL(os.path.join(here, 'spatialindex_c.dll'))
File "C:\Python37\lib\ctypes\__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
The installation of the whl-package should include the libspatialindex files, but they are not found when running the code. I tried to use Python 2.7 first to run it, then installed Python 3.7. I've checked all the dependencies and checked whether the "spatialindex_c.dll" files are at the right place, but nothing helps. Would be great to get an answer on that.

How do I connect Airflow to SQLite locally?

I'm trying to try out Airflow for the very first time and I'm trying to connect it to a local SQLite database. But I can't seem to get my head around on how to actually do it.
I've read up on Airflow's document, Set my executor to LocalExecutor and set up my sql_alchemy_conn to sqlite:////home/myName/Programs/sqlite3/DatabaseName.db but it doesn't seem to work as it throws an
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 21, in <module>
from airflow import configuration
File "/usr/local/lib/python2.7/dist-packages/airflow/__init__.py", line 35, in <module>
from airflow import configuration as conf
File "/usr/local/lib/python2.7/dist-packages/airflow/configuration.py", line 520, in <module>
conf.read(AIRFLOW_CONFIG)
File "/usr/local/lib/python2.7/dist-packages/airflow/configuration.py", line 283, in read
self._validate()
File "/usr/local/lib/python2.7/dist-packages/airflow/configuration.py", line 169, in _validate
self.get('core', 'executor')))
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the LocalExecutor
error when I tried to run airflow initdb. I tried to google around and tried vipul sharma's solution found here and set the value of my sql_alchemy_conn to mysql://:#localhost:3306/ but it still doesn't work as it throws an
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1045, "Access denied for user 'myName'#'localhost' (using password: NO)")
error. I know that the answer should be really simple but I really don't understand how to so I hope you can guide me through on what to do/read.
Use SequentialExecutor
"This executor will only run one task instance at a time, can be used for debugging. It is also the only executor that can be used with sqlite since sqlite doesn’t support multiple connections." airflow documentation
You just didn't need to change to LocalExecutor, change it back to SequentialExecutor, change sql_alchemy_conn to point to sqlite:////home/myName/Programs/sqlite3/DatabaseName.db and stop airflow services (webserver, scheduler).
Execute airflow initdb then start up the services again.
Hopefully that works.

mysql error for module DBD::mysql: libmysqlclient.so.15:

i am using mysql for orthomcl software. when i use the command
orthomclLoadBlast my_orthomcl/orthomcl.config my_orthomcl/similarSequence.txt
it gives this error
Can't load '/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/mysql/mysql.so' for module DBD::mysql: libmysqlclient.so.15: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230, line 5. at /storage/home/cdac/OrthoMCL/orthomclSoftwarev2.0.8/bin/../lib/perl/OrthoMCLEngine/Main/Base.pm line48
Compilation failed in require at /storage/home/cdac/OrthoMCL/orthomclSoftware-v2.0.8/bin/../lib/perl/OrthoMCLEngine/Main/Base.pm line 48, <F> line 5.
How to solve this problem ??
You could install a local perl in your home and you could leave vendor perl untouched.
You could use perlbrew or local::lib to maintain an up to date perl version without modifying the system installed perl.