Java heap space error in TypeDB when importing CSV files - csv

I tried to import some CSV files into my TypeDB schema using Python and got the following error.
Traceback (most recent call last):
File "C:\Users\panxx006\Desktop\test\migrate.py", line 65, in <module>
build_test_graph(inputs=inputs)
File "C:\Users\panxx006\Desktop\test\migrate.py", line 9, in build_test_graph
load_data_into_typedb(input, session)
File "C:\Users\panxx006\Desktop\test\migrate.py", line 21, in load_data_into_typedb
transaction.commit()
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\connection\transaction.py", line 95, in commit
self.execute(transaction_commit_req())
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\connection\transaction.py", line 80, in execute
return self.run_query(request, batch).get()
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\stream\bidirectional_stream.py", line 136, in get
return self._stream.fetch(self._request_id)
File "C:\Users\panxx006\AppData\Local\Programs\Python\Python39\lib\site-packages\typedb\stream\bidirectional_stream.py", line 84, in fetch
raise error
typedb.common.exception.TypeDBClientException: Java heap space
Please check server logs for the stack trace.
Could you please tell me the way to fix this error? I'm using TypeDB 2.6.3

Upgrading your version of TypeDB to the latest version will help solve this error. As of writing, TypeDB 2.13.0 is the newest version, which includes a number of improvements to memory usage so you should see fewer errors of this kind in the future.
You can download the latest version of TypeDB from the releases page on GitHub: https://github.com/vaticle/typedb/releases
Full disclosure: I work for Vaticle, the creators of TypeDB.

Related

Windows 10 Rtree installation successful from .whl file, but error when running code

I am running Python 3.7, 64bit on Windows 10 and trying desperately to get Rtree running. I use the package Rtree-0.9.1-cp37-cp37m-win_amd64.whl from Christoph Gohlke (https://www.lfd.uci.edu/~gohlke/pythonlibs/).
I have tried for very long to get it to work, but keep on getting the following error message when running a script that uses geopandas.
Traceback (most recent call last):
File "C:\Python37\lib\site-packages\rtree\core.py", line 90, in <module>
rt = ctypes.CDLL(os.path.join(here, 'spatialindex_c.dll'))
File "C:\Python37\lib\ctypes\__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
The installation of the whl-package should include the libspatialindex files, but they are not found when running the code. I tried to use Python 2.7 first to run it, then installed Python 3.7. I've checked all the dependencies and checked whether the "spatialindex_c.dll" files are at the right place, but nothing helps. Would be great to get an answer on that.

selenium.common.exceptions.WebDriverException: Message: Service

I had a trouble when i use selenium to control my Chrome.
Here is my code:
from selenium import webdriver
driver = webdriver.Chrome()
When i tried to operate it ,it runs successfully at first,the Chrome pop on the screen. However, it shut down at the few seconds.
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
driver = webdriver.Chrome('C:\Program Files (x86)\Google\Chrome\chrome.exe')
File "C:\Users\35273\AppData\Local\Programs\Python\Python35\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 62, in __init__
self.service.start()
File "C:\Users\35273\AppData\Local\Programs\Python\Python35\lib\site-packages\selenium\webdriver\common\service.py", line 86, in start
self.assert_process_still_running()
File "C:\Users\35273\AppData\Local\Programs\Python\Python35\lib\site-packages\selenium\webdriver\common\service.py", line 99, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service C:\Program Files (x86)\Google\Chrome\chrome.exe unexpectedly exited. Status code was: 0
You need to provide the path of chromedriver...download from http://chromedriver.storage.googleapis.com/index.html?path=2.24/...unzip it and provide path to it in... webdriver.chrome ("path to chromedriver")
I explain the things here:
from selenium import webdriver
driver = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\Application\chrome.exe")
This is the error if i run the above code:
C:\Python27\python.exe C:/Users/Gaurav.Gaurav-PC/PycharmProjects/Learning/StackOverflow/SeleniumQuestion/test123.py
Traceback (most recent call last):
File "C:/Users/Gaurav.Gaurav-PC/PycharmProjects/Learning/StackOverflow/SeleniumQuestion/test123.py", line 4, in <module>
driver = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\Application\chrome.exe")
File "C:\Python27\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 62, in __init__
self.service.start()
File "C:\Python27\lib\site-packages\selenium\webdriver\common\service.py", line 86, in start
self.assert_process_still_running()
File "C:\Python27\lib\site-packages\selenium\webdriver\common\service.py", line 99, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service C:\Program Files (x86)\Google
\Chrome\Application\chrome.exe unexpectedly exited. Status code was: 0
Which is same as mentioned by #Weiziyoung in original problem.
The solution is as I mentioned you need to provide the path to chromedriver in place of chrome browser like
driver = webdriver.Chrome("E:\Jars\chromedriver.exe")
It will resolve the problem
Download the appropriate driver:http://chromedriver.storage.googleapis.com/index.html
turn up E:\Python37\Scripts <--Paste the chromedriver.exe file in
i faced the same error (behave.step_registry.AmbiguousStep: #given('Lunch chrome Browser') has already been defined in existing step #given('Lunch chrome Browser') at features/steps/OrangeHRMlogin.py:4). i think it takes previous program where it mentioned "Lunch chrome browser". so i did some changes like in 2nd program i cahnged at (U'chrome browser').change the browser name or the Phase and it runs successfully.
the original chrome browser is opened instead of chrome driver that's the reason of that error download chrome driver and give it's path

Pyramid, Python3, SQLAlchemy and MySQL

I want Pyramid, Python3, SQLAlchemy and MySQL to play nice on a single machine. This question has been asked before, but it was a long enough time ago that I'm hoping the answer has changed...
Questions:
What driver works for getting a Python 3 Pyramid app to talk nicely to MySQL via SQLAalchemy? Is this even possible?
I've gotten quite far in this project thinking it would be a simple thing to switch out databases, now I'm a bit stuck and downgrading to Python 2.7 is not something I want to have to go through. So my second question is:
If there is no way to talk to MySQL nicely without downgrading, is there a database that Python 3 is ready for (besides SQLite)?
Or lastly, how would I overcome any of the problems mentioned in the 'Stuff I've Tried' section below?
Stuff I've Tried:
PyMySQL
I have PyMySQL installed and I even got it to complain about incorrect passwords and a missing database so it's getting something right. But when I tried to initialize my database (using initialize_foo_db) everything went sour:
2013-02-06 17:18:31,282 INFO [sqlalchemy.engine.base.Engine][MainThread] b'SELECT DATABASE()'
2013-02-06 17:18:31,282 INFO [sqlalchemy.engine.base.Engine][MainThread] ()
Traceback (most recent call last):
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/pool.py", line 719, in _do_get
return self._pool.get(wait, self._timeout)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/util/queue.py", line 137, in get
raise Empty
sqlalchemy.util.queue.Empty
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "../bin/initialize_mega_db", line 9, in <module>
load_entry_point('mega==0.0', 'console_scripts', 'initialize_mega_db')()
File "/home/sheena/MegaCatalogue/env/mega/mega/scripts/initializedb.py", line 46, in main
Base.metadata.create_all(engine)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/schema.py", line 2567, in create_all
tables=tables)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/base.py", line 2301, in _run_visitor
conn = self.contextual_connect(close_with_result=False)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/base.py", line 2492, in contextual_connect
self.pool.connect(),
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/pool.py", line 224, in connect
return _ConnectionFairy(self).checkout()
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/pool.py", line 387, in __init__
rec = self._connection_record = pool._do_get()
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/pool.py", line 741, in _do_get
con = self._create_connection()
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/pool.py", line 188, in _create_connection
return _ConnectionRecord(self)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/pool.py", line 273, in __init__
pool.dispatch.first_connect.exec_once(self.connection, self)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/event.py", line 279, in exec_once
self(*args, **kw)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/event.py", line 288, in __call__
fn(*args, **kw)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/strategies.py", line 168, in first_connect
dialect.initialize(c)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/dialects/mysql/base.py", line 1999, in initialize
default.DefaultDialect.initialize(self, connection)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/default.py", line 177, in initialize
self._get_default_schema_name(connection)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/dialects/mysql/base.py", line 1964, in _get_default_schema_name
return connection.execute('SELECT DATABASE()').scalar()
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/base.py", line 1449, in execute
params)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/base.py", line 1628, in _execute_text
statement, parameters
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context
context)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/SQLAlchemy-0.7.8-py3.2.egg/sqlalchemy/engine/default.py", line 333, in do_execute
cursor.execute(statement, parameters)
File "/home/sheena/MegaCatalogue/env/local/lib/python3.2/site-packages/PyMySQL3-0.5-py3.2.egg/pymysql/cursors.py", line 108, in execute
query = query % escaped_args
TypeError: unsupported operand type(s) for %: 'bytes' and 'tuple'
oursql
I tried installing oursql using pip install -r file
where file contains:
#oursql==0.9.3 but need special Python 3 build https://bugs.launchpad.net/oursql/+bug/1051512
https://launchpad.net/oursql/py3k/py3k-0.9.3/+download/oursql-0.9.3.zip
pyramid==1.3.4
And I got:
oursqlx/compat.h:13:19: fatal error: mysql.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
myconnpy
This installs without error but I can't seem to find out what I should tell SQLAlchemy to look for. In the appropriate settings.ini file I have tried:
sqlalchemy.url = mysql+myconnpy://...
as well as a bunch of other things. Google is failing me on this one...
UPDATE - a semblance of progress
I've managed to install oursql build dependencies:
sudo aptitude install python-pip libmysqlclient-dev
I then downloaded oursql0.9.3 and ran:
setup.py install
The output was this:
cython not found, using previously-cython'd .c file.
running install
running build
running build_ext
running install_lib
copying build/lib.linux-x86_64-3.2/oursql.cpython-32mu.so -> /home/sheena/MegaCatalogue/env/lib/python3.2/site-packages
running install_egg_info
Writing /path/to/virtualenv/foo/lib/python3.2/site-packages/oursql-0.9.3.egg-info
I opened a shell and apparently the module doesn't exist - it can't be imported anyway. But for some reason it is listed if I say help() then modules

Stanford Tagger in nltk not working due to JVM parameters

I am having a wired error while running following example code snippet
st = StanfordTagger('bidirectional-distsim-wsj-0-18.tagger')
st.tag('What is the airspeed of an unladen swallow ?'.split())
The first line worked properly but second line is giving following error.
Could not create the Java virtual machine.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.6/dist-packages/nltk-2.0.1rc1- py2.6.egg/nltk/tag/stanford.py", line 51, in tag
return self.batch_tag([tokens])[0]
File "/usr/local/lib/python2.6/dist-packages/nltk-2.0.1rc1-py2.6.egg/nltk/tag/stanford.py", line 77, in batch_tag
stdout=PIPE, stderr=PIPE)
File "/usr/local/lib/python2.6/dist-packages/nltk-2.0.1rc1-py2.6.egg/nltk/internals.py", line 166, in java
raise OSError('Java command failed!')
OSError: Java command failed!
I have tried adding .usr/lib/jvm into path but still not working
It wasn't working for me either. So I tried the following and its working perfectly.
st = POSTagger('path-to/stanford-postagger-full-2012-07-09/models/wsj-0-18-left3words.tagger','path-to/stanford-postagger-full-2012-07-09/stanford-postagger.jar')
and use nltk's tokenize method instead of Python's split()
taggedSentence= st.tag(nltk.word_tokenize(sentence))
I see that question is very outdated, but this days I got same error for unknown reason. It gives me a lot of headache. But I found solution.
First, I installed Oracle Java (here is instructions: How To Manually Install Oracle Java on a Debian or Ubuntu VPS)
Now, my python script told me more information on error. It outputs something like:
Forking JVM: error=12, Cannot allocate memory or error=12, Not enough space
Here you can read more about such problem: Forking the JVM
And to avoid that annoying error I need to edit /etc/sysctl.conf and add the following:
vm.overcommit_memory = 1
Then restart system for the change to take effect.

SQLAlchemy mysql dialect error with py2app

I'm having trouble getting SQLAlchemy to work with my compiled app after getting through py2app. I've done this successfully in Windows with py2exe. The following is my setup file:
APP = ['Blah.py']
DATA_FILES = []
OPTIONS = {'argv_emulation': True,
'includes': ['sip',
'PyQt4._qt',
'sqlalchemy.dialects.mysql',
'MySQLdb',
"gzip"]
}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)
This appears like it's the right way to do it, as I've seen people use it for sqlite, however I still get this error upon trying to run the app after compile:
sqlalchemy.exc.ArgumentError: Could not determine dialect for 'mysql+mysqldb'
I've recently been trying with PyInstaller and have gotten stuck at pretty much the same spot, albeit with a different error which is the following:
Traceback (most recent call last):
File "<string>", line 96, in <module>
File "/Users/tom/Downloads/pyinstaller-pyinstaller-2145d84/PyInstaller/loader/iu.py", line 386, in importHook
mod = _self_doimport(nm, ctx, fqname)
File "/Users/tom/Downloads/pyinstaller-pyinstaller-2145d84/PyInstaller/loader/iu.py", line 480, in doimport
exec co in mod.__dict__
File "build/bdist.macosx-10.7-intel/egg/MySQLdb/__init__.py", line 19, in <module>
File "/Users/tom/Downloads/pyinstaller-pyinstaller-2145d84/PyInstaller/loader/iu.py", line 386, in importHook
mod = _self_doimport(nm, ctx, fqname)
File "/Users/tom/Downloads/pyinstaller-pyinstaller-2145d84/PyInstaller/loader/iu.py", line 480, in doimport
exec co in mod.__dict__
File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 4, in __bootstrap__
File "OSX_Installer/Jango/build/pyi.darwin/Jango/out00-PYZ.pyz/pkg_resources", line 882, in resource_filename
File "OSX_Installer/Jango/build/pyi.darwin/Jango/out00-PYZ.pyz/pkg_resources", line 1352, in get_resource_filename
File "OSX_Installer/Jango/build/pyi.darwin/Jango/out00-PYZ.pyz/pkg_resources", line 1363, in _extract_resource
KeyError: '_mysql/_mysql.so'
you probably also need _mysql which should include the native mysql bindings. also, this bindings need the binary mysql libraries to be installed on the target system.
your application would probably be a lot more portable if you used a pure python mysql library, such as pymysql or MySQL Connector/Python (both are supported by sqlalchemy)