Within my Conda environment with Python 3.6.9, I've installed AllenNLP 9.2.0. I tried to install AllenNLP Server following the instruction from https://github.com/allenai/allennlp-server by running pip install --editable .
However, the installation procedure never finished as the compatibility checks with several modules, e.g. pip is looking at multiple versions of tqdm to determine which version is compatible with other requirements. This could take a while. Collecting tqdm>=4.19
Does anybody know what happens here? Should I add more restrictions to steup.py in AllenNLP server? However, there is any code included in such file.
Thanks a lot for your help.
I just tried it with AllenNLP 2.0.1 (the latest), and while it takes a long time, it does eventually resolve the packages.
That said, I would recommend two things:
Use Python 3.8 instead.
If it still doesn't work, specify a version of tqdm tightly in the requirements. My version automatically picked tqdm==4.56.2, just or reference.
Related
I am using Conda on Ubuntu 16.04. My objective is to associate each Conda environment to a specific version of CUDA / cuDNN. I had a look around and I found this interesting article, which basically suggests to put different CUDA versions into different folders and then use an environment-specific bash script (run when the environment is activated) to properly set the PATH/LD_LIBRARY_PATH variables (which creates the association with the CUDA version).
This is fine, but when I try to install frameworks such as pytorch using Conda, it forces me to install also the "cudatoolkit" package.
So, a couple of questions:
1) does downloading cudatoolkit mess up my previous CUDA configurations? which version will be used?
2) if using Conda is possible to install "cudatoolkit" and also "cudnn", why not just using conda for everything? Why even needing to apply the instructions of the above mentioned article?
Thank you.
As an answer to the first question, no, downloading and installing another CUDA toolkit won't mess up other configurations. From CUDA toolkit installer, you specify an installation directory, so just pick whatever works for you that is unique to that CUDA version. This won't affect any currently installed CUDA versions. A Pytorch install will look for a CUDA_HOME environment variable as well as in '/usr/local/cuda' (the default CUDA toolkit install dir.), so it's just this environment variable that needs to be changed.
I can't speak for the second part. Perhaps the installation using Conda will use the default installation directory for the CUDA toolkit (seems silly but this is just speculation).
While trying to install Perl modules like JSON::XS or YAML::XS, i receive the same error:
XS.xs:1:10: fatal error: 'EXTERN.h' file not found
I use MacBook, xCode is up to date, everything else that could help is up to date too.
Since OS X El Capitan, Apple introduced System Integrity Protection which restricts writing to /usr/lib /usr/bin and other sensitive directories (even to root or sudo user) that are used by the installation of Perl bundled with the Operating System. This can cause issues when it comes to installing new modules and also if trying to install XS modules ( those linked to external C libraries ).
For this reason you should not consider the default Perl installation as a working development environment, especially if you are installing custom modules.
Check out this thread on PM and others. I had since El-Capitan managed to solve this before by manually building from tarball and adding a few params or environment variables to set the paths believing that it would be best to retain use of the system Perl but this is not the way to go. This makes your environment difficult to build but also brittle and sensitive to OS updates that may either break things in many different ways.
The best practice seems to be starting with a Perl using brew install perl and work in this environment, remembering to setup your bash_profile as directed by the installer.
Also worth remembering to do a brew link perl. If you receive warnings about this clobbering what looks like system Perl libraries don't worry - these are likely modules that were installed by you over the top and it will cause you less trouble to link over these. If you have concerns, make a note of which module installs will be cleared and re-install them once your environment is configured ( ie your module installer approach is configured using cpanm or sticking with the old perl -MCPAN -e shell etc)
This new Perl setup from brew eliminates the need to continuing running sudo which adds another layer of things that can go wrong as environment variables don't follow through and permission conflicts arise etc.
Finally to simplify package/module installation I suggest doing a brew install cpanminus. If you had previously already installed this, you can ensure the paths etc are configured by doing a brew reinstall cpanminus
If you want to take it another step further then you can install perlbrew as well which will give you the ability to run multiple versions of Perl as your user and configure these with their own libs and modules which can be very useful particularly if aligning with your production environment for testing etc.
One problem you may face if moving from system Perl to this kind of approach is needing to deal with any hangovers from installing things with sudo. It wis worth taking a little time to get all this set up right though and your issues going forward will be greatly reduced and you won't be left with that nagging feeling that you don't want to change anything for fear of it all breaking.
I have also come across a Perl Blog Article that suggests a fix for XS issues with perlbrew on Mojave
This Gist described updating your cpan shell install root though this shouldn't be necessary unless your cpan is stuck in an old config after taking steps above.
I've also raised this as a new issue on PerlMonks
After reading https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035624 and installing the Additional headers via
sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
I successfully compiled without the missing 'EXTERN.h' error
In order to follow the common advice I also tried with Perlbrew to install a dedicated development version of Perl. Especially with the advice in mind First, do not use the system Perl on MacOS. The installed version is for Apple, not for you (see the discussion here: https://www.perlmonks.org/?node_id=1224727).
Unfortunately, the following error occurred:
Test Summary Report
-------------------
porting/libperl.t (Wstat: 65280 Tests: 35 Failed: 0)
Non-zero exit status: 255
Parse errors: No plan found in TAP output
Files=2653, Tests=1217766, 708 wallclock secs (52.74 usr 9.40 sys + 395.38 cusr 49.90 csys = 507.42 CPU)
Result: FAIL
make: *** [test_harness] Error 1
##### Brew Failed #####
Therefore, I decided to install it the following way (and not following the advice due to the error).
Even after having the above mentioned macOS SDK headers already installed on Catalina (macOS 10.15.2) it didn't work for me. I faced the issue during the installation of the Perl module Mac-SystemDirectory-0.13. The following steps (by identifying the missing file in hope of having a more generic approach for more or less equivalent issues) did the trick:
Locate the header file (in this case EXTERN.h)
sudo find /Library -type f -name EXTERN.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/EXTERN.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Perl/5.28/darwin-thread-multi-2level/CORE/EXTERN.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/EXTERN.h
Ensure the installed Perl version (here 5.18) match the header file:
perl -v | grep version
This is perl 5, version 18, subversion 4 (v5.18.4) built for darwin-thread-multi-2level
Export the path for the C-Compiler (note MacOSX10.15.sdk for Catalina and Perl Version 5.18)
export CPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE
Invoke the Makefile.PL with perl
perl Makefile.PL
BTW — For anybody who's still struggling with this, my workaround was:
bash% module="Sub::Util" # For example
bash% cpanm --configure-args="INC=-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE" "$module"
Please try this
CPATH=$(dirname $(find /usr/local/Cellar/ -name EXTERN.h)) cpan JSON::XS
For Big Sur and perl 5.30, EXTERN.h is at /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE
I'm trying to upgrade CPAN itself and got that error. But I have /usr/bin/cpan and I can't write there so I have to tweak it to write the updated version to /usr/local/bin/cpan.
No promises, but yum install perl-devel worked for me.
As #huyz has helpfully pointed out, if you hit this error on a Mac, you don't have this option, even though this is probably your issue, and you need to follow one of the above methods of getting a version of Perl that isn't missing important chunks, as per other answers.
But if, dear reader, you hit this error on a linux host, as I did, then this might be an option for you.
Building on what E Lisse suggested, you might also have luck looking in
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/
For example:
CPATH=$(dirname $(find /System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/ -name EXTERN.h)) cpan JSON::XS
You could also find where EXTERN.h is located and add that to your shell by default, e.g. in your .bashrc or .zshrc file:
export CPATH=/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/
This earlier SO question suggests mysql.connector, which is what I am actually using. However, as noted in a comment to the accepted answer in the aforementioned question, this is not a standard python module.
My problem: whenever my ISP upgrades to a newer Python version, I have to notice the crashes and bug them to install mysql.connector for me.
I could explicitly call for an older version of Python 3.x. However, all other things being equal, I'd like to use the latest Python, and it seems like MySQL is just such a standard of standards that there really should be a standard Python 3 library for it. Is there? I'm not doing anything fancy with the database (or with python), so basic functionality is o.k.
Just use a venv and isolate yourself from ISP python upgrades:
pyvenv ./mysqlvenv
./mysqlvenv/bin/pip install mysql-connector-python --allow-external mysql-connector-python
install your other dependencies using ./mysqlvenv/bin/pip install foo and Bob's your uncle.
I'm trying to install MySQL in my CentOS 6.4 (X86_64) with the command:
"yum install mysql mysql-server"
And I'm having problems with the installation, the error presented is the following:
Check the error log in the following link http://pastie.org/10022423
Any Idea why this error ? thanks for you helps!
The problem is you already have a version of some MySQL packages installed that are from a different distribution/vendor than CentOS.
Since different people created the packages using different names and standards, yum does not know how to fix this situation.
You have libmysqlclient16-5.1.69-1.w6.x86_64 installed already providing the client libraries, but the CentOS package is mysql-libs. This package appears to be provided by "webtatic":
https://webtatic.com/packages/mysql55/
http://rpm.pbone.net/index.php3/stat/4/idpl/21966562/dir//com/libmysqlclient16-5.1.69-1.w6.x86_64.rpm.html
Likely you need to consider removing the libmysqlclient package and installing mysql-libs in it's place. That will probably remove some items because of dependencies, you might be able to get away with manually removing it with the rpm --nodeps -e, but it may also be the packages cannot have their dependency satisfied by mysql-libs so slightly risk move if you don't fully check everything first. This often works, though.
As an alternative looks like you can install the mysql server package from webtatic as well, based on their instructions here, that might be a less complex alternative if you are happy to continue relying on their repository:
https://webtatic.com/packages/mysql55/
As far that I know, that error will shows up when you attempt to install packages that already have been installed before. So, you probably should reinstall your mysql.
Source : https://www.centos.org/docs/5/html/5.1/Deployment_Guide/s3-rpm-conflicting-files.html
Currently:
SQLAlchemy installed and working (or at least import v0.8.0b2)
Mysql (v5.5.16)
Distribute (0.6.34)
Oracle mysql-python connector
Python 3.2
Windows 7 32/64 (note that I installed Python 32bits)
The problem is that MySQLdb or Oursql is required and I didn't managed to get any of them working.
Found this but didn't manage to get it working neither.
Edit: If you are aware of an other orm that works with Python3, I'm interested.
I was successful in getting Oracle's MySQL connector for python working with SQLAlchemy on Python 3.3. Your connection string needs to start with "mysql+mysqlconnector://...". After I changed my connection string everything (well, simple things) started working.
The MySQL connector docs can be found here: https://dev.mysql.com/doc/connector-python/en/
The package is up on PyPi: https://pypi.org/project/mysql-connector-python/
Here are the SQLAlchemy docs about using the Python connector: http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqlconnector
For others who arrive here, this should do it:
pip install mysql-connector==2.1.4 # version avoids Protobuf error
URI = 'mysql+mysqlconnector://$USER:$PASS#$HOST/$DB'
I tried Oracle's connection, as suggested by #Brad Campbell, but unfortunately it was extremely slow, much slower than the "real" MySQL-Python connection I had been using with SQLAlchemy on Python 2.
After checking SQLAlchemy themselves,
http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqldb
To use MySQL-Python on Python 3, they recommend a fork of it, mysqlclient,
https://github.com/PyMySQL/mysqlclient-python
It is available via pip with pip install mysqlclient, but there are almost certainly other steps you'll need to do to set it up initially. After that though, I was seeing the performance go back to what I was used to, which was about 5x faster than with Oracle's connector.
I've gotten oursql + SQLAlchemy 0.8.1 + Python 3.3 to work. Building off of LukeCarrier's port, I modified oursql.c to use the correct import levels, and it worked! Try this, and be sure to follow the readme:
https://github.com/clintron/py3k-oursql
You may also need to have the latest version of Cython.