What STANDARD module should I use with Python 3 to talk to MySQL? - mysql

This earlier SO question suggests mysql.connector, which is what I am actually using. However, as noted in a comment to the accepted answer in the aforementioned question, this is not a standard python module.
My problem: whenever my ISP upgrades to a newer Python version, I have to notice the crashes and bug them to install mysql.connector for me.
I could explicitly call for an older version of Python 3.x. However, all other things being equal, I'd like to use the latest Python, and it seems like MySQL is just such a standard of standards that there really should be a standard Python 3 library for it. Is there? I'm not doing anything fancy with the database (or with python), so basic functionality is o.k.

Just use a venv and isolate yourself from ISP python upgrades:
pyvenv ./mysqlvenv
./mysqlvenv/bin/pip install mysql-connector-python --allow-external mysql-connector-python
install your other dependencies using ./mysqlvenv/bin/pip install foo and Bob's your uncle.

Related

AllenNLP Server: pip is looking at multiple versions of each package

Within my Conda environment with Python 3.6.9, I've installed AllenNLP 9.2.0. I tried to install AllenNLP Server following the instruction from https://github.com/allenai/allennlp-server by running pip install --editable .
However, the installation procedure never finished as the compatibility checks with several modules, e.g. pip is looking at multiple versions of tqdm to determine which version is compatible with other requirements. This could take a while. Collecting tqdm>=4.19
Does anybody know what happens here? Should I add more restrictions to steup.py in AllenNLP server? However, there is any code included in such file.
Thanks a lot for your help.
I just tried it with AllenNLP 2.0.1 (the latest), and while it takes a long time, it does eventually resolve the packages.
That said, I would recommend two things:
Use Python 3.8 instead.
If it still doesn't work, specify a version of tqdm tightly in the requirements. My version automatically picked tqdm==4.56.2, just or reference.

"Fatal error: 'EXTERN.h' file not found" while installing Perl modules

While trying to install Perl modules like JSON::XS or YAML::XS, i receive the same error:
XS.xs:1:10: fatal error: 'EXTERN.h' file not found
I use MacBook, xCode is up to date, everything else that could help is up to date too.
Since OS X El Capitan, Apple introduced System Integrity Protection which restricts writing to /usr/lib /usr/bin and other sensitive directories (even to root or sudo user) that are used by the installation of Perl bundled with the Operating System. This can cause issues when it comes to installing new modules and also if trying to install XS modules ( those linked to external C libraries ).
For this reason you should not consider the default Perl installation as a working development environment, especially if you are installing custom modules.
Check out this thread on PM and others. I had since El-Capitan managed to solve this before by manually building from tarball and adding a few params or environment variables to set the paths believing that it would be best to retain use of the system Perl but this is not the way to go. This makes your environment difficult to build but also brittle and sensitive to OS updates that may either break things in many different ways.
The best practice seems to be starting with a Perl using brew install perl and work in this environment, remembering to setup your bash_profile as directed by the installer.
Also worth remembering to do a brew link perl. If you receive warnings about this clobbering what looks like system Perl libraries don't worry - these are likely modules that were installed by you over the top and it will cause you less trouble to link over these. If you have concerns, make a note of which module installs will be cleared and re-install them once your environment is configured ( ie your module installer approach is configured using cpanm or sticking with the old perl -MCPAN -e shell etc)
This new Perl setup from brew eliminates the need to continuing running sudo which adds another layer of things that can go wrong as environment variables don't follow through and permission conflicts arise etc.
Finally to simplify package/module installation I suggest doing a brew install cpanminus. If you had previously already installed this, you can ensure the paths etc are configured by doing a brew reinstall cpanminus
If you want to take it another step further then you can install perlbrew as well which will give you the ability to run multiple versions of Perl as your user and configure these with their own libs and modules which can be very useful particularly if aligning with your production environment for testing etc.
One problem you may face if moving from system Perl to this kind of approach is needing to deal with any hangovers from installing things with sudo. It wis worth taking a little time to get all this set up right though and your issues going forward will be greatly reduced and you won't be left with that nagging feeling that you don't want to change anything for fear of it all breaking.
I have also come across a Perl Blog Article that suggests a fix for XS issues with perlbrew on Mojave
This Gist described updating your cpan shell install root though this shouldn't be necessary unless your cpan is stuck in an old config after taking steps above.
I've also raised this as a new issue on PerlMonks
After reading https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035624 and installing the Additional headers via
sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
I successfully compiled without the missing 'EXTERN.h' error
In order to follow the common advice I also tried with Perlbrew to install a dedicated development version of Perl. Especially with the advice in mind First, do not use the system Perl on MacOS. The installed version is for Apple, not for you (see the discussion here: https://www.perlmonks.org/?node_id=1224727).
Unfortunately, the following error occurred:
Test Summary Report
-------------------
porting/libperl.t (Wstat: 65280 Tests: 35 Failed: 0)
Non-zero exit status: 255
Parse errors: No plan found in TAP output
Files=2653, Tests=1217766, 708 wallclock secs (52.74 usr 9.40 sys + 395.38 cusr 49.90 csys = 507.42 CPU)
Result: FAIL
make: *** [test_harness] Error 1
##### Brew Failed #####
Therefore, I decided to install it the following way (and not following the advice due to the error).
Even after having the above mentioned macOS SDK headers already installed on Catalina (macOS 10.15.2) it didn't work for me. I faced the issue during the installation of the Perl module Mac-SystemDirectory-0.13. The following steps (by identifying the missing file in hope of having a more generic approach for more or less equivalent issues) did the trick:
Locate the header file (in this case EXTERN.h)
sudo find /Library -type f -name EXTERN.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/EXTERN.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Perl/5.28/darwin-thread-multi-2level/CORE/EXTERN.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/EXTERN.h
Ensure the installed Perl version (here 5.18) match the header file:
perl -v | grep version
This is perl 5, version 18, subversion 4 (v5.18.4) built for darwin-thread-multi-2level
Export the path for the C-Compiler (note MacOSX10.15.sdk for Catalina and Perl Version 5.18)
export CPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE
Invoke the Makefile.PL with perl
perl Makefile.PL
BTW — For anybody who's still struggling with this, my workaround was:
bash% module="Sub::Util" # For example
bash% cpanm --configure-args="INC=-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE" "$module"
Please try this
CPATH=$(dirname $(find /usr/local/Cellar/ -name EXTERN.h)) cpan JSON::XS
For Big Sur and perl 5.30, EXTERN.h is at /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE
I'm trying to upgrade CPAN itself and got that error. But I have /usr/bin/cpan and I can't write there so I have to tweak it to write the updated version to /usr/local/bin/cpan.
No promises, but yum install perl-devel worked for me.
As #huyz has helpfully pointed out, if you hit this error on a Mac, you don't have this option, even though this is probably your issue, and you need to follow one of the above methods of getting a version of Perl that isn't missing important chunks, as per other answers.
But if, dear reader, you hit this error on a linux host, as I did, then this might be an option for you.
Building on what E Lisse suggested, you might also have luck looking in
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/
For example:
CPATH=$(dirname $(find /System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/ -name EXTERN.h)) cpan JSON::XS
You could also find where EXTERN.h is located and add that to your shell by default, e.g. in your .bashrc or .zshrc file:
export CPATH=/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/

Are there any code complexity metering tools that work well with ES6?

My project is ES6 (webpack + babel). Plato is great, but doesn't work with ES6 yet. I took a look at escomplex and the series of related packages, but it looks like the author is no longer maintaining this and it seems like it doesn't support ES6 anyways.
I'm using the ESLint complexity rule to at least warn me if there's an issue (it's set to warn at 6), but it's not the same as a pretty report.
Any suggestions?
There is a fork created by deedubs that uses espree instead esprima. I've installed it and it runs on ES6 styled code like a charm, in order to install the fork version instead the default distro execute the following command:
npm install -g git+https://github.com/deedubs/es6-plato.git
Whenever you want to uninstall it just do it like you would do it with vanilla plato install by writing
npm uninstall plato
Here's a screenshot for a file report based on a ES6 code after installing the Plato forked version
It's been about 3 weeks since you asked the question so I am not sure if this will be still useful for you but I hope it can help others with the same problem we were facing on using this amazing tool.
I published a version of plato that does this here: https://www.npmjs.com/package/es6-plato
updates lodash and dependencies
fixes some common bugs in plato
properly parses es6
this includes classes
npm install --save es6-plato
read the docs to use it with gulp, or via the js api.

MonoDevelop ValaBinding install failed

I installed monodevelop in ubuntu 13.04 from git with the help of README.md, but I received an error
checking for LIBVALA... no
configure: error: Package requirements (libvala-0.12 >= 0.12) were not met:
No package 'libvala-0.12' found
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.
Alternatively, you may set the environment variables LIBVALA_CFLAGS
and LIBVALA_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
I have installed libvala-0.20-0 and libvala-0.20-dev by apt-get.
I have googled for this question and found some links,but helpless.
my mono was at version 3.6.1.
THX FOR HELP.
update 2014-06-19
I found the config.log file and see the error
configure:4075: checking for LIBVALA
configure:4084: $PKG_CONFIG --exists --print-errors "libvala-0.12 >= $LIBVALA_REQUIRED_VERSION"
Package libvala-0.12 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libvala-0.12.pc'
to the PKG_CONFIG_PATH environment variable
Although I have installed libvala-0.20-0 and libvala-0.20-dev, I doesn't own a libvala-0.**.pc file.
What can I do the next?
I have resolved thest problem by finding a libvala-0.12.pc file :)
While the Vala language, valac, and most of the bindings distributed with Vala are meant to be API stable, libvala is not. libvala-0.20 provides a very different API/ABI from libvala-0.12, and apparently MonoDevelop's Vala plugin hasn't been updated since Vala 0.12.
If you don't want the Vala plugin, I'm sure MonoDevelop provides a way to disable it (probably passing --disable-vala or something similar to ./configure). If you need the Vala plugin, though, you'll have to install libvala-0.12 or update the plugin to use a newer version of libvala.
now the problem is solved.
I used cn.archive.ubuntu.com/ubuntu for my apt-get.
These's no libvala-**.pc file in the package.
So whatever version I installed by apt-get, I can not get a libvala-*.pc file.
I searched in google for libvala-12.0.pc and changed my source to cz.archive.ubuntu.com/ubuntu, then I got that file.
Now I can run my monodevelop with valabinding.
Thx for #nemequ , I didn't use a higher libvala, so I don't know if it will be different when using a libvala-hight-than-12.0.pc file.
Hope these is usefull for others.
If you don't want to install vavabind after configure once ,just use ./configure --select, it's written in README. My bad :(

Python 3 and mysql through SQLAlchemy

Currently:
SQLAlchemy installed and working (or at least import v0.8.0b2)
Mysql (v5.5.16)
Distribute (0.6.34)
Oracle mysql-python connector
Python 3.2
Windows 7 32/64 (note that I installed Python 32bits)
The problem is that MySQLdb or Oursql is required and I didn't managed to get any of them working.
Found this but didn't manage to get it working neither.
Edit: If you are aware of an other orm that works with Python3, I'm interested.
I was successful in getting Oracle's MySQL connector for python working with SQLAlchemy on Python 3.3. Your connection string needs to start with "mysql+mysqlconnector://...". After I changed my connection string everything (well, simple things) started working.
The MySQL connector docs can be found here: https://dev.mysql.com/doc/connector-python/en/
The package is up on PyPi: https://pypi.org/project/mysql-connector-python/
Here are the SQLAlchemy docs about using the Python connector: http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqlconnector
For others who arrive here, this should do it:
pip install mysql-connector==2.1.4 # version avoids Protobuf error
URI = 'mysql+mysqlconnector://$USER:$PASS#$HOST/$DB'
I tried Oracle's connection, as suggested by #Brad Campbell, but unfortunately it was extremely slow, much slower than the "real" MySQL-Python connection I had been using with SQLAlchemy on Python 2.
After checking SQLAlchemy themselves,
http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqldb
To use MySQL-Python on Python 3, they recommend a fork of it, mysqlclient,
https://github.com/PyMySQL/mysqlclient-python
It is available via pip with pip install mysqlclient, but there are almost certainly other steps you'll need to do to set it up initially. After that though, I was seeing the performance go back to what I was used to, which was about 5x faster than with Oracle's connector.
I've gotten oursql + SQLAlchemy 0.8.1 + Python 3.3 to work. Building off of LukeCarrier's port, I modified oursql.c to use the correct import levels, and it worked! Try this, and be sure to follow the readme:
https://github.com/clintron/py3k-oursql
You may also need to have the latest version of Cython.