Integrating Apache Superset and Apache Drill - sqlalchemy

I installed Apache Drill through the link in the Drill Documentation. Apache Drill works fine. I also installed and got Apache Superset running using docker. Superset also works totally fine.
But my goal is to integrate Superset and Drill together. The only tutorial I was able to find was a tutorial from Dataist. When following this tutorial they ask us to add a database.
Since I am running both Drill and Superset in my local machine they ask us to type drill+sadrill://localhost:8047/dfs/test?use_ssl=False as the SQLAlchemy URI. They ask us to test the connection by pressing test connection.
When pressing test connection I get an error message as follows.
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:drill.sadrill", "stacktrace": "Traceback (most recent call last):\n File \"/home/superset/superset/views/core.py\", line 1755, in testconn\n engine = database.get_sqla_engine(user_name=username)\n File \"/home/superset/superset/utils/core.py\", line 132, in __call__\n value = self.func(*args, **kwargs)\n File \"/home/superset/superset/models/core.py\", line 911, in get_sqla_engine\n return create_engine(url, **params)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/__init__.py\", line 435, in create_engine\n return strategy.create(*args, **kwargs)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py\", line 61, in create\n entrypoint = u._get_entrypoint()\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/url.py\", line 172, in _get_entrypoint\n cls = registry.load(name)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py\", line 240, in load\n \"Can't load plugin: %s:%s\" % (self.group, name)\nsqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:drill.sadrill\n"}
Can someone figure out why I am getting this error. And also if there are any other tutorials to follow which give a better idea on how to set up Drill and superset.

I have encountered a similar issue while trying to connect elasticsearch. I guess the docker image you used was amancevice/superset. This issue cause because your image is not using the latest SQLAlchemy, SQLAlchemy-Utils packages. Upgrade or reinstall these packages to fix the issue.
To uninstall:
pip uninstall SQLAlchemy
pip uninstall SQLAlchemy-Utils
To install again (latest version):
pip install SQLAlchemy
pip install SQLAlchemy-Utils
I have reported this issue here https://github.com/amancevice/docker-superset/issues/158 maybe it get fixed with the upcoming images.

Related

Unable to call apoc.load.json on Neo4j 3.2.6

I am using Neo4j Community Edition 3.2.6 along with apoc-3.2.3.5-all.jar on Mac. When I try to use any query on the browser that contains call apoc.load.json, I keep getting Failed to invoke procedure apoc.load.json: Caused by: java.lang.NoClassDefFoundError: Could not initialize class apoc.util.JsonUtil. I can confirm that Neo4j is able to find my jar under plugins directory as apoc.load.json is listed when I use CALL dbms.procedures(). Does anyone know what could possibly be wrong that I am getting the error?
I tried the same versions on my Mac, and could not reproduce that error.
If you installed the Neo4j Desktop for OSX, you need to put plugin jar files in 2 folders:
- /Applications/Neo4j\ Community\ Edition\ <version>.app/Contents/Resources/app/plugins
- /Users/<user>/Documents/Neo4j/default.graphdb/plugins

Getting Error while building mysql for python on Mac OS X El Capitan

I read some of the related topics and tried to implement those solutions to my situation but still I get the following error after this command :
(project2_env) Efe-MacBook-Air:MySQL-python-1.2.4b4 efe$ python setup.py build
The Error message is :
Extracting in /var/folders/rv/vbf7xqh1601_xjkrn85w7hp00000gn/T/tmpptpsggg7
Traceback (most recent call last):
File "/Users/efe/virtualenvs/downloads/MySQL-python-1.2.4b4/distribute_setup.py", line 143, in use_setuptools
raise ImportError
ImportError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 7, in <module>
use_setuptools()
File "/Users/efe/virtualenvs/downloads/MySQL-python-1.2.4b4/distribute_setup.py", line 145, in use_setuptools
return _do_download(version, download_base, to_dir, download_delay)
File "/Users/efe/virtualenvs/downloads/MySQL-python-1.2.4b4/distribute_setup.py", line 125, in _do_download
_build_egg(egg, tarball, to_dir)
File "/Users/efe/virtualenvs/downloads/MySQL-python-1.2.4b4/distribute_setup.py", line 99, in _build_egg
_extractall(tar)
File "/Users/efe/virtualenvs/downloads/MySQL-python-1.2.4b4/distribute_setup.py", line 486, in _extractall
self.chown(tarinfo, dirpath)
TypeError: chown() missing 1 required positional argument: 'numeric_owner'
Edit: I reinstalled homebrew, then I run this following command. That is successfully installed.
brew install mysql
However, I cannot still import MySQLdb in python.
As it is more complicated than the previous answer, i won't told you about the custom installation of python mysql.
Found this, the Jude's way :
Install mysql via homebrew, then you can install mysql python via pip.
xcode-select --install
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
//don't know how to install mysql via homebrew, but it should be done here
pip install MySQL-python
https://stackoverflow.com/a/25356073/6660122
Taken from the Python2 doc, still relevant in 3.5 :
"Alternate installation: the user scheme
This scheme is designed to be the most convenient solution for users that don’t have write permission to the global site-packages directory or don’t want to install into it. It is enabled with a simple option: "
python setup.py install --user
Could help in your case.
Notice there are different scheme to use : --home, or --prefix and --exec-prefix, or --install-base and --install-platbase
Finally I found a solution for myself and by myself. I gave up using directly MySQLdb library and instead of that, I installed PyMySQL. By the way I had already installed MySQL Workbench before, so mysql server was already running.
Here is what I do step by step. I hope someone can get benefit from it.
1) Create a virtual environment:
virtualenv virt1
2) Make your virtual environment activated:
source virt1/bin/activate
3) Now, you are in your virtual environment. Install PyMySQL:
pip install PyMySQL
4) Now, try whether your MySQL connection is OK or not with a simple py executable:
#!/usr/bin/python
import pymysql.cursors
# Connect to the database
connection = pymysql.connect(host='localhost',
user='YourDBuserName',
password='YourDBpassword',
db='YourDBname',
charset='utf8',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `id`, `name` FROM `YourTableName` WHERE `id`=%s"
cursor.execute(sql, (1,))
result = cursor.fetchone()
print(result)
finally:
connection.close()
5) You should see that your first row from the table appears on the screen.
Note: The part "import pymysql.cursors" could be tricky. First, I wrote as "import PyMySQL.cursors" and that didn't work!

I think OSX-gcc-installer broke some things and I'm not sure what to do

Long story short is I foolishly installed OSX-gcc-installer about a month ago and at first it prevented me, I believe, from installing Ruby gems on my machine. With the help of someone I did get this one issue fixed. Going through other the other posts on this website I was not able to find the solution that I'm looking for, so I decided to post here.
When I run brew doctor I get the following:
albys-mbp:folder alby$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!
Warning: You have an outdated version of /usr/bin/install_name_tool installed.
This will cause binary package installations to fail.
This can happen if you install osx-gcc-installer or RailsInstaller.
To restore it, you must reinstall OS X or restore the binary from
the OS packages.
Warning: Broken symlinks were found. Remove them with `brew prune`:
/usr/local/bin/aclocal
/usr/local/bin/aclocal-1.14
/usr/local/bin/autoconf
/usr/local/bin/autoheader
/usr/local/bin/autom4te
/usr/local/bin/automake
/usr/local/bin/automake-1.14
/usr/local/bin/autoreconf
/usr/local/bin/autoscan
/usr/local/bin/autoupdate
/usr/local/bin/cscope
/usr/local/bin/erb
/usr/local/bin/gem
/usr/local/bin/glibtool
/usr/local/bin/glibtoolize
/usr/local/bin/gpg-error
/usr/local/bin/gpg-error-config
/usr/local/bin/ifnames
/usr/local/bin/irb
/usr/local/bin/ksba-config
/usr/local/bin/ocs
/usr/local/bin/pkg-config
/usr/local/bin/rake
/usr/local/bin/rdoc
/usr/local/bin/ri
/usr/local/bin/ruby
/usr/local/bin/testrb
/usr/local/include/gpg-error.h
/usr/local/include/ksba.h
/usr/local/include/libltdl
/usr/local/include/ltdl.h
/usr/local/include/ruby-2.1.0
/usr/local/include/yaml.h
/usr/local/lib/libgpg-error.0.dylib
/usr/local/lib/libgpg-error.dylib
/usr/local/lib/libksba.8.dylib
/usr/local/lib/libksba.dylib
/usr/local/lib/libltdl.7.dylib
/usr/local/lib/libltdl.a
/usr/local/lib/libltdl.dylib
/usr/local/lib/libruby.2.1.0-static.a
/usr/local/lib/libruby.2.1.0.dylib
/usr/local/lib/libruby.2.1.dylib
/usr/local/lib/libruby.dylib
/usr/local/lib/libyaml-0.2.dylib
/usr/local/lib/libyaml.a
/usr/local/lib/libyaml.dylib
/usr/local/lib/pkgconfig/ruby-2.1.pc
/usr/local/lib/pkgconfig/yaml-0.1.pc
/usr/local/lib/ruby/2.1.0
/usr/local/lib/ruby/gems
/usr/local/share/aclocal/README
/usr/local/share/aclocal/argz.m4
/usr/local/share/aclocal/dirlist
/usr/local/share/aclocal/gpg-error.m4
/usr/local/share/aclocal/ksba.m4
/usr/local/share/aclocal/libtool.m4
/usr/local/share/aclocal/ltdl.m4
/usr/local/share/aclocal/ltoptions.m4
/usr/local/share/aclocal/ltsugar.m4
/usr/local/share/aclocal/ltversion.m4
/usr/local/share/aclocal/lt~obsolete.m4
/usr/local/share/aclocal/pkg.m4
/usr/local/share/aclocal-1.14
/usr/local/share/autoconf
/usr/local/share/automake-1.14
/usr/local/share/common-lisp
/usr/local/share/doc/automake
/usr/local/share/doc/pkg-config/pkg-config-guide.html
/usr/local/share/emacs
/usr/local/share/info/gpgrt.info
/usr/local/share/info/ksba.info
/usr/local/share/info/libtool.info
/usr/local/share/info/libtool.info-1
/usr/local/share/info/libtool.info-2
/usr/local/share/libtool
/usr/local/share/man/man1/aclocal-1.14.1
/usr/local/share/man/man1/aclocal.1
/usr/local/share/man/man1/autoconf.1
/usr/local/share/man/man1/autoheader.1
/usr/local/share/man/man1/autom4te.1
/usr/local/share/man/man1/automake-1.14.1
/usr/local/share/man/man1/automake.1
/usr/local/share/man/man1/autoreconf.1
/usr/local/share/man/man1/autoscan.1
/usr/local/share/man/man1/autoupdate.1
/usr/local/share/man/man1/config.guess.1
/usr/local/share/man/man1/config.sub.1
/usr/local/share/man/man1/cscope.1
/usr/local/share/man/man1/erb.1
/usr/local/share/man/man1/glibtool.1
/usr/local/share/man/man1/glibtoolize.1
/usr/local/share/man/man1/gpg-error-config.1
/usr/local/share/man/man1/ifnames.1
/usr/local/share/man/man1/irb.1
/usr/local/share/man/man1/pkg-config.1
/usr/local/share/man/man1/rake.1
/usr/local/share/man/man1/ri.1
/usr/local/share/man/man1/ruby.1
/usr/local/Library/LinkedKegs/autoconf
/usr/local/Library/LinkedKegs/automake
/usr/local/Library/LinkedKegs/cscope
/usr/local/Library/LinkedKegs/libgpg-error
/usr/local/Library/LinkedKegs/libksba
/usr/local/Library/LinkedKegs/libtool
/usr/local/Library/LinkedKegs/libyaml
/usr/local/Library/LinkedKegs/pkg-config
/usr/local/Library/LinkedKegs/ruby
Warning: You seem to have osx-gcc-installer installed.
Homebrew doesn't support osx-gcc-installer. It causes many builds to fail and
is an unlicensed distribution of really old Xcode files.
Please run `xcode-select --install` to install the CLT.
Warning: Some installed formula are missing dependencies.
You should `brew install` the missing dependencies:
brew install openssl
Run `brew missing` for more details.
alby-mbp:folder alby$
As you can also see above, I am missing openssl. This is because I uninstalled it and attempted to reinstall it thinking that this would help, but I was not able to reinstall. Here is what I get when I try to do brew install openssl:
albys-mbp:folder alby$ brew install openssl
Warning: You seem to have osx-gcc-installer installed.
Homebrew doesn't support osx-gcc-installer. It causes many builds to fail and
is an unlicensed distribution of really old Xcode files.
Please run `xcode-select --install` to install the CLT.
Warning: You have an outdated version of /usr/bin/install_name_tool installed.
This will cause binary package installations to fail.
This can happen if you install osx-gcc-installer or RailsInstaller.
To restore it, you must reinstall OS X or restore the binary from
the OS packages.
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/openssl-1.0.1j_1.mavericks.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/openssl-1.0.1j_1.mavericks.bottle.tar.gz
Error: SHA1 mismatch
Expected: 65e125a4777eb6dfb63f01a18f724246123dd79e
Actual: eac5e2d21af64224fc533ebb793b99a2aea434c7
Archive: /Library/Caches/Homebrew/openssl-1.0.1j_1.mavericks.bottle.tar.gz
To retry an incomplete download, remove the file above.
Warning: Bottle installation failed: building from source.
==> Installing openssl dependency: makedepend
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/makedepend-1.0.5.mavericks.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/makedepend-1.0.5.mavericks.bottle.tar.gz
Error: SHA1 mismatch
Expected: 83db1daee01e4eb752c711934eb88850b3ee70d6
Actual: eac5e2d21af64224fc533ebb793b99a2aea434c7
Archive: /Library/Caches/Homebrew/makedepend-1.0.5.mavericks.bottle.tar.gz
To retry an incomplete download, remove the file above.
Warning: Bottle installation failed: building from source.
Error: /usr/local/opt/pkg-config not present or broken
Please reinstall pkg-config. Sorry :(
albys-mbp:folder alby$
Another issue is that when I try and run mysql in bash, I get the following:
albys-mbp:folder alby$ mysql
dyld: Library not loaded: ##HOMEBREW_PREFIX##/opt/openssl/lib/libssl.1.0.0.dylib
Referenced from: /usr/local/bin/mysql
Reason: image not found
Trace/BPT trap: 5
albys-mbp:folder alby$
I am mostly new to programming and am not sure where to begin to solve this issue. I was able to backup my HD to an external HD, but this is after installing the gcc-installer which overwrote my Command Line Tools. So even if I were to do a reinstall, would that even work? I've never had to reinstall my OS X before; would I be able to pick and choose files from the external HD or is it all restored?
Also, I have OS X 10.9.
Thanks!

xcode: Mysql connector library not work on iphone5 (armv7s) ,any solution?

I built my App using Mysql Connector/C to connect a remote Mysql database, its works fine on the simulator (no errors, no warnings) but when i try to run it on my device (iphone5) i got this error:
No architectures to compile for (ARCHS=armv7 armv7s, VALID_ARCHS=armv7 armv7s)
i tried -as in some answers- to change setting (Architectures - Build Active Architectures- Valid Architectures) but the error still, only when i change the setting (Architectures & Valid Architectures) to "armv6" its build without error but many warnings appears says:
warning: no rule to process file '(my App dir)/main.m' of type sourcecode.c.objc for architecture armv6
and also for all .m files, when i tried to start the App i got message:
Xcode cannot run using selected device
I know that the Connector library need to update , but are there any solution ?
your need compile the connector lib in xcode for iOS (armv6, armv7, armv7s, i386), then use lipo tool to combine output libs.
direct connect your mysql in app is not safe, a suggest way is setup a Apache+PHP+MySQL server, then on iPhone useing ASIHTTPRequest to connect your server.

mysql error for module DBD::mysql: libmysqlclient.so.15:

i am using mysql for orthomcl software. when i use the command
orthomclLoadBlast my_orthomcl/orthomcl.config my_orthomcl/similarSequence.txt
it gives this error
Can't load '/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/mysql/mysql.so' for module DBD::mysql: libmysqlclient.so.15: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230, line 5. at /storage/home/cdac/OrthoMCL/orthomclSoftwarev2.0.8/bin/../lib/perl/OrthoMCLEngine/Main/Base.pm line48
Compilation failed in require at /storage/home/cdac/OrthoMCL/orthomclSoftware-v2.0.8/bin/../lib/perl/OrthoMCLEngine/Main/Base.pm line 48, <F> line 5.
How to solve this problem ??
You could install a local perl in your home and you could leave vendor perl untouched.
You could use perlbrew or local::lib to maintain an up to date perl version without modifying the system installed perl.