Using MySQL with dev_appserver (Google App Engine) and Google SQL Service - mysql

I am using Google App Engine and Google SQL Service, and would like to use the option
dev_appserver.py --mysql_user=username myapp
in order to use a local MySQL database for development purposes, and SQL Service in prod environment.
I have MySQLdb installed and working:
/usr/bin>>python
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>>
However, when I run the dev_appserver command, I get the following error:
zipimporter('/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg', 'MySQLdb/')
ERROR 2011-09-12 09:34:53,541 rdbms_mysqldb.py:90] The rdbms API is not available because the MySQLdb library could not be loaded.
I have verified that the MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg is in the location (/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/) and that it is accessible.
Any idea what might be causing this error?
Full stacktrace provided below for information:
/Users/eddieboyd>>dev_appserver.py --debug --mysql_user=mysql eddiehelloworld
Warning: You are using a Python runtime (2.7) that is more recent than the production runtime environment (2.5). Your application may use features that are not available in the production environment and may not work correctly when deployed to production.
INFO 2011-09-12 09:34:53,290 appengine_rpc.py:159] Server: appengine.google.com
INFO 2011-09-12 09:34:53,304 appcfg.py:449] Checking for updates to the SDK.
DEBUG 2011-09-12 09:34:53,305 appengine_rpc.py:364] Sending HTTPS request:
POST /api/updatecheck?release=1.5.3&timestamp=1311108376&api_versions=%5B%271%27%5D HTTPS/1.1
Host: appengine.google.com
X-appcfg-api-version: 1
Content-type: application/octet-stream
User-agent: appcfg_py/1.5.3 Darwin/10.8.0 Python/2.7.2.final.0
INFO 2011-09-12 09:34:53,465 appcfg.py:466] The SDK is up to date.
WARNING 2011-09-12 09:34:53,465 datastore_file_stub.py:512] Could not read datastore data from /var/folders/++/++71vE++6+0++4RjPqRgNE+0Eyo/-Tmp-/dev_appserver.datastore
INFO 2011-09-12 09:34:53,478 py_zipimport.py:148] zipimporter('/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg', 'MySQLdb/')
ERROR 2011-09-12 09:34:53,541 rdbms_mysqldb.py:90] The rdbms API is not available because the MySQLdb library could not be loaded.
ERROR 2011-09-12 09:34:53,541 dev_appserver_main.py:638] <type 'exceptions.NotImplementedError'>: Unable to find the MySQLdb library. Please see the SDK documentation for installation instructions.
DEBUG 2011-09-12 09:34:53,543 dev_appserver_main.py:640] Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_main.py", line 635, in main
dev_appserver.SetupStubs(appinfo.application, **option_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 4668, in SetupStubs
rdbms_mysqldb.connect(database='')
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/rdbms_mysqldb.py", line 96, in connect
'Unable to find the MySQLdb library. Please see the SDK '
NotImplementedError: Unable to find the MySQLdb library. Please see the SDK documentation for installation instructions.

Poor man's solution.
I added
"import MySQLdb"
in dev_appserver.py.
It works now.

Ensure to use the latest version of the AppEngine SDK which currently is the version 1.6.2.
After that install the package for python mysql support: python-mysqldb.

Maybe your system does not have the MySQL module required. On Fedora 16 64 bits, I solved this by installing the MySQL python module:
yum install MySQL-python

Related

Cannot install jinja2 with jython pip?

I am using Windows 7 with Jython 2.7.0. I am able to run jython --version and pip --version in the bin directory and get back what I expect, 2.7.0 and 1.6, with a bunch of extra information.
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\myUser>cd ../..
C:\>cd jython2.7.0/bin
C:\jython2.7.0\bin>pip --version
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/C:/jy
thon2.7.0/jython.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.Py
SystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect
ive access operations
WARNING: All illegal access operations will be denied in a future release
Feb 16, 2021 1:37:50 PM org.python.netty.util.internal.PlatformDependent <clinit
>
INFO: Your platform does not provide complete low-level API for accessing direct
buffers reliably. Unless explicitly requested, heap buffer will always be prefe
rred to avoid potential system unstability.
pip 1.6.dev1 from C:\jython2.7.0\Lib\site-packages (python 2.7)
I try to install jinja2 with pip install jinja2 and have also tried it captialized since I saw that as well pip install Jinja2 and get the following
C:\jython2.7.0\bin>pip install jinja2
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/C:/jy
thon2.7.0/jython.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.Py
SystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflect
ive access operations
WARNING: All illegal access operations will be denied in a future release
Feb 16, 2021 1:44:35 PM org.python.netty.util.internal.PlatformDependent <clinit
>
INFO: Your platform does not provide complete low-level API for accessing direct
buffers reliably. Unless explicitly requested, heap buffer will always be prefe
rred to avoid potential system unstability.
Downloading/unpacking jinja2
Got this failure javax.net.ssl.SSLException: Tag mismatch! during SSL handshake
(<_realsocket at 0x215 type=client open_count=1 channel=[id: 0x53d8a502, /192.16
8.1.59:64975 => pypi.python.org/199.232.36.223:443] timeout=15.0>)
Got this failure javax.net.ssl.SSLException: Tag mismatch! during SSL handshake
(<_realsocket at 0x216 type=client open_count=1 channel=[id: 0x3d4b1e67, /192.16
8.1.59:64976 :> pypi.python.org/199.232.36.223:443] timeout=15.0>)
←[31m Cannot fetch index base URL https://pypi.python.org/simple/
←[0mGot this failure javax.net.ssl.SSLException: Tag mismatch! during SSL handsh
ake (<_realsocket at 0x217 type=client open_count=1 channel=[id: 0x5f10319d, /19
2.168.1.59:64977 :> pypi.python.org/199.232.36.223:443] timeout=15.0>)
←[31m Could not find any downloads that satisfy the requirement jinja2
←[0mCleaning up...
←[31mNo distributions at all found for jinja2
←[0m←[31mStoring debug log for failure in C:\Users\bkarabinchak\.pip\pip.log
←[0m
This line ←[31mNo distributions at all found for jinja2 makes me think there is no Jinja2 for jython, nor Cheetah which I also tried to install. I tried those as they were referenced here as jython HTML Templaters.
Can someone either 1) Help me troubleshoot installing jinja2 or 2) Recommend a HTML Template package that does exist for jython if jinja2/Cheetah do no work?

Integrating Apache Superset and Apache Drill

I installed Apache Drill through the link in the Drill Documentation. Apache Drill works fine. I also installed and got Apache Superset running using docker. Superset also works totally fine.
But my goal is to integrate Superset and Drill together. The only tutorial I was able to find was a tutorial from Dataist. When following this tutorial they ask us to add a database.
Since I am running both Drill and Superset in my local machine they ask us to type drill+sadrill://localhost:8047/dfs/test?use_ssl=False as the SQLAlchemy URI. They ask us to test the connection by pressing test connection.
When pressing test connection I get an error message as follows.
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:drill.sadrill", "stacktrace": "Traceback (most recent call last):\n File \"/home/superset/superset/views/core.py\", line 1755, in testconn\n engine = database.get_sqla_engine(user_name=username)\n File \"/home/superset/superset/utils/core.py\", line 132, in __call__\n value = self.func(*args, **kwargs)\n File \"/home/superset/superset/models/core.py\", line 911, in get_sqla_engine\n return create_engine(url, **params)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/__init__.py\", line 435, in create_engine\n return strategy.create(*args, **kwargs)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py\", line 61, in create\n entrypoint = u._get_entrypoint()\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/url.py\", line 172, in _get_entrypoint\n cls = registry.load(name)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py\", line 240, in load\n \"Can't load plugin: %s:%s\" % (self.group, name)\nsqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:drill.sadrill\n"}
Can someone figure out why I am getting this error. And also if there are any other tutorials to follow which give a better idea on how to set up Drill and superset.
I have encountered a similar issue while trying to connect elasticsearch. I guess the docker image you used was amancevice/superset. This issue cause because your image is not using the latest SQLAlchemy, SQLAlchemy-Utils packages. Upgrade or reinstall these packages to fix the issue.
To uninstall:
pip uninstall SQLAlchemy
pip uninstall SQLAlchemy-Utils
To install again (latest version):
pip install SQLAlchemy
pip install SQLAlchemy-Utils
I have reported this issue here https://github.com/amancevice/docker-superset/issues/158 maybe it get fixed with the upcoming images.

Building Custom Platform on ElasticBeanstalk Packer Error

I am trying to build a custom platform using the provided example (NodePlatform_Ubuntu) at
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/custom-platforms.html#custom-platforms-pda
I installed EB CLI (my OS is MacOSX El Captain):
➜ custom-platform ebp --version
EB CLI 3.10.1 (Python 2.7.1)
For some reason, EB CLI insists on using Python 2.7.1 instead of the already installed Python 3.x (python3 command works), but this shouldn't matter according to http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install.html it supports Python 2.7
While "ebp init" works perfectly. the "ebp create" fails.
➜ custom-platform ebp create
Creating application version archive "app-170427_145319".
Uploading custom-platform/app-170427_145319.zip to S3. This may take a while.
Upload Complete.
Note: An environment called 'eb-custom-platform-builder-packer' has been created in order to build your application. This environment will not automatically be terminated and it does have a cost associated with it. Once your platform creation has completed you can terminate this builder environment using the command 'eb terminate'.
INFO: createPlatform is starting.
INFO: Initiated platform version creation for 'custom-platform/1.0.2'.
INFO: Creating Packer builder environment 'eb-custom-platform-builder-packer'.
ERROR: Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.
INFO: Failed to create platform version 'custom-platform/1.0.2'.
This is the error:
ERROR: Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.
More logs (debug mode):
2017-04-27 14:19:55,698 (DEBUG) ebcli.lib.aws : Response: {u'Events': [{u'PlatformArn': 'arn:aws:elasticbeanstalk:us-west-2:107875334514:platform/custom-platform/1.0.1', u'Message': "Failed to create platform version 'custom-platform/1.0.1'.", u'Severity': 'INFO', u'EventDate': datetime.datetime(2017, 4, 27, 11, 19, 52, 84000, tzinfo=tzutc()), u'RequestId': 'f762956d-2b3a-11e7-8cd7-c96ae1e26915'}, {u'PlatformArn': 'arn:aws:elasticbeanstalk:us-west-2:107875334514:platform/custom-platform/1.0.1', u'Message': 'Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.', u'Severity': 'ERROR', u'EventDate': datetime.datetime(2017, 4, 27, 11, 19, 51, 861000, tzinfo=tzutc()), u'RequestId': 'f762956d-2b3a-11e7-8cd7-c96ae1e26915'}], 'ResponseMetadata': {'date': 'Thu, 27 Apr 2017 11:19:52 GMT', 'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '6f7dcea9-2b3b-11e7-8cd7-c96ae1e26915'}}
ERROR: Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.
INFO: Failed to create platform version 'custom-platform/1.0.1'.
Note that the app-170427_145319.zip is successfully uploaded to S3.
Any idea what's wrong?
It is was failing with instance profile(role) aws-elasticbeanstalk-custom-platform-ec2-role not getting necessary role inline policy attached to it.
Because of lot of modifications and changes, apparently due to an EB CLI bug, the instance profile got corrupted and was not taking InstanceProfile ARN when tried to create a profile with same name "aws-elasticbeanstalk-custom-platform-ec2-role".
So, we went ahead and created a new one named "custom-platform" and attached necessary inline policy permissions I have mentioned in the case.
Once that IAM issues are sorted, we went onto packer bundle,did "ebp init" and then ran "ebp create" but with -ip flag and used this instance profile custom-platform, as in
ebp create -ip custom-platform

Asterisk res_config_mysql module is missing

I have installed asterisk latest package and using mysql DB for realtime simulation. But in the CLI i am getting the error:
config.c:2444 find_engine: Realtime mapping for 'sippeers' found to engine 'mysql', but the engine is not available
and while executing command MODULE LOAD RES_CONFIG_MYSQL i am getting the following message.
loader.c:918 load_resource: Module 'RES_CONFIG_MYSQL' could not be loaded.
My machine is with linux mint 16. 64 bit.
If you have version <1.4 you have download asterisk-addons and compile it too
Otherwise you have do
make menuconfig
before make and select mysql addon

Using SQL Alchemy and pyodbc with IronPython 2.6.1

I'm using IronPython and the clr module to retrieve SQL Server information via SMO. I'd like to retrieve/store this data in a SQL Server database using SQL Alchemy, but am having some trouble loading the pyodbc module.
Here's the setup:
IronPython 2.6.1 (installed at D:\Program Files\IronPython)
CPython 2.6.5 (installed at D:\Python26)
SQL Alchemy 0.6.1 (installed at D:\Python26\Lib\site-packages\sqlalchemy)
pyodbc 2.1.7 (installed at D:\Python26\Lib\site-packages)
I have these entries in the IronPython site.py to import CPython standard and third-party libraries:
# Add CPython standard libs and DLLs
import sys
sys.path.append(r"D:\Python26\Lib")
sys.path.append(r"D:\Python26\DLLs")
sys.path.append(r"D:\Python26\lib-tk")
sys.path.append(r"D:\Python26")
# Add CPython third-party libs
sys.path.append(r"D:\Python26\Lib\site-packages")
# sqlite3
sys.path.append(r"D:\Python26\Lib\sqlite3")
# Add SQL Server SMO
sys.path.append(r"D:\Program Files\Microsoft SQL Server\100\SDK\Assemblies")
import clr
clr.AddReferenceToFile('Microsoft.SqlServer.Smo.dll')
clr.AddReferenceToFile('Microsoft.SqlServer.SqlEnum.dll')
clr.AddReferenceToFile('Microsoft.SqlServer.ConnectionInfo.dll')
SQL Alchemy imports OK in IronPython, put I receive this error message when trying to connect to SQL Server:
IronPython 2.6.1 (2.6.10920.0) on .NET 2.0.50727.3607
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlalchemy
>>> e = sqlalchemy.MetaData("mssql://")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Python26\Lib\site-packages\sqlalchemy\schema.py", line 1780, in __init__
File "D:\Python26\Lib\site-packages\sqlalchemy\schema.py", line 1828, in _bind_to
File "D:\Python26\Lib\site-packages\sqlalchemy\engine\__init__.py", line 241, in create_engine
File "D:\Python26\Lib\site-packages\sqlalchemy\engine\strategies.py", line 60, in create
File "D:\Python26\Lib\site-packages\sqlalchemy\connectors\pyodbc.py", line 29, in dbapi
ImportError: No module named pyodbc
This code works just fine in CPython, but it looks like the pyodbc module isn't accessible from IronPython.
Any suggestions? I realize that this may not be the best way to approach the problem, so I'm open to tackling this a different way. Just wanted to get some experience with using SQL Alchemy and pyodbc.
its very likely that pyodbc is not compatible with IronPython, as it was designed for usage with cPython.
IronPython certainly has some kind of ODBC (actually, ADO.net seems like where its at) compatibility built into it, but a DBAPI would be the most direct way to get SQLAlchemy working with it.
So here's some MS-specific non-DBAPI example: http://www.ironpython.info/index.php/Accessing_SQL_Server
someone talking about DBAPI in 2006: http://hex-dump.blogspot.com/2006/10/ironpython-and-adonet-part-2.html
something a little more recent: http://bitbucket.org/jdhardy/adonet-dbapi/
It says something that MS pours however much money into IronPython but zero into a compliant DBAPI driver.
You could try using SQLAlchemy's adodbapi support instead; the latest version of adodbapi (2.3.0) supports IronPython.
You should only have to make sure the adodbapi package is on sys.path, and then use 'mssql+adodbapi://' instead of 'mssql://' in your connection string.
adodbapi seems the way to go, but here's a snippet from adodbapi.py that ships with SQL Alchemy under the dialects folder
"""
The adodbapi dialect is not implemented for 0.6 at this time.
"""
SQLAlchemy can not directly run under IronPython, because pyodbc currently is not compatible with IronPython.
However, you can use pypyodbc under IronPython as a dbi-2.0 complaint library, which is similar to pyodbc,and enables running sqlalchemy under Ironpython, this How-to describes the 4 steps to enable it.
Disclaimer: I'm the maintianer of pypyodbc.