Flask-SQLAlchemy error MySQL server has gone away - mysql

I'm newbie running a Flask app connected to a MySQL remote server with Flask-SQLAlchemy.
The app has very little traffic and it's usual to stay idle for more than 8 hours, I then get disconnected from the MySQL server.
this is my app code:
from flask import Flask, render_template, request, redirect, jsonify, make_response
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import or_
app = Flask(__name__)
app.config['DEBUG'] = True
app.config['SQLALCHEMY_ENGINE_OPTIONS'] = {'pool_pre_ping': True, 'pool_recycle': 300, 'echo':'debug',}
app.config['SQLALCHEMY_DATABASE_URI']='mysql+pymysql://user:pass#myipcode/db'
db = SQLAlchemy(app)
Everything works ok until no querys are performed for 8 hours, then I lose db connection and logs show this error code:
"MySQL server has gone away (%r)" % (e,)) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))
I did some research and was adviced to set 'SQLALCHEMY_ENGINE_OPTIONS' as written in the code sample but it behaves the same with or without such engine options, connection is not recycled every 300 seconds as It should, and pool_pre_ping doesn't seem to make any difference. 'echo':'debug' option works as intended since I get every transaction logged.
What should I do to prevent the connection from being disconnected even after a long period of inactivty?
EDIT:
To add some additional info:
The database is hosted in Cloud SQL from GCP .
I'm lost...
Any help would be greatly appreciated.

Finally I figured it out myself,
It had to do with the fact that my app was running on a mountpoint like http://ServerIP/app instead of http://ServerIP/ because it was initially intended as a staging server.
I was using uWSGI and, in order to make it work in the aforementioned path, I had to specify a mount parameter in the [UWSGI] block from app.ini file.
When the uWSGI server started it showed like it was mounting two apps, one in '' and the other in '/app', and I guess that created a conflict that made the app unable to manage connections on MySQL server.
Mounting the app in http://ServerIP/ worked like a charm.

Related

Flask App loses connection to MySQL database after about a day

I'm using python flask as a backend, running it as a service in the GCP instance. It is working correctly once started the service. But after a day when I try to make an API call it losses its connection with MySQL and shows the following
error mysql.connector.errors.InterfaceError: 2013:
I tried using pymysql and mysql.connector but facing the same error. Is there any way to solve this?

SQLAlchemy AppEngine standard - Lost connection to MySQL server

I'm trying to connect to a Google Cloud SQL second generation in Python from AppEngine standard (Python 2.7).
Until now, I was using MySQLDB driver directly and it was fine.
I've tried to switch to SQLAlchemy, but now I'm always having this error when the code is deployed (it seems to work fine in local) resulting in a error 500 (It's not just some connections which are lost, it constantly fails) :
OperationalError: (_mysql_exceptions.OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38") (Background on this error at: http://sqlalche.me/e/e3q8)
I don't understand because the setup doesn't differ from before, so it must be related to the way I use SQLAlchemy.
I use something like this :
create_engine("mysql+mysqldb://appuser:password#x.x.x.x/db_name?unix_socket=/cloudsql/gcpProject:europe-west1:instanceName")
I've tried different values (with, without the ip, ...). But it is still the same. Is is a version compatibility problem ?
I use
MySQL-python in the app.yaml and SQLAlchemy 1.2.4 :
app.yaml :
- name: MySQLdb
version: "latest"
requirements.txt :
SQLAlchemy==1.2.4
It was a problem in the url. I was adding in a specific part of the code "/dbname" at the end of the connection string, resulting in something like this :
mysql+mysqldb://appuser:password#/db_name?unix_socket=/cloudsql/gcpProject:europe-west1:instanceName/dbname
So in the end, the meaning of this error can also be that the unix socket is wrong.
There are a number of causes for connection loss to Google CloudSQL server but quite rightly, you have to ensure that your setup is appropriate first. I don't think this issue is about version compatibility.
According to the documentation, for your application to be able to connect to your Cloud SQL instance when the app is deployed, you require to add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the app.yaml file(Your displayed app.yaml does not seem to contain these environment variables).
I recommend you review the details in the link for details on how to set up your CloudSQL instance and connecting to the instance.

Web2py scheduler stopping the app from running

I cann't get the scheduler run in my ubuntu or debian linux machines. Originally I had db = DAL('sqlite://storage.sqlite') which causes a weird situation where the website becomes unavailable (the browser cannot establish a connection to the server at 127.0.0.1) but when I run the same app without scheduler (without -K appname), and check the app database, it shows that the scheduler task has been running successfully. What is causing connection to the server to break?
Second, I tried using pymysql or mysqldb instead of sqlite. but, I get this error "Error in URI 'pymysql' or database not supported" even without -K myapp option.

create_engine problems mysql 5.7 and sqlalchemy

I've inherited an application making use of python & sqlalchemy to interact with a mysql database. When I issue:
mysql_engine = sqlalchemy.create_engine('mysql://uname:pwd#192.168.xx.xx:3306/testdb', connect_args={'use_unicode':True,'charset':'utf8', 'init_command':'SET NAMES UTF8'}, poolclass=NullPool)
, at startup, an exception is thrown:
cmd = unicode("USE testdb")
with mysql_engine.begin() as conn:
conn.execute(cmd)
sqlalchemy.exc.OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '192.168.xx.xx' (101)") None None
However, using IDLE I can do:
>>> import MySQLdb
>>> Con = MySQLdb.Connect(host="192.168.xx.xx", port=3306, user="uname", passwd="pwd", db="testdb")
>>> Cursor = Con.cursor()
>>> sql = "USE testdb"
>>> Cursor.execute(sql)
The application at this point defaults to using an onboard sqlite database. After this I can quite happily switch to the MySQL database using the create_engine statement above. However, on reboot the MySQL database connection will fail again, defaulting to the onboard sqlite db, etc, etc.
Has anyone got any suggestions as to how this could be happening?
Just thought I would update this - the problem still occurs exactly as described above. I've updated the app so that the user can manually connect to the MySQL db by selecting a menu option. This calls the identical code which exceptions when the app is starting, but works just fine once the app is up and running.
The MySQL instance is completely separate from the app and running throughout, so it should be available to receive connections at all times.
I guess the fundamental question i'm grappling with is how can the same connect code work when the app is up and running, but throw an exception when it is starting?
Is there any artifact of SQLAlchemy that can cause it to fail to create usable connections that isn't dependant on the connection parameters or the remote database?
Ahhh, it all seems so obvious now...
The reason for the exception on startup was because the network interface hadn't finished configuring when the application would make its first request to the remote database. (Which is why the same thing would be successful when attempted at a later time).
As communication with the remote database is a prerequisite for the application, I now do something like this:
if grep -Fxq "mysql" /path/to/my/db/config.config
then
while ! ip a | grep inet.*wlan0 ; do sleep 1; echo "waiting for network..."; done;
fi
... in the startup script for my application - ensuring that the network interface has finished configuring before the application can run.
Of course, the application will never run if the interface doesn't configure, so it still needs some finessing to allow it to timeout and default to using a local database...

OperationalError Lost connection to MySQL server at 'reading initial communication packet' DigitalOcean

Im having trouble connecting to my sql server on digital ocean through my flask webapp. Im using flask-sqlalchemy to bind the mysql database to Flask.
Im able to access the mysql server through the phpmyadmin interface at myipaddress:5000/phpmyadmin
Since Im using Nginx (I bound it to port 80). Bound Apache to Port 5000. So my phpmyadmin interface is accessible at
myipaddress:5000/phpmyadmin
In my flask app, i specify the
SQLALCHEMY_DATABASE_URI ='mysql://root:password#myipaddress:5000/databasename'
when i try to create the tables on my database using the shell with db.create_all() - it just doesnt respond. The cursor blinks forever and then i get the operational error that i quote on the title afte a few minutes
Im able to get the same setup running on my local dev machine. So i know its not a flask configuration problem but just a mysql access issue. I have my webapp up on digitalocean (Not sure if mysql server is behind a firewall or something like that making it inaccessible
On the
/etc/mysql/my.cnf
for the bind-address under mysql_d section, i tried all possible combinations and restarted the mysql server with no success
i tried localhost, 127.0.0.1, 127.0.0.1:5000, myipaddress for the bind-address (Also tried commenting it out) without any results.
Also i tried to get the user, current_user on the table properties from the mysql command line, it's listed as root#localhost for both
From this post:Lost connection to MySQL server at 'reading initial communication packet', system error: 0; i get the idea that its related to firewall but not sure how to rectify this.
Anyidea how can i connect my flask app to the mysql server ? Any help would be much appreciated
I was specifying the mysql address as
mysql://root:password#myipaddress:5000/databasename
But since my flask app and the mysql server are running on the same server, the flask app was be able to access the mysql server when i replaced the myipaddress:5000 with localhost:5000
mysql://root:password#localhost:5000/databasename
The phpmyadmin is just a web interface for your database, if you're looking at connecting your Flask application to your MySql database, you need to point the SQLALCHEMY_DATABASE_URI to the actual MySQL database, rather then the web interface to that database.
Usually MySql runs on port 3306. I believe that SQLAlchemy is clever enough to know that's the default, so your connection string should just be: SQLALCHEMY_DATABASE_URI ='mysql://root:password#localhost/databasename' or if you are running on a different port/external IP address: SQLALCHEMY_DATABASE_URI ='mysql://root:password#myipaddress:4444/databasename'. Remember, if you're connecting to a MySQL database on an external server (not the same one as your Flask app is running on) you will have to change the configuration to allow that kind of access.