Is there a solution in fixing 'MySQL connection not available' for very second request in Flask application? - mysql

Question

I am running a Flask application on local and production server. I have no issues with the local, I am facing 'MySQL connection not available' for every second database request on production server. After reloading or after performing rollback operation, it is getting executed, but the issue repeats. I tried major solutions available on the internet by changing, pool_recycle, 'wait_timeout' and other timeouts to same value of 1600, but didn't work. I finally destroyed the application and reinstalled everything on the server but the issue is still on. Please help in this, thank you.
My production MySQL config:
class ProductionConfig(Config):
SECRET_KEY = 'secret_key'
DEBUG=False
SQLALCHEMY_DATABASE_URI = "mysql+pymysql://{username}:
{password}#{hostname}/{databasename}".format(
username="myusername",
password="password",
hostname="myhostname",
databasename="mydatabase",
)
SQLALCHEMY_POOL_RECYCLE = 299
SQLALCHEMY_TRACK_MODIFICATIONS = False
Error Message:
Exception on /add_questions/1/2 [POST]
Traceback (most recent call last):
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 713, in _write_bytes
self._sock.sendall(data)
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
cursor.execute(statement, parameters)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute
result = self._query(query)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query
conn.query(q)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 516, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 771, in _execute_command
self._write_bytes(packet)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 718, in _write_bytes
"MySQL server has gone away (%r)" % (e,))
pymysql.err.OperationalError: (2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask_login/utils.py", line 270, in decorated_view
elif not current_user.is_authenticated:
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/werkzeug/local.py", line 347, in __getattr__
return getattr(self._get_current_object(), name)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/werkzeug/local.py", line 306, in _get_current_object
return self.__local()
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask_login/utils.py", line 26, in <lambda>
current_user = LocalProxy(lambda: _get_user())
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask_login/utils.py", line 346, in _get_user
current_app.login_manager._load_user()
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/flask_login/login_manager.py", line 318, in _load_user
user = self._user_callback(user_id)
File "/home/ulznrcvr/jaihindpro/application/auth/views.py", line 26, in load_user
return User.query.get(int(user_id))
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 1004, in get
return self._get_impl(ident, loading.load_on_pk_identity)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 1121, in _get_impl
return db_load_fn(self, primary_key_identity)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 287, in load_on_pk_identity
return q.one()
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3360, in one
ret = self.one_or_none()
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3329, in one_or_none
ret = list(self)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3405, in __iter__
return self._execute_and_instances(context)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3430, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 984, in execute
return meth(self, multiparams, params)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
distilled_params,
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
e, statement, parameters, cursor, context
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
cursor.execute(statement, parameters)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute
result = self._query(query)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query
conn.query(q)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 516, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 771, in _execute_command
self._write_bytes(packet)
File "/home/ulznrcvr/virtualenv/jaihindpro/3.6/lib/python3.6/site-packages/pymysql/connections.py", line 718, in _write_bytes
"MySQL server has gone away (%r)" % (e,))
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")
[SQL: SELECT user.id AS user_id, user.student_id AS user_student_id, user.role_id AS user_role_id, user.course_id AS user_course_id, user.sub_course_id AS user_sub_course_id, user.batch_id AS user_batch_id, user.full_name AS user_full_name, user.email AS user_email, user.phone_number AS user_phone_number, user.password AS user_password, user.active AS user_active, user.confirmed AS user_confirmed, user.confirmed_date AS user_confirmed_date, user.subscribed AS user_subscribed, user.start_date AS user_start_date, user.end_date AS user_end_date, user.re_url AS user_re_url, user.online AS user_online
FROM user
WHERE user.id = %(param_1)s]
[parameters: {'param_1': 2}]
(Background on this error at: http://sqlalche.me/e/e3q8)

Answer 1

I figured it out myself, I flagged "pool_pre_ping" attribute to "True" in engine settings.
SQLALCHEMY_ENGINE_OPTIONS = {
"pool_pre_ping": True,
"pool_recycle": 300,
}
Ref: https://medium.com/#heyjcmc/controlling-the-flask-sqlalchemy-engine-a0f8fae15b47