I'm trying to use Flask-MySQLdb. Every time I go to a page I get the error _mysql_exceptions.ProgrammingError: closing a closed connection. Why isn't this working?
from flask import Flask, render_template, request, json
from flask_mysqldb import MySQL
application = Flask(__name__)
mysql = MySQL(application)
application.config['MYSQL_USER'] = 'root'
application.config['MYSQL_PASSWORD'] = 'password'
application.config['MYSQL_DB'] = 'db name'
application.config['MYSQL_HOST'] = 'localhost'
mysql.init_app(application)
#application.route('/')
def index():
cur = mysql.connection.cursor()
cur.execute("SHOW TABLES")
rv= cur.fetchall()
return str(rv)
if __name__ == "__main__":
application.run()
You've initialize the extension on the same application twice. Each time it's initialized, it registers a function that closes the connection. So the second function closes a connection that was already closed by the first registered function. Either remove application from MySQL (preferable) or remove the explicit call to mysql.init_app.
mysql = MySQL()
mysql.init_app(application)
Related
I want to test my DBConnect class, which manages and returns connections made with sqlalchemy.
I want to specifically test that the constructor of DBConnect calls sqlalchemy.create_engine and stores the engine in the connection.
My test function is this:
from unittest import mock
from catcom.db_connect import DBConnect
import os
class MockEngine:
def connect(self):
return "test_connection"
#mock.patch("sqlalchemy.create_engine", return_value=MockEngine())
def test_from_key_uses_correct_key(mock):
print(mock)
db = DBConnect("user", "pw", "127.0.0.1", "1234", "db")
assert db.connection == "test_connection"
However, I see a response of:
E sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "127.0.0.1", port 1234 failed: Connection refused
E Is the server running on that host and accepting TCP/IP connections?
How can I mock the create_engine method without it being actually called?
UPDATE
Adding the DB Connect class and relevant methods.
from sqlalchemy import create_engine
import os
class DBConnect:
"""Manages database connection using SQLalchemy"""
def __init__(self, user: str, password: str, host: str, port: str, database: str):
self.user = user
self.password = password
self.host = host
self.port = port
self.database = database
self.connection = self.connect()
def connect(self):
"""creates a connection object that is used to query database using SQLalchemy core
automatically created in __init__"""
engine = create_engine(
f"postgresql+psycopg2://{self.user}:{self.password}#{self.host}:{self.port}/{self.database}"
)
return engine.connect()
You are importing create_engine out of the sqlalchemy namespace with from sqlalchemy import create_engine, which puts it in the module's namespace.
That's what you need to patch, see where to patch.
#mock.patch('catcom.db_connect.create_engine', return_value='test_success')
Although, patching catcom.db_connect.create_engine to return 'test_success' will cause a problem when the DBConnect.connect method tries to call the connect method of the engine (which will be a string).
I already have a mysql connection from Flask like this:
app.config['MYSQL_HOST'] = 'one.hostname.net'
app.config['MYSQL_USER'] = 'my_username'
app.config['MYSQL_PASSWORD'] = 'my_password'
app.config['MYSQL_DB'] = 'user_mydb'
mysql = MySQL(app)
with this setup I am able to use mysql database connection in flask. but when it comes to celery task, which is inside the same python file as flask.
#mycelery.task(bind=True, name='mytask')
def mytask(self, userid, port):
cursor = mysql.connection.cursor(MySQLdb.cursors.DictCursor)
cursor.execute('SELECT * FROM mytable WHERE id = %s', (userid,))
It throws me an error saying
cursor = mysql.connection.cursor(MySQLdb.cursors.DictCursor)
AttributeError: 'NoneType' object has no attribute 'cursor'
I understand it is because the celery has no MySQL connection made. But How can I establish the connection? so that I don't have to connect with the MySQL server whenever I create a task just like how it is don't for flask where we already established the connection by MySQL = MySQL(app)??
Here is my celery setup, if it is helpful to add something to this code
mycelery = Celery(app.name)
mycelery.conf.update({
'broker_url': 'filesystem://',
'broker_transport_options': {
'data_folder_in': 'app/broker/out',
'data_folder_out': 'app/broker/out',
'data_folder_processed': 'app/broker/processed'
},
'result_persistent': False,
'task_serializer': 'json',
'result_serializer': 'json',
'accept_content': ['json']})
Firstly if you want to do dabatabase operation you can do in celery task, but you dont have to connect db with celery.
You can connect flask with db, and install celery in your project and make db operation in your celery task.
Sample:
app.py
from flask import Flask
from flaskext.mysql import MySQL
app = Flask(__name__)
mysql = MySQL()
app.config['MYSQL_DATABASE_USER'] = ''
app.config['MYSQL_DATABASE_PASSWORD'] = ''
app.config['MYSQL_DATABASE_DB'] = ''
app.config['MYSQL_DATABASE_HOST'] = ''
mysql.init_app(app)
tasks.py
#celery.task
def db_connect_things():
conn = mysql.connect()
cursor =conn.cursor()
sql_query = """select from where """
cursor.execute(sql_query)
...
celery_config.py
from celery import Celery
celery = Celery(__name__)
celery = Celery('tasks', broker=) # rabbit,redis, ..
celery.conf.update({'CELERY_ACCEPT_CONTENT': ['pickle', 'json', 'msgpack', 'yaml']})
celery.conf.add_defaults(...)
celery.conf.update(CELERYBEAT_SCHEDULE={
'db_connect_things': {
'task': 'application.lib.tasks.db_connect_things',
'schedule': crontab(minute=0, hour='*/12'),
}})
class ContextTask(celery.Task):
...
So I'm trying to deploy my Django project using lambda, with zappa. I'm using MySQL for DB engine. Now after doing some research, I realized that I needed to create a custom Django command to create DB, since I'm using MySQL. So I created crate_db command, zappa updated, then ran zappa manage dev create_db. Then I got this error: 2004 (HY000): Can't create TCP/IP socket (97)
below is my create_db.py file, for your information.
import sys
import logging
import mysql.connector
import os
from django.core.management.base import BaseCommand, CommandError
from django.conf import settings
rds_host = os.environ.get("MY HOST")
db_name = os.environ.get("")
user_name = os.environ.get("MY USERNAME")
password = os.environ.get("MY PASSWORD")
port = os.environ.get("3306")
logger = logging.getLogger()
logger.setLevel(logging.INFO)
class Command(BaseCommand):
help = 'Creates the initial database'
def handle(self, *args, **options):
print('Starting db creation')
try:
db = mysql.connector.connect(host=rds_host, user=user_name,
password=password, db="mysql", connect_timeout=10)
c = db.cursor()
print("connected to db server")
c.execute("""CREATE DATABASE bookcake_db;""")
c.execute("""GRANT ALL ON bookcake_db.* TO 'ryan'#'%'""")
c.close()
print("closed db connection")
except mysql.connector.Error as err:
logger.error("Something went wrong: {}".format(err))
sys.exit()
Any ideas? Thanks.
I have a Python Flask Server setup in an Ubuntu Machine and a MySQL from XAMPP as backend for the same.
How ever when I try to access the database tables from my python program it shows as
pymysql.err.InternalError: (1109, "Unknown table 'ALL_PLUGINS' in information_schema")
but i can access the database directly in MySQL admin page
the sample program I used to access the data.
from flaskext.mysql import MySQL
from flask import (Flask, request, session, g, redirect, url_for, abort, render_template, flash, Response)
import os
from werkzeug.utils import secure_filename
mysql = MySQL()
app = Flask(__name__)
app.config['MYSQL_DATABASE_USER'] = 'root'
app.config['MYSQL_DATABASE_PASSWORD'] = 'root'
app.config['MYSQL_DATABASE_DB'] = 'information_schema'
app.config['MYSQL_DATABASE_HOST'] = 'localhost'
mysql.init_app(app)
#app.route('/')
def insert_student():
qry = "SELECT * FROM ALL_PLUGINS "
conn = mysql.connect()
cursor = conn.cursor()
cursor.execute(qry)
data = cursor.fetchall()
print(data)
conn.commit()
return "Sucess"
if __name__ == '__main__':
app.secret_key = 'super secret key'
app.debug = True
app.run()
instead of normal running I ge the following
the screen shot
Mysql does not have an all_plugins table in information schema. The plugins table (well, view) is called plugins.
So, your query should be:
SELECT * FROM PLUGINS
Based on the comment from #snakecharmerb:
Mariadb, on the other hand, does have all_plugins table, which presumably is the cause of the confusion.
Im having a little problem when using mysql in python with Flask framework. I have set up a function(get_db()) to connect to the database once a request and serve the same connection to functions requesting it later on(within the same request).
import mysql.connector #mysql-connector-python==8.0.12
def get_db():
if 'db' not in g:
try:
click.echo("Trying to connect")
cnx = mysql.connector.connect(user='stack', password='overflow',
host='127.0.0.1',
database=DB_NAME)
g.db = cnx
return g.db
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
click.echo("Connection failed! Check username and password")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
click.echo("Database does not exist")
else:
click.echo("Unknown error: {} ".format(err))
else:
cnx.close()
# Connection to db cannot
# be established.
# click.echo("Will exit from database")
exit(1)
click.echo("Using existing connection")
return g.db
In other functions I use it like this:
(...)
cnx = get_db()
cursor = cnx.cursor()
(...)
The first function that uses the database works fine. Then when another one tries to connect the cursor failes because cnx doesn't have a connection:
raise errors.OperationalError("MySQL Connection not available.")
Does anyone have a way to handle this?
One solution is to create the connection again for each function but for sake of performance I rather reuse a connection if possible.
The solution I have currently ended up with involves a reconnect at each time a method needs the db connection.
I don't know if this will get the overhead a connect() get's but it is suitable for the current use case.
The code will look like this:
def get_db():
if 'db' not in g:
try:
cnx = mysql.connector.connect(user='tom', password='jerry',
host='127.0.0.1',
database=DB_NAME)
g.db = cnx
click.echo("Returns new connection")
return g.db
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
click.echo("Connection failed! Check username and password")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
click.echo("Database does not exist")
else:
click.echo("Unknown error: {} ".format(err))
cnx = g.db
cnx.reconnect() <--- Doing a reconnect each time
return cnx
you can use pooling : https://dev.mysql.com/doc/connector-python/en/connector-python-connection-pooling.html
import mysql.connector
conf = {"user":"username", "password":"*****", host="hostname"}
pool_cnc = mysql.connector.connect(pool_name="my_pool", **conf)
# Take one connection using the name of the pool:
cnc1 = mysql.connector.connect(pool_name = pool_cnc.pool_name)
cnc1.is_connected()
# Should output True
At the given link there is also an example about the explicitly pooling connection