I have mysql server on python anywhere platform. I have a FastAPI app. I would like to access to my database from my FastAPI app. I know that I can reach the database directly from FastAPI app and I saw many tutorials but I am finding the set up a bit complicated and as a beginner I would like to keep it simple.
From pythonanywhere platform support I know I can reach my database from python script with ssh tunneling. I have tested the solution and it worked well.
If I hash my password not as it is in the following script, do you believe this is a suitable solution ?
This is database.py
#database.py
import MySQLdb
import sshtunnel
sshtunnel.SSH_TIMEOUT = 5.0
sshtunnel.TUNNEL_TIMEOUT = 5.0
def get_data():
with sshtunnel.SSHTunnelForwarder(
('ssh.pythonanywhere.com'),
ssh_username='username', ssh_password='hashed',
remote_bind_address=('username.mysql.pythonanywhere-services.com', 3306)
) as tunnel:
connection = MySQLdb.connect(
user='username',
passwd='hashed',
host='127.0.0.1', port=tunnel.local_bind_port,
db='username$dbName',
)
# Do stuff
with connection as con:
with con.cursor() as c:
c.execute("SELECT * FROM table;")
res = c.fetchall()
return res
This is the fastapi app script main.py
#main.py
from fastapi import FastAPI
from database import get_data
app = FastAPI()
#app.get('/mesures')
def get_mesures():
return get_data()
Again, I know this is not best solution but just would like your thoughts.
Ok I close this post as answered based on replies in comment.
Assuming that the FastAPI app is hosted somewhere else, SSH tunneling is the only option to connect to your PA-hosted MySQL db. You need a paid account in order for that to work
Currently FastAPI web apps would not work on PythonAnywhere, but we're working on this -- once it's ready, I guess you will be able to host the web app on PA and connect it with your db in a less "hacky" way. For now tunneling is the only option.
Related
I'm making a Flask application that is using sqlalchemy as a layer between the application and a postgres database. Currently I'm using a 'config.py' file that fetches the sensible connection info from system variables. But my IT admin says it's not sufficiently safe as we will be hosting the server ourselves rather than using PAAS. What would be the most smooth and efficient way to provide the db connetion to sqalchemy without exposing the sensitive connection info to anybody that have access to the server (and thereby being able to read the system variables)?
I'm using VisualStudio as IDE, so dev environment is windows, but would like to be able to deploy on linux if needed.
This is my 'runserver.py' file:
...
from config import DevelopmentConfig, ProductionConfig, TestingConfig
app = create_app(ProductionConfig)
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '6388'))
except ValueError:
PORT = 6388
app.run(HOST, PORT)
And this is my '__init__.py' file:
def create_app(config=DevelopmentConfig):
app = Flask(__name__)
app.config.from_object(config)
db.init_app(app)
...
I'm newbie running a Flask app connected to a MySQL remote server with Flask-SQLAlchemy.
The app has very little traffic and it's usual to stay idle for more than 8 hours, I then get disconnected from the MySQL server.
this is my app code:
from flask import Flask, render_template, request, redirect, jsonify, make_response
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import or_
app = Flask(__name__)
app.config['DEBUG'] = True
app.config['SQLALCHEMY_ENGINE_OPTIONS'] = {'pool_pre_ping': True, 'pool_recycle': 300, 'echo':'debug',}
app.config['SQLALCHEMY_DATABASE_URI']='mysql+pymysql://user:pass#myipcode/db'
db = SQLAlchemy(app)
Everything works ok until no querys are performed for 8 hours, then I lose db connection and logs show this error code:
"MySQL server has gone away (%r)" % (e,)) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))
I did some research and was adviced to set 'SQLALCHEMY_ENGINE_OPTIONS' as written in the code sample but it behaves the same with or without such engine options, connection is not recycled every 300 seconds as It should, and pool_pre_ping doesn't seem to make any difference. 'echo':'debug' option works as intended since I get every transaction logged.
What should I do to prevent the connection from being disconnected even after a long period of inactivty?
EDIT:
To add some additional info:
The database is hosted in Cloud SQL from GCP .
I'm lost...
Any help would be greatly appreciated.
Finally I figured it out myself,
It had to do with the fact that my app was running on a mountpoint like http://ServerIP/app instead of http://ServerIP/ because it was initially intended as a staging server.
I was using uWSGI and, in order to make it work in the aforementioned path, I had to specify a mount parameter in the [UWSGI] block from app.ini file.
When the uWSGI server started it showed like it was mounting two apps, one in '' and the other in '/app', and I guess that created a conflict that made the app unable to manage connections on MySQL server.
Mounting the app in http://ServerIP/ worked like a charm.
I have a DigitalOcean droplet running a MySQL server working perfectly well. I am trying to connect to this server via a IronPython desktop app in Visual Studio.
Here is my code for connecting to the database:
import wpf
import clr
clr.AddReference('System.Data')
from System.Data import DataSet
from System.Data.Odbc import OdbcConnection, OdbcDataAdapter
from System.Windows import Application, Window
class MyWindow(Window):
def __init__(self):
wpf.LoadComponent(self, 'Vulpes.xaml')
connectString = "Server=***.***.***.***;Database=****;Uid=root;Password=****"
query = "SELECT * FROM users"
connection = OdbcConnection(connectString)
adaptor = OdbcDataAdapter(query, connection)
dataSet = DataSet()
connection.Open()
adaptor.Fill(dataSet)
connection.Close()
if __name__ == '__main__':
Application().Run(MyWindow())
The issue here is with the connection.Open() and connection.Close(functions), if I comment out those parts, everything works great. The username, server address, password, and DB name are all correct and the query is a valid query.
I'm thinking my method is just outdated, and while I'd love to do something more modern, IronPython only supports a very limited amount of Python modules, and so I'm having trouble finding another way to do it this way. There's not even a list of all the supported modules in their documentation, so you just have to look through MySQL modules and do trial-and-error.
If anyone has a better way of connecting to a MySQL DB via IronPython, I would love to hear it.
Thanks for your time!
I have SQLAlchemy CORE 1.0.9 with Pyramid framework 1.7. And I am using the following configuration for connecting to a postgres 9.4 database:
# file __ini__.py
from .factories import root_factory
from pyramid.config import Configurator
from sqlalchemy import engine_from_config
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application."""
config = Configurator(settings=settings, root_factory=root_factory)
engine = engine_from_config(settings, prefix='sqlalchemy.')
# Retrieves database connection
def get_db(request):
connection = engine.connect()
def disconnect(request):
connection.close()
request.add_finished_callback(disconnect)
return connection
config.add_request_method(get_db, 'db', reify=True)
config.scan()
return config.make_wsgi_app()
After a few hours using the app I start getting the following error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: remaining connection slots are reserved for non-replication superuser connections
Apparently I have reached the maximum number of connections. It seems like connections.close() doesn't really close the connection, just returns the connection to the pool. I know I could use the NullPool to disable pooling but probably that will have a huge impact in the performance.
Does somebody know the right way to configure SQLAlchemy Core to get a good performance and close the connections properly?
Please abstain from sending links to pyramid tutorials. I am NOT interested in SQLAlchemy ORM setups. Only SQLAlchemy Core please.
Actually everything was fine in the previous setup. The problem was caused by Celery workers not closing connections.
I've inherited an application making use of python & sqlalchemy to interact with a mysql database. When I issue:
mysql_engine = sqlalchemy.create_engine('mysql://uname:pwd#192.168.xx.xx:3306/testdb', connect_args={'use_unicode':True,'charset':'utf8', 'init_command':'SET NAMES UTF8'}, poolclass=NullPool)
, at startup, an exception is thrown:
cmd = unicode("USE testdb")
with mysql_engine.begin() as conn:
conn.execute(cmd)
sqlalchemy.exc.OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '192.168.xx.xx' (101)") None None
However, using IDLE I can do:
>>> import MySQLdb
>>> Con = MySQLdb.Connect(host="192.168.xx.xx", port=3306, user="uname", passwd="pwd", db="testdb")
>>> Cursor = Con.cursor()
>>> sql = "USE testdb"
>>> Cursor.execute(sql)
The application at this point defaults to using an onboard sqlite database. After this I can quite happily switch to the MySQL database using the create_engine statement above. However, on reboot the MySQL database connection will fail again, defaulting to the onboard sqlite db, etc, etc.
Has anyone got any suggestions as to how this could be happening?
Just thought I would update this - the problem still occurs exactly as described above. I've updated the app so that the user can manually connect to the MySQL db by selecting a menu option. This calls the identical code which exceptions when the app is starting, but works just fine once the app is up and running.
The MySQL instance is completely separate from the app and running throughout, so it should be available to receive connections at all times.
I guess the fundamental question i'm grappling with is how can the same connect code work when the app is up and running, but throw an exception when it is starting?
Is there any artifact of SQLAlchemy that can cause it to fail to create usable connections that isn't dependant on the connection parameters or the remote database?
Ahhh, it all seems so obvious now...
The reason for the exception on startup was because the network interface hadn't finished configuring when the application would make its first request to the remote database. (Which is why the same thing would be successful when attempted at a later time).
As communication with the remote database is a prerequisite for the application, I now do something like this:
if grep -Fxq "mysql" /path/to/my/db/config.config
then
while ! ip a | grep inet.*wlan0 ; do sleep 1; echo "waiting for network..."; done;
fi
... in the startup script for my application - ensuring that the network interface has finished configuring before the application can run.
Of course, the application will never run if the interface doesn't configure, so it still needs some finessing to allow it to timeout and default to using a local database...