I'm making a Flask application that is using sqlalchemy as a layer between the application and a postgres database. Currently I'm using a 'config.py' file that fetches the sensible connection info from system variables. But my IT admin says it's not sufficiently safe as we will be hosting the server ourselves rather than using PAAS. What would be the most smooth and efficient way to provide the db connetion to sqalchemy without exposing the sensitive connection info to anybody that have access to the server (and thereby being able to read the system variables)?
I'm using VisualStudio as IDE, so dev environment is windows, but would like to be able to deploy on linux if needed.
This is my 'runserver.py' file:
...
from config import DevelopmentConfig, ProductionConfig, TestingConfig
app = create_app(ProductionConfig)
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '6388'))
except ValueError:
PORT = 6388
app.run(HOST, PORT)
And this is my '__init__.py' file:
def create_app(config=DevelopmentConfig):
app = Flask(__name__)
app.config.from_object(config)
db.init_app(app)
...
Related
I'm currently a little stuck as to why I am unable to connect to a DB from a Kubernetes cluster hosting my Fast API app.
I've gone through the steps of ensuring that my DB has the Kubernetes cluster and pool whitelisted for incoming connections.
I have also ensured I have the correct environment variables when attempting to connect, so the following both exist and are correct within my environment (redacted for security):
POSTGRES_USER: "***************"
POSTGRES_PASSWORD: "***************"
POSTGRES_SERVER: "***************"
POSTGRES_PORT: "25060"
POSTGRES_DB: "***************"
I am also constructing the Database URL within my Fast API's app as follows:
from typing import Any, Dict, List, Optional, Union
from pydantic import AnyHttpUrl, BaseSettings, HttpUrl, PostgresDsn, validator
class Settings(BaseSettings):
...
POSTGRES_SERVER: str
POSTGRES_USER: str
POSTGRES_PASSWORD: str
POSTGRES_DB: str
POSTGRES_PORT: str
SQLALCHEMY_DATABASE_URI: Optional[PostgresDsn] = None
#validator("SQLALCHEMY_DATABASE_URI", pre=True)
def assemble_db_connection(cls, v: Optional[str], values: Dict[str, Any]) -> Any:
if isinstance(v, str):
return v
return PostgresDsn.build(
scheme="postgresql",
user=values.get("POSTGRES_USER"),
password=values.get("POSTGRES_PASSWORD"),
host=values.get("POSTGRES_SERVER"),
port=values.get("POSTGRES_PORT"),
path=f"/{values.get('POSTGRES_DB') or ''}",
)
I'm seeing everything running correctly from what I can see, so I am able to connect to the DB with those credentials outside of SQLAlchemy and the Fast API app.
However, when I run alembic upgrade head to run migrations within the container on the cluster (or pod), I am seeing the following error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection timed out
Is the server running on host "*******" () and accepting
TCP/IP connections on port 5432?
I totally understand that 5432 is not accepting incoming connections, as I have specified a different port value ...
Is there something I have done wrong ... or some extra steps I need to take, or is there something more subtle going wrong with SQLAlchemy?
Can anyone advise any steps to take to try and understand why it is attempting to make a connection on 5432 when I specified a port value in the env?
I have mysql server on python anywhere platform. I have a FastAPI app. I would like to access to my database from my FastAPI app. I know that I can reach the database directly from FastAPI app and I saw many tutorials but I am finding the set up a bit complicated and as a beginner I would like to keep it simple.
From pythonanywhere platform support I know I can reach my database from python script with ssh tunneling. I have tested the solution and it worked well.
If I hash my password not as it is in the following script, do you believe this is a suitable solution ?
This is database.py
#database.py
import MySQLdb
import sshtunnel
sshtunnel.SSH_TIMEOUT = 5.0
sshtunnel.TUNNEL_TIMEOUT = 5.0
def get_data():
with sshtunnel.SSHTunnelForwarder(
('ssh.pythonanywhere.com'),
ssh_username='username', ssh_password='hashed',
remote_bind_address=('username.mysql.pythonanywhere-services.com', 3306)
) as tunnel:
connection = MySQLdb.connect(
user='username',
passwd='hashed',
host='127.0.0.1', port=tunnel.local_bind_port,
db='username$dbName',
)
# Do stuff
with connection as con:
with con.cursor() as c:
c.execute("SELECT * FROM table;")
res = c.fetchall()
return res
This is the fastapi app script main.py
#main.py
from fastapi import FastAPI
from database import get_data
app = FastAPI()
#app.get('/mesures')
def get_mesures():
return get_data()
Again, I know this is not best solution but just would like your thoughts.
Ok I close this post as answered based on replies in comment.
Assuming that the FastAPI app is hosted somewhere else, SSH tunneling is the only option to connect to your PA-hosted MySQL db. You need a paid account in order for that to work
Currently FastAPI web apps would not work on PythonAnywhere, but we're working on this -- once it's ready, I guess you will be able to host the web app on PA and connect it with your db in a less "hacky" way. For now tunneling is the only option.
I am currently using Flask-uWSGI-Websockets to provide websocket functionality for my application. I use Flask-SQLAlchemy to connect to my MySQL database.
Flask-uWSGI-Websockets uses gevent to manage websocket connections.
The problem I am currently having is that when a websocket connection is ended, the database connection set up by Flask-SQLAlchemy will keep on living.
I have tried calling db.session.close() and db.engine.dispose() after every websocket connection, but this had no effect.
Calling gevent.monkey.patch_all() at the beginning of my app does not make a difference.
A simple representation of what I am doing is this:
from gevent.monkey import patch_all
patch_all()
from flask import Flask
from flask_uwsgi_websocket import GeventWebSocket
from flask_sqlalchemy import SQLAlchemy
app = Flask()
ws = GeventWebSocket()
db = SQLAlchemy()
db.init_app(app)
ws.init_app(app)
#ws.route('/ws')
def websocket(client):
""" handle messages """
while client.connected is True:
msg = client.recv()
# do some db stuff with the message
# The following part is executed when the connection is broken,
# i tried this for removing the connection, but the actual
# connection will stay open (i can see this on the mysql server).
db.session.close()
db.engine.dispose()
I have same situation. and solution for me located in mysql configuration file my.cnf:
[mysqld]
interactive_timeout=180
wait_timeout=180
you must restart mysql service after save my.cnf.
if you don't want to restart mysql service you can use sql queries:
SET GLOBAL interactive_timeout = 180;
SET GLOBAL wait_timeout = 180;
See also wait_timeout and interactive_timeout on mysql.com
I'm trying to connect using my VB.NET app to a WAMP server that is located on another PC (on local network), but I'm getting an error:
Unable to connect to any of the specified MySQL hosts.
That happens just if I'm trying to connect via my app (if I type on local PC in browser xxx.xxx.xxx.xxx/phpmyadmin it works, and on server machine it works via browser and via app).
Dim build As New MySqlConnectionStringBuilder
build.Server = "192.168.1.4"
build.Database = "gydytojo asistentas_new"
build.UserID = "root"
build.Password = ""
build.Port = "3306"
build.CharacterSet = "utf8"
build.MaximumPoolSize = "1000"
Return build.ConnectionString`
Are you sure this is no network issue? Did you try disabling firewall (on your pc and/or on remote). In most cases database servers are not accesible over the net - so this would make your server vulnerable - but maybe the solution for you.
You may have a look at this question on security.stackExchange.com discussing this topic.
What is the best practice for placing DataBase servers in secure network topologies
I'm trying to deploy a Django app on Heroku with an RDS instance as the database backend. Everything is working until I try to encrypt the connection, then I get this error:
OperationalError at /path/
(2026, 'SSL connection error')
Here's the setup:
Standard Django application
MySQL RDS instance with security group allowing connections from all IP addresses
MySQL user is setup to allow connections from any host
Amazon's pem has been downloaded and is specified in Django settings
On Heroku:
DATABASE_URL: mysql2://username:password#instance.us-east-1.rds.amazonaws.com:3306/name_staging?sslca=path/to/mysql-ssl-ca-cert.pem
In Django settings:
DATABASES = {
'default': dj_database_url.config()
}
DATABASES['default']['OPTIONS'] = {'ssl': {'ca': 'mysql-ssl-ca-cert.pem'}}`
I've tried searching and have read a lot about setting this type of environment up in Rails, but the documentation about doing this with Django is light to non-existent.
Has anyone out there successfully deployed a similar setup or does anyone have thoughts on how to solve this error?
Update:
Connecting via cli works as well as connecting directly using MySQLdb in the python interpreter.
Solved:
The path to the pem file has to be absolute and you can't use python to attempt to build the absolute path.
DATABASES = {
'default': dj_database_url.config()
}
DATABASES['default']['OPTIONS'] = {
'ssl': {'ca': '/app/project_name/rds/mysql-ssl-ca-cert.pem'}
}
Again, detecting the path like this does not work, the path must be hard coded:
DATABASES['default']['OPTIONS'] = {
'ssl': {'ca': os.path.join(os.path.dirname(__file__), 'rds', 'mysql-ssl-ca-cert.pem')}
}