cant connect to MySQL on aws RDS with flask-sqlalchemy - mysql

I am trying to connect to an RDS instance with flask-sqlalchemy. But i cant get it to work. I am getting the error
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1049, "Unknown database 'dbname'")ยจ
My database URI looks like this
SQLALCHEMY_DATABASE_URI = 'mysql://username:password#dbname.xxxxxxxxxxxx.eu-central-1.rds.amazonaws.com:3306/dbname'
I have tried using the pymysql plugin, but it didnt make any difference.
I think my security groups are setup correctly.
On the RDS instance it is this
(I dont know if security groups contain any personnel info)

Try creating the database first:
import pandas as pd
import sqlalchemy
mysql_engine = sqlalchemy.create_engine("mysql://username:password#dbname.xxxxxxxxxxxx.eu-central-1.rds.amazonaws.com:3306")
pd.read_sql_query("create database dbname", con=mysql_engine)

Related

Importing big sql file ends up with [ERROR_RDBMS] exit status 1

I'm trying to import a big sql file (around 10GB) in Google SQL. I have create my instance and had tried so far with MySQL 5.6 and MySQL 5.7 both getting the same issue when trying to import the database: [ERROR_RDBMS] exit status 1. I have tried the user interface method for import as well as the console one (gcloud sql import sql mydb gs://my-path/mydb.sql --database=mydb). Also tried to find more details in Log explorer, but there was no error or anything else be notices for the mysql.err log.
In the same time I manage to connect my MySQL Workbench to the instance and I could see that there where few database tables that were imported. The only import seems that crushed on importing one of the biggest tables which is around 5GB.
Can anyone suggest what might cause this error and how I can find more info about it. Thank you.
I didn't found how to see the error on google console but figure it our that I can import the database through the MySQL Workbench and there I got the full error. If any one know a way to do this on the google cloud I'll accept her/his answer.

How to set TLS for a RDS proxy with SQLAlchemy

I have an AWS Lambda function (Python3.8) in which I am trying to connect to a RDS proxy using SQLAlchemy. I've confirmed that the function configuration will allow for this by also connecting to the proxy directly using PyMySQL. When I run the function, I get an error message of "(pymysql.err.InternalError) (3159, 'This RDS Proxy requires TLS connections')\n(Background on this error at: http://sqlalche.me/e/2j85)". The "background" for that error says nothing about TLS. I understand what I need to do (tell SQLAlchemy to connect using SSL/TLS), but I cannot figure out the syntax to do so. Below is my current code.
import pymysql
import sqlalchemy
from database_info import make_connection_str
print('connecting to database')
CONN_STR = make_connection_str()
ENGINE = sqlalchemy.create_engine(CONN_STR)
METADATA = sqlalchemy.MetaData(ENGINE)
TABLE = sqlalchemy.Table('active_prospect', METADATA, autoload=True) #error comes with this line
Things I've tried have been related to the following
According to https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy.html ssl-mode=REQUIRED is needed.
pymysql.connect(... ssl={"true": True}) works.
Based on those two things, I've tried every combination of (ssl-mode, ssl) = ('true', 'required') in two different ways.
CONN_STR += '?ssl-mode=REQUIRED'
create_engine(CONN_STR, connect_args={'ssl': 'True'})
Results have varied. In some cases the error above has been replaced by AttributeError: 'str' object has no attribute 'get', still being raised from the same line.
I faced the same issue when i was trying to connect to RDS using RDS proxy from lambda. Got fixed by
In lambda configuration, you will see the section 'Database proxies'. Go to 'Add database proxy' and add the proxy here. Basically, it will modify the IAM Role of the lambda to connect to proxy.
In lambda code, I just replaced the host to the proxy endpoint (it worked without SSL argument)

Connect to AWS RDS MySql database with Python

I am currently trying to connect to my MySql database created on AWS with a python program using the library PyMySQL.
import pymysql.cursors
connection = pymysql.connect(host='.blablablablabla.us-east.rds.amazonaws.com',
user='Username',
password='Password',
db='nameofmydb',
)
When I run the above code I get the following error:
pymysql.err.InternalError: (1049, "Unknown database 'nameofmydb'")
What am I doing wrong? The arguments I gave to the function are correct. Could it be a problem with the MySql driver?
Thank you in advance
The message is the standard one, returned by mysql server. It means what it says, it can not find the database. Doesn't look like a connectivity problem or a driver issue: it reaches the server and the server returns a message.
Just make sure you have created the database using the CREATE DATABASE command. Use the mysql CLI client to connect to the database and type SHOW DATABASES to see all the available databases.

Amazon RDS: Automatically Changing Replication Role During Import

I created an RDS instance on AWS (Aurora MySQL with replication). When I start importing an SQL file into the database I get an error message ERROR 2013 (HY000) at line 2805: Lost connection to MySQL server during query and AWS changes the replication role from writer to reader.
Is there a way to disable changing of the replication role during import or some other way to fix it?

PythonAnywhere error trying to access MySQL database (Access denied)

I have setup an environment in PythonAnywhere with a simple database (new_db), following all the guides and tutorials.
I have a simple python Flask app which is trying to connect to the database using the following code:
from flask import Flask, render_template, request, redirect, url_for
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config["DEBUG"] = True
SQLALCHEMY_DATABASE_URI = "mysql+mysqlconnector://{username}:{password}#{hostname}/{databasename}".format(
username="WazzalJohn",
password="my_password_here",
hostname="WazzalJohn.mysql.pythonanywhere-services.com",
databasename="WazzalJohn$new_db>",
)
app.config["SQLALCHEMY_DATABASE_URI"] = SQLALCHEMY_DATABASE_URI
app.config["SQLALCHEMY_POOL_RECYCLE"] = 299
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = True
db = SQLAlchemy(app)
But i keep getting the error:
ProgrammingError: (mysql.connector.errors.ProgrammingError) 1044 (42000): Access denied for user 'WazzalJohn'#'%' to database 'WazzalJohn$new_db>'
I have tried creating a new database but get the same problem. I thought PythonAnywhere handled all these types of issues so don't know what to do... has anyone had the same problem specifically on PythonAnywhere?
I have now tried to log in through a BASH console and got similar message - screen shot is here
In your original code, you have an error in the database name:
databasename="WazzalJohn$new_db>",
...should be:
databasename="WazzalJohn$new_db",
...without the extra ">".
The problem in the screenshot is that you're not specifying a database on the command line; you need to do something like:
mysql -u TWarren -h TWarren.mysql.pythonanywhere-services.com -p 'TWarren$default'
...where 'TWarren$default' should be replaced with the actual database name. The single quotes around the database name are important, BTW -- if you don't use them, Bash will interpret "$default" as an environment variable and mysql will try to connect to the wrong database.