How to connect MySQLdb code to google cloud SQL? - mysql

I have a flask server which I want to deploy on Google Cloud Platform. The code uses MySQLdb library to connect with local MySQL instance in the following manner:
#app.route('/show_table', methods=['POST'])
def login():
db = MySQLdb.connect("localhost", "root", "", "db_name")
cursor = db.cursor()
query = "select * from table_name;"
cursor.execute(query)
res = cursor.fetchall()
return res, 200
But instead of local MySQL instance, I want to connect this code to Cloud SQL so that it reads data from the cloud. What changes should I make to this code? I have currently created a project in Google Cloud Platform and a Cloud SQL instance inside this project. I have also created the required tables inside this instance by following
this tutorial.

You shouldn't have to change your code too much, it just depends on how you're going to connect to the database. The Google documentation has step by step information on how to connect to Cloud SQL from an external application.
Since you're not using Java or GO, there's two options:
Use the Cloud SQL proxy
Whitelist the public IP address of your server on the Cloud SQL instance page
All the steps are in the documentation, but it basically says that if you use the proxy, you'll need to enable the Cloud SQL Admin API, install the proxy client on your local machine and authenticate it. There's a few authentication options but the recommended way is creating a credentials file from a service account using the console and passing the file as a parameter when you first start the proxy. Once you've got the proxy running the documentation has examples on how to connect using either TCP or UNIX sockets. With TCP you'll be using the proxy as localhost so you won't have to change your code. Using UNIX sockets you'll use the instance connection name which you'll find in your instance details on the GCP console. MySQLdb supports both.
With the second option you need to allow access to your Cloud SQL instance from a specific IP address range. Go to the connections tab in your Cloud SQL instance details page and add the IP address (using CIDR notation) you want to use to connect to your database. Once it's whitelisted then you can use the Public IP of your Cloud SQL instance, which you'll find in instance details, in place of localhost to connect to your database.

Related

Azure web app with MySQL cannot connect to database through Virtual Network Service Endpoints

I have a PHP web app (Snipe-IT) that was previously working.
We went through a security review, and the recommendation was made to enable service endpoints (Microsoft.Sql) on the application subnet so that web apps connect to databases over the Azure backbone. Everything is working fine for the other MSSQL apps, except for our Snipe-IT application, which connects to a GP_Gen5_2 MySQL instance.
The error message says:
D'oh! Looks like we can't connect to your database. Please update your database settings in your .env file. Your database says:
SQLSTATE[HY000] [9009] Client connections to Basic tier servers through Virtual Network Service Endpoints are not supported. Virtual Network Service Endpoints are supported for General Purpose and Memory Optimized severs. (SQL: select 2 + 2)
This error message doesn't match the environment configuration because the database in question is a General Purpose, 2 vCore(s), 50 GB database.
There is no networking tab on the MySQL server, but in the Connection security tab we have tried enabling access to Azure resources, allowing the full public internet (0.0.0.0 - 255.255.255.255), enabling and disabling forced SSL, but nothing changes the result.
Azure won't let us create a Private Endpoint on the MySQL database because 'the subnet contains external resources'
I am hoping there is a way to resolve this without having to rebuild both the app and database in a different App Service Environment and subnet.
Thanks

Connecting from google cloud run to google cloud (mysql) using .net core

I have a .net core app installed as a docker on google cloud run, this app that needs to be connected to cloud sql (mysql).
When using the private ip address it, it's not working.
When using public IP, it's working, but It's not a good solution for production.
this is my connection string:
"ConnectionString": "server=10.4.16.6;database=mydb;user=root;pwd=mypwd"
When I create the app, Im able to select the database i need to connect to:
But this is not helping to connect.
The relevant docs are explaining how to do it for python and java explictly.
If you do not want to use public IP then you would need to rely on service account to connect to Cloud SQL. However, .net MySQL driver has no understanding of GCP IAM and Service accounts. So you will need to use a proxy called Cloud SQL Proxy. Cloud SQL Proxy understands IAM and Service accounts.
The flow will basically look like this:
Your app -> Regular MySQL Port -> Cloud SQL Proxy(Installed in the
app's network or locally) -> CloudSQL
You will need to do the following:
Create a service account
Assign the role of Cloud SQL Client to the created service account
Download the service account key in the json format
Set env variable GOOGLE_APPLICATION_CREDENTIALS=C:\Downloaded.json
Download Cloud SQL Proxy
Run it `cloud_sql_proxy -instances=projectname:regioname:instanceid=tcp:3306
At this point you MySQL proxy ready to accept connections at 3306, modify the connection string to take localhost or wherever you installed the Cloud SQL Proxy.
Learn more at About the Cloud SQL Proxy
You can create the Cloud Run app from the console (and select the Cloud SQL Connection) or from the gcloud command line and specify
--add-cloudsql-instances <INSTANCE-NAME> --set-env-vars INSTANCE-CONNECTION-NAME="INSTANCE_CONNECTION_NAME"
These settings automatically enable and configures the Cloud SQL proxy. You can connect to the proxy, from your asp.net Core app, using the unix domain socket using the format: /cloudsql/INSTANCE_CONNECTION_NAME.
I used the following connection string in my appsettings.json and it worked for me:
"Server=/cloudsql/INSTANCE_CONNECTION_NAME;Database=DB_NAME;Uid=USER_NAME;Pwd=PASSWORD;Protocol=unix"
NB. Make sure you have given the service account that your Cloud Run app is running under Cloud SQL Client role in IAM

not able to login in google cloud storage sql instance

I made instance in Google Cloud > Storage > SQL > MYSQL. Now I have to import database in it so I am trying to connect to MYSQL
using adminer, but it gives following error
SQLSTATE[HY000] [2002] Operation timed out
I also tried via HeidiSQL > MariaDB or MYSQL (SSL Tunnel) but here also it does not connect and gives error
https://prnt.sc/o58in0
I am mentioning
Host: Public Ip of SQL instance
Username: root
Any suggestions, what could be the fix?
Your error typically means that a firewall blocked your connection attempt.
For Cloud SQL, this typically means that you have not enabled instance-level access.
Since your error is 10060, which is a Windows WinSock error message, I am assuming that you are running on Windows. The steps are similar for Linux.
There are two methods to enable access to Cloud SQL:
Cloud SQL Proxy
This is the recommended method and is secure. Download and install the Cloud SQL Proxy to your the computer that you want to connect from. If you have set up the Cloud SDK (gcloud), Cloud SQL Proxy will use your Cloud SDK credentials. If not, then you will need to create and download a service account with (usually) the Cloud SQL Client role.
You will need the "instance connection name" from Cloud SQL Instance details page.
Download Cloud SQL Proxy (link) to your computer. Remember the full path to the program. Save the program as "cloud_sql_proxy.exe"
Start the Cloud SQL Proxy
[full_path_to_program]\cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
OR
[full_path_to_program]\cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:3306 -credential_file=[PATH_TO_KEY_FILE]
Connect using mysql.exe: `mysql -u USERNAME -p --host 127.0.0.1
Public IP Method
The second method is not secure unless you also enable SSL, which I will not cover in this answer. I do not recommend this method except for knowledgeable persons.
Figure out what your public IP address is. Go to any website such as "MXToolbox" https://mxtoolbox.com/whatismyip/
Go to the Google Cloud Console
Go to the Cloud SQL Instances Details page
Select the Connections tab
Click Add network
In the Network field, enter your Public IP address using CIDR notice: eg 145.11.22.33/32
Enter a name such as "My Home IP"
Click Done
Click Save
Note: Most home Internet services change your Public IP address periodically. This means that you will need to update your Public IP address in Cloud SQL if you are using the second method.

Error connecting from Google AppEngine Standard to Google Cloud SQL instance (Using NodeJS)

I've created a small NodeJS REST API connecting to Google Cloud SQL. using Google AppEngine Flexible environment it works, but when I try to use the Standard environment Google Cloud SQL connection is refused.
I've tried to use the internal IP as well as the socketPath to connect.
I've created a repository to showcase the error: https://github.com/cbullokles/appengine-error
According to documentation, I expect to connect to MySQL in both cases, but it is not working as expected. Is it a non-documented limitation? Or maybe I didn't read correctly AppEngine docs.
The way to connect to Cloud SQL instances from App Engine Standard is using the Unix domain socket for connecting to Cloud SQL instances that provides the App Engine standard environment.
If you use the Node.js mysql package configure it this way to run in App Engine:
const db = mysql.createConnection ({
socketPath: '/cloudsql/<PROJECT_NAME>:<REGION_NAME>:<INSTANCE_NAME>',
user: 'user',
password: '1234',
database: 'yourDatabase'
});
The <PROJECT_ID>:<REGION>:<INSTANCE_ID> is a unique string for each Cloud SQL instance. It is listed under Instance connection name on the Instance details page for your instance, or under connectionName when using the gcloud sql instances describe <INSTANCE_ID> command.
Also the App Engine Standard official documentation for Node.js has some code samples for connecting using the knex library.

Google App script and local database connectivity

I have mysql server installed on my machine. I would like to connect to the same local mysql instance using Google app script's jdbc service. I am getting connection failed error. So, I would like to confirm:
1. is it possible to connect to mysql instance running on private machine(laptop) using jdbc service.
2. If not is it always mandatory to connect to public IP. What is the alternative to connect to mysql instance running on private machine.
This definitely works, but you need to permit Google's machines to connect to your machine. That means opening a hole in your firewall, possibly setting up port forwarding on your router, etc.
There is an alternative to poking a hole in your security arrangement, it is called Secure Data Connector, and it lets your machine initiate the connection to Google - https://developers.google.com/secure-data-connector/
Unfortunately, I don't believe that SDC supports JDBC yet (I believe it only supports HTTP based communication).
(P.S. One very simple work around is to mirror your local database to some kind of cloud service, and then permit access to that replica from Google).