I'm trying to deploy my Node.js app on Google App Engine and it deployed fine, but it can't connect to Google Cloud SQL for some reason. Here's what it throws:
Error: connect ENOENT /cloudsql/my-project-id:asia-east1:my-sql-instance
Here's how I configured the connection:
if (process.env.INSTANCE_CONNECTION_NAME) {
exports.mysqlConfig = {
user: process.env.GCLOUD_SQL_USERNAME,
password: process.env.GCLOUD_SQL_PASSWORD,
socketPath: '/cloudsql/' + process.env.INSTANCE_CONNECTION_NAME
}
} else {
// Use settings for localhost
}
I'm using node-mysql module to connect to the database.
The App Engine and Cloud SQL are already in the same project.
My theory is that the App Engine and the Cloud SQL has to be in the same project and same region, but I'm not sure.
Check your logs for any errors during startup using:
the following cmd gcloud app logs tail -s default or,
with the log viewer https://console.cloud.google.com/logs/viewer
Chances are that you have not enabled the Cloud SQL API for your project: https://console.developers.google.com/apis/api/sqladmin/overview
make sure you have added following setting in app.yaml
beta_settings:
# The connection name of your instance, available by using
# 'gcloud beta sql instances describe [INSTANCE_NAME]' or from
# the Instance details page in the Google Cloud Platform Console.
cloud_sql_instances: YOUR_INSTANCE_CONNECTION_NAME
ref:https://cloud.google.com/appengine/docs/flexible/nodejs/using-cloud-sql-postgres
Apparently the order you do things matters...
enable Cloud SQL API
then (re)deploy your app (gcloud app deploy)
When I did deploy -> create databases -> enable sql ipi I got the ENOENT error
For anyone using 2nd gen Cloud Functions - they added a portion in the documentation:
If you're using Cloud Functions (2nd gen) and not Cloud Functions (1st
gen), the following are required (also see Configure Cloud Run):
They go on to list the steps required. They're a bit scary, but do work eventually.
(If you find yourself looking for the SQL Connection in the new Cloud Run revision, notice there is a separate "Connections" tab for this)
Related
I have a Cloud Run container that uses a Serverless Connector to connect to a Cloud SQL instance all in the same project. This configuration works just fine.
I have moved the Cloud SQL instance to another project in the same organisation and setup a Serverless Connector there as per the instructions. I have tested this Serverless Connector with a Cloud Function in the same project that accesses the database and reports the number of rows in a table, this works without problems.
I have now updated the Cloud Run instance to point to the new connector reference. I have used the specified format: projects/PROJECT_ID/locations/europe-west3/connectors/CONNECTOR_NAME. When I release a new revision of the container, I get the error message: "Could not find specified network to attach to app." I see the message "Ready condition status changed to False for Service {service name} with message: Deploying Revision." in the Cloud Run logs for this service.
Any ideas on how to get this working please?
Documentation:
Configuring Serverless VPC Access
Configure connectors in the Shared VPC host project
Info:
Command gcloud compute networks vpc-access connectors describe --region=europe-west3 projects/PROJECT_ID/locations/europe-west3/connectors/CONNECTOR_NAME gives the output:
connectedProjects:
- company-service-dev
- a-project-name
ipCidrRange: 10.8.0.0/28
machineType: f1-micro
maxInstances: 3
maxThroughput: 300
minInstances: 2
minThroughput: 200
name: projects/PROJECT_ID/locations/europe-west3/connectors/CONNECTOR_NAME
network: company-project-servicename
state: READY
The connector MUST be in the same region AND the same project as the Cloud Run service.
The wrong solution is to create a peering between the Cloud Run project VPC and the Cloud SQL project VPC. But it won't work because of network transitivity issue (CLoud SQL to Project create 1 peering and Cloud Run VPC to Project create another peering -> 2 peering in a row aren't transitive).
The correct solution is to create Shared VPC architecture to share the same VPC and therefore not to require to perform peering between project.
Another ack exists: you can create a VPN between Cloud Run project VPC and Cloud SQL project VPC. It's ugly, but it works.
Solved!
Problem: Configuration. There was a VPC created for the Cloud SQL db to get an IP address assigned in. The Serverless Connector was created and had access to the same network. I, mistakenly, thought that was all that is needed. As #guillaume-blaquiere points out, this is for a single project only.
To fix: Create a Shared VPC configuration in the host project. In the Google Cloud Console it was as easy as turning on Shared VPC (VPC Network > Shared VPC). Setup a configuration with pretty much the default options it gives you and then you can use the Serverless Connector reference projects/PROJECT_ID/locations/europe-west3/connectors/CONNECTOR_NAME in your Cloud Run or Cloud Functions and all works just fine!
Note there is a similar post here but not fixing my issue:
I had wamp server which comes with mysql server, the path is there. I installed SDK, able to connect to it:
C:\Program Files (x86)\Google\Cloud SDK>gcloud beta sql connect welynxmysql --user=root
However in Cloud Shell, it throws out an error complaining below:
You need the [cloud_sql_proxy] component to use the sql connect
command. ERROR: (gcloud.beta.sql.connect) The cloud_sql_proxy
component(s) is unavailable through the packaging system you are
currently using. Please consider using a separate installation of the
Cloud SDK created through the default mechanism described at:
https://cloud.google.com/sdk/
What am I missing here? How can I connect to mysql instance/database in Cloud Shell?
Thank you very much.
What am I missing here? How can I connect to mysql instance/database in Cloud Shell?
If you are not able to connect to your Cloud SQL instance, you could try to either follow:
Quickstart for Cloud SQL for MySQL
Quickstart for using the proxy for local testing
You could also try to reset your Cloud Shell (note: this will delete all your files within Cloud Shell) as stated here:
Important: This will permanently delete all files in your home directory.
Then, you could try to do a fresh installation of the Google Cloud's SDK.
If none of those works for you, you could try to authorize your account again within the Cloud Shell as follows:
Before beginning, make sure that your Cloud Shell is using your project. If you are not sure you could also run this command to set your project:
gcloud config set project [PROJECT_ID]
Where PROJECT_ID is the project ID where the Cloud SQL instance is located. Then:
Go to your Cloud Shell and run this command (please notice the sudo at
the beginning):
sudo gcloud auth login
Click on the link displayed on the Cloud Shell and log in with your
Google account.
Copy the code displayed on your screen.
Switch to the Cloud Shell, paste it there and hit enter.
You can now run the command as follows:
gcloud beta sql connect [YOUR_INSTANCE_NAME]
or:
gcloud sql connect [YOUR_INSTANCE_NAME]
Where YOUR_INSTANCE_NAME is the name of your Cloud SQL instance.
I hope it helps.
I have a .net core app installed as a docker on google cloud run, this app that needs to be connected to cloud sql (mysql).
When using the private ip address it, it's not working.
When using public IP, it's working, but It's not a good solution for production.
this is my connection string:
"ConnectionString": "server=10.4.16.6;database=mydb;user=root;pwd=mypwd"
When I create the app, Im able to select the database i need to connect to:
But this is not helping to connect.
The relevant docs are explaining how to do it for python and java explictly.
If you do not want to use public IP then you would need to rely on service account to connect to Cloud SQL. However, .net MySQL driver has no understanding of GCP IAM and Service accounts. So you will need to use a proxy called Cloud SQL Proxy. Cloud SQL Proxy understands IAM and Service accounts.
The flow will basically look like this:
Your app -> Regular MySQL Port -> Cloud SQL Proxy(Installed in the
app's network or locally) -> CloudSQL
You will need to do the following:
Create a service account
Assign the role of Cloud SQL Client to the created service account
Download the service account key in the json format
Set env variable GOOGLE_APPLICATION_CREDENTIALS=C:\Downloaded.json
Download Cloud SQL Proxy
Run it `cloud_sql_proxy -instances=projectname:regioname:instanceid=tcp:3306
At this point you MySQL proxy ready to accept connections at 3306, modify the connection string to take localhost or wherever you installed the Cloud SQL Proxy.
Learn more at About the Cloud SQL Proxy
You can create the Cloud Run app from the console (and select the Cloud SQL Connection) or from the gcloud command line and specify
--add-cloudsql-instances <INSTANCE-NAME> --set-env-vars INSTANCE-CONNECTION-NAME="INSTANCE_CONNECTION_NAME"
These settings automatically enable and configures the Cloud SQL proxy. You can connect to the proxy, from your asp.net Core app, using the unix domain socket using the format: /cloudsql/INSTANCE_CONNECTION_NAME.
I used the following connection string in my appsettings.json and it worked for me:
"Server=/cloudsql/INSTANCE_CONNECTION_NAME;Database=DB_NAME;Uid=USER_NAME;Pwd=PASSWORD;Protocol=unix"
NB. Make sure you have given the service account that your Cloud Run app is running under Cloud SQL Client role in IAM
I'm following the tutorial for deploying a Django app on Google App Engine here. I'm able to run the app locally using cloud_sql_proxy, but when I deploy the app to GAE the app fails with the error:
(2003, "Can't connect to MySQL server on 'localhost' ([Errno 111] Connection refused)")
Checking the error output, it looks like django is trying to connect to the unix socket at /cloudsql/<project>:<region>:<instance> as expected, it's just failing for some reason. What could be causing this issue?
It looks like the problem was with the project I was using for the tutorial. Somehow the Google App Engine service account for that project had gotten its permissions revoked. After restoring those permissions (giving it the Editor role) the tutorial worked on the original project.
You need to change the database connnection in the Django settings.
appengine/standard_python37/django/mysite/settings.py
The settings.py file contains the configuration for your SQL database. The code in settings.py uses the GAE_APPLICATION environment variable to determine whether the app is running on App Engine or running on your local computer:
When the app runs on App Engine, it connects to the MySQL host by using the /cloudsql Unix socket.
When the app runs on your local computer, it connects to the MySQL host by using TCP, which requires a username and password.
Read the bottom of the documentation, here
Edit: make sure you've enabled the Cloud SQL API.
I've created a small NodeJS REST API connecting to Google Cloud SQL. using Google AppEngine Flexible environment it works, but when I try to use the Standard environment Google Cloud SQL connection is refused.
I've tried to use the internal IP as well as the socketPath to connect.
I've created a repository to showcase the error: https://github.com/cbullokles/appengine-error
According to documentation, I expect to connect to MySQL in both cases, but it is not working as expected. Is it a non-documented limitation? Or maybe I didn't read correctly AppEngine docs.
The way to connect to Cloud SQL instances from App Engine Standard is using the Unix domain socket for connecting to Cloud SQL instances that provides the App Engine standard environment.
If you use the Node.js mysql package configure it this way to run in App Engine:
const db = mysql.createConnection ({
socketPath: '/cloudsql/<PROJECT_NAME>:<REGION_NAME>:<INSTANCE_NAME>',
user: 'user',
password: '1234',
database: 'yourDatabase'
});
The <PROJECT_ID>:<REGION>:<INSTANCE_ID> is a unique string for each Cloud SQL instance. It is listed under Instance connection name on the Instance details page for your instance, or under connectionName when using the gcloud sql instances describe <INSTANCE_ID> command.
Also the App Engine Standard official documentation for Node.js has some code samples for connecting using the knex library.