I am using Google App Engine and Cloud SQL (My SQL 2nd Gen Instance) in my Node JS application and I am having troubles.
I am connecting fine in local using cloud proxy, but as soon as I deploy it fails.
I am using Sequelize to connect to cloud SQL.
I do have correct password/username/instance connection name.
I have enabled API, otherwise I would not be able to connect from proxy.
In my "app.yaml" I do have correct "beta_settings" tag with correct "cloud_sql_instances" value.
Message from Cloud SQL:
`{insertId: "s=7560d77466cb46cebc3933acc1c41082;i=1a5ff5;b=30867f100f0d483ea84a5718c3948aed;m=886b8bfcbb;t=578fd8d7ad54f;x=a821eba2c35efe29-0#a1"
logName: "projects/[CORRECT PROJECT]/logs/cloudsql.googleapis.com%2Fmysql.err"
receiveTimestamp: "2018-10-24T18:21:25.190291090Z"
resource: {
labels: {
database_id: "[CORRECT DB ID]"
project_id: "[CORRECT PROJECT ID]"
region: "us-central"
}
type: "cloudsql_database"
}
severity: "ERROR"
textPayload: "2018-10-24T18:21:19.249532Z 53509 [Note] Access denied for user 'root'#'cloudsqlproxy~66.85.23.1' (using password: NO)"
timestamp: "2018-10-24T18:21:19.249743Z" }`
Message fron APP Engine (Express, Sequelize):
`Error: connect ECONNREFUSED 127.0.0.1:3306 at TCPConnectWrap.afterConnect [as oncomplete]`
It seems like similar question has been asked numerous times, and I have read more than 10 stackoverflow questions and answers and I still do not have right answer for it.
Can I get some help?
Thank you.
I know this is old - but I've been having the same problem with my NodeJS app connecting to CloudSQL Postgres. A little digging on the AppEngine instance eventually showed that while the docs show 127.0.0.1 as the host, it's actually running the CloudSQL proxy docker image - so while it's 127.0.0.1 from the instance itself, my database hostname actually needed to be "cloudsql" from the app container.
Try connecting to cloudsql:3306 and see if it works!
Related
I'm trying to connect to the MySQL server on PlanetScale, but can't as it requires SSL.
Here's their doc for that, but it's unclear what it says.
https://planetscale.com/docs/concepts/secure-connections
Here's the connection URL: DATABASE_URL='mysql://co30rXXXXXXX:pscale_pw_XXXXXXX#hoqx01444p30.us-east-4.psdb.cloud/restaurant?ssl={"rejectUnauthorized":true}'
Here's what I see from my terminal when I run yarn run migration-run
yarn run v1.22.18 $ npx prisma migrate dev Environment variables
loaded from .env Prisma schema loaded from prisma/schema.prisma
Datasource "db": MySQL database "restaurant" at
"hoqx0XXXXX.us-east-4.psdb.cloud:3306"
Error: Migration engine error: unknown error: Code: UNAVAILABLE server
does not allow insecure connections, client must use SSL/TLS
error Command failed with exit code 1. info Visit
https://yarnpkg.com/en/docs/cli/run for documentation about this
command.
Is there anyone who has tried to connect to PlanetScale DB from Node.js on localhost? I have tried some other suggestions from Stackoverflow, but don't seem to work.
?ssl={"rejectUnauthorized":false}&sslcert=/etc/ssl/certs/ca-certificates.crt
Adding these params at the end of the connection link, the issue has been fixed. :)
SSL ISSUE ON WINDOWS
If you're working on a Windows machine and using a .env file for your connection string, here is what worked for me to run locally (windows does not have a default /etc/ssl/certs/ reference as answered here).
You get your connection string from the PlanetScale console, via "overview" > "connect"
This will look something like:
DATABASE_URL='mysql://xxxxxx:*****#aws-eu-west-1.connect.psdb.cloud/dbName?ssl={"rejectUnauthorized":true}'
When plainly using this you will most likley get the follow error message (as the question states):
Code: UNAVAILABLE server does not allow insecure connections, client must use SSL/TLS
You therefore need to provide a local cert, one can be downloaded from the following trusted location:
https://curl.se/docs/caextract.html
Next, you need to save this file to a logical location on disk that can be referenced in your connection string, for example c:/temp/cacert.pem
Once saved you can then append then following to your connection string:
&sslcert=C:\\temp\\cacert.pem
Restart your server and you should be all set! 🎉
The equivelant ssl cert update in NodeJs would look as follows:
const connection = mysql.createConnection({
host: 'hostNameHere',
user: 'userNameHere',
password: 'passwordHere',
database: 'dbHere',
ssl: {
ca: fs.readFileSync('C:\\temp\\cacert.pem')
}
});
I deploy a docker container with Ghost inside, to the Google Cloud Run.
The Cloud Run service has a service account with Cloud SQL Client role.
I've added the SQL instance into the connections of the Cloud Run Service.
The Ghost's configuration file is the below property
"database": {
"client": "mysql",
"connection": {
"socketPath": "/cloudsql/xxxxxxx",
"user": "xxxxxxx",
"password": "xxxxxxx",
"database": "ghost1"
}
},
I have an Google Cloud SQL - MySQL instance up and running. I can connect to it through Public IP and using the same credentials.
After I deploy the container, I am getting a "We'll be right back" page on the Ghost.
When I look into the logs on each side, I see some errors which I do not understand the root cause.
Examples of the logs at Google Cloud SQL:
2021-11-14T06:40:37.183921Z 6971 [Warning] User 'mysql.session'#'localhost' was assigned access 0x8000 but was allowed to have only 0x0.
2021-11-14T06:49:09.008652Z 7002 [Note] Aborted connection 7002 to db: 'ghost1' user: 'xxxxxxx' host: 'cloudsqlproxy~107.178.207.100' (Got an error reading communication packets)
2021-11-14T06:50:29.721121Z 7471 [Note] Got timeout reading communication packets
Examples of the logs at Google Cloud Run
DatabaseError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call? at DatabaseError.KnexMigrateError
I have tried a lot of combinations like using VPC connector and Private IP but I keep getting the same network timeout errors all the time. I suspect that the Ghost mysql adaptor library (knex) is doing something wrong but I am not sure whether that is true and if there is something I can do about it.
Thanks for your help
For what it's worth, we have an example app that connects over Unix sockets.
As long as you're connecting to public IP, you won't need a Serverless VPC Access connector or a Private IP.
Also, you might double check that your socket path is correct. It should look like this: /cloudsql/project-name:region:instance-name.
There's a similar question here that might help:
How do I run Ghost with MySQL on GCP?
Make sure you allow SSL connections only and create a new certificate chain for your instance.
"ssl": {
"cert": "<database_cert>",
"ca": "<database_ca>",
"key": "<database_key>"
}
example here
Activating SSL connections only and adding the certificate chain did not work for me.
I am using the unix socketPath to connect from Ghost to a CloudSQL database like mentioned above: /cloudsql/project-name:region:instance-name.
When running Ghost version 5.x.x the db connection stopped working after some random time. The Cloud Run logs showed:
Error: connect ETIMEDOUT
and
Error: Connection lost: The server closed the connection.
I tested my configuration by running Ghost on localhost, using the cloudsql-proxy for authentication and it worked without problems.
I finally got it working on Cloud Run by patching the node_modules/knex/lib/client.js. Re-initialise the connection pool and try to reconnect to the database in case the pool has been destroyed.
The approach is also explained here in more detail: https://stackoverflow.com/a/69191743/2546954
I am trying to connect to cloud sql hosted in gcp from eclipse, not able to do so. I have whitelisted my ip using the connection settings on cloud sql instance. I have added the driver also in eclipse for mysql.
I think I am making mistake in constructing the connection string. Please refer below details.
connectionName: abc-dev:europe-west1:abc-instance
So in connection section
database: abc-dev:europe-west1:abc-instance
url: jdbc:mysql://35.233.100.100:3306/abcInsoles
user: root
password: xyz123
It throws exception:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
I am running it from local.
Can you throw some insights?
Many thanks.
The "database" value is incorrect - this is the name of the database on the instance itself (not the instance's connection string).
Since you are using Java, you may be interested in the Cloud SQL JDBC Socket Factory - it allows you to connect to the Cloud SQL instance without whitelisting an IP.
I have followed this guide https://cloud.google.com/community/tutorials/run-sailsjs-on-google-app-engine
on how to deployed Sails to Google App Engine.
However, my app used MySQL as the database and I have set it up like this in "connection.js".
mysql: {
adapter: 'sails-mysql',
host: '104.199.196.99', // Internal IP Address of the Cloud SQL Instance
user: 'root',
password: 'xxxxxx',
database: 'xxxxxxxx'
}
The error I got when trying to connect with Cloud SQL (when I make a requrest to log-in which involved connection to database) was:
Sending 500 ("Server Error") response:
Error (E_UNKNOWN) :: Encountered an unexpected error
Could not connect to MySQL:
Error: connect ETIMEDOUT
Did I set it up correctly?
For those who seek the solution for connecting to Google Cloud SQL using Google App Engine with Sails.js, I have figured it out.
Connecting to the Google Cloud SQL using host address doest not work here, because the Firewall Rules and the Access Control of the Cloud SQL will prevent this kind of connection to the whitelisted addresses. Unfortunately, Google App Engine could not be assigned with Static IP Address. Therefore, the App Engine could not be whitelisted by the Cloud SQL or you would need to whitelist 0.0.0.0/0 which is everything and that is not secure.
The solution is to use this configuration in your connection.js.
googleCloudSQL: {
adapter: 'sails-mysql',
socketPath: '/cloudsql/' + process.env.INSTANCE_CONNECTION_NAME,
user: 'XXXXX',
password: 'XXXXXXX',
database: 'XXXXXXXX'
}
By connecting to the Cloud SQL via socket path using the Instance Connection Name of the Cloud SQL, this is the native way of doing it. If your App Engine is already in the same project with your Cloud SQL then it will already be authorized to connect. However, if they are in different projects, you would need to set up service account and IAM.
I'm trying to connect my server code running as a Docker container in our Kubernetes cluster (hosted on Google Container Engine) to a Google Cloud SQL managed MySQL 5.7 instance. The issue I'm running into is that every connection is being rejected by the database server with Access denied for user 'USER'#'IP' (using password: YES). The database credentials (username, password, database name, and SSL certificates) are all correct and work when connecting via other MySQL clients or the same application running as a container on a local instance.
I've verified that all credentials are the same on the local and the server-hosted versions of the app and that the user I'm connecting with has the wildcard % host specified. Not really sure what to check next here, to be honest...
An edited version of the connection code is below:
let connectionCreds = {
host: Config.SQL.HOST,
user: Config.SQL.USER,
password: Config.SQL.PASSWORD,
database: Config.SQL.DATABASE,
charset: 'utf8mb4',
};
if (Config.SQL.SSL_ENABLE) {
connectionCreds['ssl'] = {
key: fs.readFileSync(Config.SQL.SSL_CLIENT_KEY_PATH),
cert: fs.readFileSync(Config.SQL.SSL_CLIENT_CERT_PATH),
ca: fs.readFileSync(Config.SQL.SSL_SERVER_CA_PATH)
}
}
this.connection = MySQL.createConnection(connectionCreds);
Additional information: the server application is built in Node using the mysql2 library to connect to the database. There are no special firewall rules in place that are causing network issues, and that's confirmed by the fact that the library IS connecting, but failing to authenticate.
After setting up Cloud SQL Proxy I managed to figure out what the actual error was: somewhere between the secret and the pod configuration an extra newline was being added to the database name, causing any connection attempt to fail. With the proxy set up this was made clear because there was an actual error message to that effect displayed.
(notably all of my logging around the credentials that I was using to validate that the credentials were accurate didn't explicitly display the newline and was disguised by the fact that the console display added line breaks to wrap the display, and it happened to line up exactly with where the database name ended)
Have you read the documentation on https://cloud.google.com/sql/docs/mysql/connect-container-engine ?
In Container Engine, you need to set up a Cloud SQL Proxy container alongside your application pod and talk to it. The Cloud SQL Proxy will then make the actual call to Cloud SQL service.
If the container worked locally, I assume you have Application Default Credentials set on your development machine. It could be failing because those credentials are not on your container as a Service Account file. Try configuring a Service Account file, or create your GKE cluster with --scopes argument that gives your instances access to Cloud SQL.