I am working on an Azure app services in conjunction with a flexible mysql database server. I have successfully deployed my website to NodeJS v18.LTS, but my server is Throwing: SequelizeHostNotFoundError: getaddrinfo ENOTFOUND mynameserver.mysql.database.azure.com in my app services log stream. In the following question I find a possible solution by adding the ip address of the connecting host to my database instance instead of a FQDN https://stackoverflow.com/questions/25521755/errorerror-getaddrinfo-enotfound-mysql.
However, this configuration is completely discouraged.
https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/dns-configuration-patterns-for-azure-database-for-postgresql/ba-p/2560287
How can I correctly set up my Flexible Server for MySQL instance to work in my production App Services environment without violating this policy?
this is my connection instance configuration:
const sequelize = new Sequelize(
process.env.DATABASE,
process.env.USER,
process.env.MySQLPASSWORD,
{
host: process.env.HOST, // String conection xxxx.mysql.database.azure.com
dialect: process.env.dialect,
});
here I have an alternate approach of connecting to azure MySQL flexible server where I have used mysql2 npm package.
now here I am directly hard coding config data in the code, but you can easily read the application setting using same way you have used before just make sure that you first reading the respective setting for e.g.: username in a variable and then add that variable while configuring the connection to MySQL .
var username = process.env.USER
Here we use the create connection function to connect to the MySQL database and then use the query function to runa query .
The below is code form an express api:
app.use('/', (req,res)=>{
const mysql = require('mysql2');
const connection = mysql.createConnection({
host:'',
user:'',
database:'',
password:'',
port:'',
});
connection.query("CREATE TABLE TESTTABLE ( TEST int)",(err)=>{
console.log(err);
});
res.send("Hello World");
});
Here I have connected the database to MySQL Workbench where I created the table using the above code.
Here in the server I have disabled the ssl mandate
I have created an EC2 instance, installed and configured MySQL on it (not RDS), I have created a database (airpollutiondata) and a table named (CO2).
I am trying to write a Lambda function (running node.js) that will connect to MySQL (which is running on my EC2 instance) and run a select statement to pull some data.
I have tried everything that I can think of (sample code below without the select statement).
I am hoping that someone might be able to steer me in the correct direction. Here is the code that I have tested unsuccessfully in AWS Lamda.
var con = mysql.createConnection({
host: "localhost",
user: "yourusername",
password: "yourpassword",
database: 'airpollutiondata'
});
con.connect(function(err) {
if (err) throw err;
console.log("Connected!");
});
I tried to find resources online that would steer me in the correct direction, but everything seems to be geared towards using RDS with MySql (unfortunately this isn't the way that I set this up)
You do not provide any information about what exactly does happen. But I will assume that your code does work except for the host field, where you need to replace localhost with the private IP address of your instance.
Make sure that mysql does listen on all interfaces - see https://unix.stackexchange.com/questions/43025/how-to-allow-mysql-remote-connections-via-particular-interface
An AWS lambda does not have by default any access to your VPC, which means it has no way how to connect there. https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html
Make sure that the security group you assign to your lambda can create outbound connections to your EC2.
Make sure that the security group you assign to your EC2 accepts connection from security group of your lambda.
I'm trying to connect to my new AWS RDS I just made.
I followed the "Setting up for RDS" (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html), then the "Tutorial: Create an Amazon VPC for Use with a DB Instance" (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateVPC.html), then the "Creating a MySQL DB Instance and Connecting to a Database on a MySQL DB Instance" (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html) but I'm not able to connect to my DB from my computer or my dedicated server on the web.
Following the previous docs, I have this config :
My DB instance
The VPC
The subnetworks
Example of subnetwork's details :
The first security group :
The second security group, calling the first one :
For the first security group, I put both my private IP and the IP of my dedicated server, and their ports.
I even tried to put 0.0.0.0/0 for SSH and TCP, it didn't work either.
For the DB instance, I tried to add the two security group instead of only the db-securitygroup, it didn't work.
I tried to use a different Port for the DB instance, it didn't work.
With MySQL Workbench or with PDO on my dedicated server, I'm unable to connect to the DB : "SQLSTATE[HY000] [2003] Can't connect to MySQL server on [...]"
I think your security groups are incorrect. If the RDS instance is the only thing you currently have running in the VPC, then you should only have one security group, which is assigned to the RDS server, and that security group should have a rule for port 3306 that allows ingress from your personal IP address, and your dedicated server's IP address.
Take a look to this instruction, pay attention to step 3, 4 and 5. It is for ElasticSearch but I think in your case steps are similar
I have more than 20 lambda function for my mobile app API, as in starting the user based is less so it was all going good but now as the user increase (3000 to 4000) I am facing too many connection issues in my lambda function because of which I started getting internal server error form my API, I know i am missing something while creating the connection in lambda but after lot of hit and try i was not able to find out that missing link, the below code I am using for creating the connection
var con;
exports.handler = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
if (!con || con.state == "disconnected" || con === "undefined") {
con = secret
.then((result) => {
var data = JSON.parse(result.SecretString);
var connection = mysql.createConnection(
{
"host": data.host,
"user": data.username,
"password": data.password,
"database": data.db
}
);
connection.connect();
return connection;
}).catch((err) => {
throw err;
});
}
I have tried adding con.destroy() before sending the response, but it does not seem to solve the problem, so if there is anything else I can do then pls let me know.
It's complicated to know exactly what's going on, my first guess always revolves on setting context.callbackWaitsForEmptyEventLoop = false and storing the connection outside the function scope - both which you've correctly done.
With that said, managing connection pools in Lambda kind of goes the other way serverless is by definition, it "lacks" ephemerality. This does not mean that you can't scale with connections, you'll have to dig deeper infos on your issue.
Jeremy Daly provides good practices on dealing with this on the following posts on his blog:
Reusing DB connections
Managing RDS connections + AWS Lambda
Also, he has made a lib that manages this for you, it's called serverless-mysql - which is built to address this specific issue.
Personal experience: I had trouble with connections + lambdas, and due to this fact I've migrated to their DataAPI solution (I had to migrate my RDS to Aurora Serverless, which is not a big pain) - its GA release was about 2/3 weeks ago.
If you want more info on Aurora SLS, check it out here.
Another way to tackle this kind of issue is to user an AWS RDS Proxy: https://aws.amazon.com/fr/rds/proxy/
Many applications, including those built on modern serverless architectures, can have a large number of open connections to the database server, and may open and close database connections at a high rate, exhausting database memory and compute resources. Amazon RDS Proxy allows applications to pool and share connections established with the database, improving database efficiency and application scalability. With RDS Proxy, failover times for Aurora and RDS databases are reduced by up to 66% and database credentials, authentication, and access can be managed through integration with AWS Secrets Manager and AWS Identity and Access Management (IAM).
I was trying to use AWS Aurora Serverless for MySQL in my project, but I am impossible to connect to it, though I have the endpoint, username, password.
What I have done:
From AWS console managment, I select RDS > Instances > Aurora > Serverless
Leave the default settings
Create database
AWS will only create an AWS Cluster
I open MySQL Workbench, and use endpoint, username, password to connect the database
Ressult:
Your connection attempt failed for user 'admin' from your host to
server at xxxxx.cluster-abcdefg1234.eu-west-1.rds.amazonaws.com:3306:
Can't connect to MySQL server on
'xxxxx.cluster-abcdefg1234.eu-west-1.rds.amazonaws.com' (60)
Did I make any wrong steps ? Please advice me.
****EDIT****
I tried to create another Aurora database with capacity type: Provisioned. I can connect to the endpoint seamlessly with username and password by MySql workbench. It means that the port 3306 is opened for workbench.
About the security group:
From https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/aurora-serverless.html :
You can't give an Aurora Serverless DB cluster a public IP address.
You can access an Aurora Serverless DB cluster only from within a
virtual private cloud (VPC) based on the Amazon VPC service.
You can't access an Aurora Serverless DB cluster's endpoint through an
AWS VPN connection or an inter-region VPC peering connection. There
are limitations in accessing a cluster's endpoint through an
intra-region VPC peering connection; for more information, see
Interface VPC Endpoints (AWS PrivateLink) in the Amazon VPC User
Guide. However, you can access an Aurora Serverless cluster's
endpoint through an AWS Direct Connect connection.
So, aside from SSH-ing through an EC2 instance, you can also access your serverless cluster with mySQL Workbench with AWS Direct Connect.
You can also set up a mySQL Workbench through a RDP connection to a Windows EC2 and access the Serverless cluster. This instance only needs to be up when you need to access the Aurora.
If one of the setups here don't work, the usual suspects are the VPC Security group, firewall rules vs port number configured on the cluster or IAM configuration if connecting using IAM.
One way to connect to an Aurora Serverless DB cluster is by using an Amazon EC2 instance. You cannot
create publicly accessible Aurora Serverless DB clusters in the Preview. This task walks you through
creating a publicly accessible Amazon EC2 instance in your VPC. You can use this Amazon EC2 instance to
connect to an Aurora Serverless DB cluster.
This is directly from the docs provided upon preview signup. Please try creating an EC2 instance and using SSH Tunnel method in your MYSQL Workbench or SQL UI of choice. During the preview the Aurora Serverless is not allowed to be set to publicly accessible.
To connect to Aurora serverless or any database in private subnet you will need a 'jump host' which can be any EC2 instance in a public subnet.
Follow Below Steps:
Open the security group attached to the database, and add new rule as below:-
Type:MYSQL/Aurora, Protocol:TCP, PortRange:3306,
Source:securitygroupofEC2 (you can all security group by entering
'sg-')
Open the security group attached to the EC2, and make port 22 is open. If not, add a new rule as below:-
Type:SSH, Protocol:TCP, PortRange:22, Source:MY IP
Open Workbench, Click New connection
- Standard TCP/IP over SSH
- SSH Hostname : < your EC2 Public IP > #34.3.3.1
- SSH Username : < your username > #common ones are : ubuntu, ec2-user, admin
- SSH KeyFile: < attach your EC2 .pem file>
- MYSQL Hostname: <database endpoint name> #mydb.tbgvsblc6.eu-west-1.rds.amazonaws.com
- MYSQL Port: 3306
- Username : <database username>
- Password: <database password>
Click 'test connection' and boom done!!
A common pattern used by customers for connecting to VPC only services (like Aurora Serverless, Amazon Neptune, Amazon DocDB etc) is to have a middle layer (EC2 instance, or ALB etc) and making the middle layer accessible from outside the VPC. If your use case is just trying out some queries or connecting a workbench, then the easiest thing to do is:
Resolve the DNS of the serverless db and obtain its IP
Create an ALB in your VPC, with a target group to the IP that you found in #1
Create a new security group and attach that to your ALB
Update the SG to allow inbound from where ever you want. If you want public internet access, then allow inbound from all IPs, enable an internet gateway in your VPC, and use a public subnet for your ALB.
Once all of this is done, you would end up with a new DNS - that points to your ALB. Make sure that your ALB is set up correctly by:
Using telnet to connect to your ALB endpoint. telnet alb-endpoint alb-port. If it succeeds, then you have a full end to end connection (not jsut to your ALB, but all the way through).
Verify ALB metrics to make sure that all health checks are passing.
Once this is done, use the ALB endpoint in workbench, and you are good to go.
This pattern is recommended only for non production systems. The concerning step is the one where you resolve the DNS to an IP - that IP is ephemeral, it can change when scale compute or failover happens in the background.
Hope this helps, let me know if you need more details on any step. Here is a related answer for Neptune:
Connect to Neptune on AWS from local machine
We can't connect Aurora Serverless directly from MySQL Workbench as only private IPs assigned to Aurora Serverless, not public IP ones.
We can connect Aurora Serverless from EC2 but can't connect Aurora Serverless through the Mysql Workbench SSH tunnel.
We can't connect Aurora Serverless through ALB as ALB allow only HTTP and HTTPS traffic.
you can telnet ALB-RDS-DNS from local but can't connect to MySQL Workbench
Then what is a solution here;
We can connect Aurora Serverless through NLB as NLB allow traffic over TCP protocol;
Steps 1: Create NLB and add listener Load Balancer Protocol: TCP, and Load Balancer Port
:3306
Step 2: Select the VPC (It should be the same VPC of Aurora Serverless Cluster), and add subnets (public)
Step 3: Navigate to Configure Routing, select Target type: IP, and Protocol: TCP,Port:3306
Step 4: Use DNS Checker to get private IP of Aurora Serverless Cluster, and add those IPs with port 3306
Step 5: Create NLB
Now modify the Security group of Aurora Serverless Cluster, allow traffic from either 0.0.0.0 (not recommended) or VPC CIDR
Now, go to Mysql Workbench and use the NLB DNS name, and try to connect using the correct username and password of Aurora Serverless Cluster.
New AWS Feature: Aurora Serverless v2.0 Public IP Address Available
Like many of you I've been waiting and hoping for this for some time.
As of today April 27, 2022 RDS Aurora MySQL Serverless now has a Public option. You must create a separate security group for that option and set inbound rules.
Copy your endpoint, user, and password and you're good to go.
Look at the Comparison of Aurora Serverless v2 and Aurora Serverless v1 requirements
Worked like a charm for me.
Data API and Query Editor for connecting to Aurora Serverless are now available in some more regions.
https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-rds-data-api-and-query-editor-available-additional-regions/
You should be using an EC2 instance that has access to your dbinstance.
This EC2 instance should have port 22 opened for ssh.
Now use port forwarding from local to EC2 to db instance.
Now in your work bench give hostname 127.0.0.1 and port <forwarded port>.
Aurora serverless does not have public endpoint to connect from any of the ide like MYSQL workbench,Sequel pro etc. But we can connect through cli by launching an instance in same vpc in which aurora serverless resides.
Besides you can checkout cloud9 an aws cloud ide. This is in turn ec2 only but will have UI also and can be shared with teams and bunch of other features.
Initially, I was got stuck in the same scenario
Points to be noted while connecting AWS RDS Aurora
Cant connect Public, you need an EC2 instance with the same region where Aurora is been created.
Aurora Public access should be checked No(it worked for me).
You need to create the security group, where you should add Inbound and Outbound rules(IpAddress of EC2 instances).
Ex: Type = MYSQL/AURORA, Protocol=TCP, PortRange=3306,Source=Custom and your IP Address Range,
modify instance and security group to the instance and apply the changes immediately.
While creating Aurora, u will create MasterName, Pwd, and default schema to connect.
After creating, go to cluster and take the cluster endpoint and log in with your EC2 Instance and with MySQL Workbench, Hostname as your cluster endpoint, username and pwd entered while creating aurora database.
This can be achieved using haproxy
Install Haproxy on Centos-> yum install haproxy
delete existing configuration in this file /etc/haproxy/haproxy.cfg and add the below lines(make sure you replace your RDS endpoint url in below configuration)
global
user haproxy
group haproxy
defaults
retries 2
timeout connect 3000
timeout server 5000
timeout client 5000
listen mysql-cluster
bind 0.0.0.0:3307
mode tcp
server mysql-1 test.cluster-crkxsds.us-west-2.rds.amazonaws.com:3306
After modifying the file,start the haproxy -> service haproxy start
You can connect Aurora RDS in MYSQL Workbench using Public IP with port no 3307
We have installed softether vpn in one of ec2 instance in vpc public subnet. We connected the softether vpn from linux / mac os / windows like regualr vpn. After then we were able to access all the private resources like aws aurora serverless as like regualr endpoints from mysql workbench, pgadmin, etc tools, even the django admin shell commands from local computer.
Hope this should help.
https://www.softether.org/4-docs/1-manual/2._SoftEther_VPN_Essential_Architecture/2.4_VPN_Server_Manager
My guess is your security group is not correctly setup for access. You need to explicitly allow remote access on that port to that instance.
From the official docs:
Two common causes of connection failures to a new DB instance are:
The DB instance was created using a security group that does not authorize connections from the device or Amazon EC2 instance where the
MySQL application or utility is running. If the DB instance was
created in a VPC, it must have a VPC security group that authorizes
the connections. If the DB instance was created outside of a VPC, it
must have a DB security group that authorizes the connections.
The DB instance was created using the default port of 3306, and your company has firewall rules blocking connections to that port from
devices in your company network. To fix this failure, recreate the
instance with a different port.
See here for more information:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToInstance.html