I want to configure a DB farm in a single node with containers. My idea is to access in each of these DB with a subdomain, for example mysql1.example.com:3306, mysql2.example.com:3306, mysql3.example.com:3306.
I'm trying to implement this model with HAProxy, it seems that the first time that I connect to one database through the HAProxy it works. When I reconnect I get:
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
The template I use in HAproxy is:
global
maxconn 256
debug
defaults
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
listen www
bind *:3306
mode tcp
acl host_mysql hdr(host) -i mysql1.example.com
server mysql_db_1 172.31.20.75:3307
acl host_mysql hdr(host) -i mysql2.example.com
server mysql_db_2 172.31.20.75:3308
acl host_mysql hdr(host) -i mysql3.example.com
server mysql_db_3 172.31.20.75:3309
I auto-respond. It's not possible to create this implemetation due to Mysql uses TCP protocol, so it not include the URL in the header. For this reason HAproxy can't redirect to the correct server.
I'm thinking to implement this environment using virtual IP's assigned to each database. Another implementation would be running all databases in the same server and different ports.
Related
I am facing difficulty in establishing remote database connection (MySQL / MariaDB) to AWS-RDS. On the server side (AWS RDS) everything looks fine and even Inbound rules are set to accept My IP ( TCP Protocol over Port 3306). I am working on Windows machine so the error might be related to Windows Firewall. For this reason, I even added new Outbound and Inbound rule in Firewall Manager and set Remote Port to 3306. But still I end with below connection error:
The following error occurred:
Can't connect to MySQL server on 'xxx.rds.amazonaws.com' (10060) QMYSQL: Unable to connect
Windows Firewall setting (similar for both Outbound & Inbound):
Amazon RDS is publicly accessible but still I am getting 10060 error. Is there something missing in Amazon (like in VPC) or is this caused by windows firewall?
I came across similar posts but those solutions didn't help. AWS RDS connection error
Can anyone please let me know what could be the issue and how to resolve it?
Update
After setting up VPC Flow logs to monitor the traffic, log messages are stored in S3 bucket with interval of 10mins. From 2 different logs, I could find:
version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status
2 xx164 eni-08f88xxxx xx.xx.xxx.119 xxx.xx.2.17 34010 3306 6 5 260 1661944383 1661944424 REJECT OK
2 xx164 eni-08f88xxxx xx.xx.xxx.119 xx.xx.2.17 33953 3306 6 5 260 1661938359 1661938418 REJECT OK
The srcaddr, dstaddr and dstport is same in both cases but, srcport is different. Is this expected behavior? The srcaddr doesn't match with the IP addr set in security group (inbound rule). I don't understand which IP addr is taken as srcaddr.
Whereas doing telnet rdsendpoint 3306 results in Connecting To <rdsendpoint>...Could not open connection to the host, on port 3306: Connect failed
Thanks in advance.
P.S: please let me know if any info is missing here.
I create 1 master and 2 replication in AWS RDS and 1 EC2 with haproxy
listen rds-cluster
bind 172.30.0.xxx:3306
mode tcp
option mysql-check user ha_check
balance roundrobin
server mysql-1 replica1.xxxx.ap-southeast-1.rds.amazonaws.com:3306 check weight 1 fall 2 fastinter 1000
server mysql-2 replica2.xxxx.ap-southeast-1.rds.amazonaws.com:3306 check weight 1 fall 2 fastinter 1000
If I can connect directly using endpoint to replica server,
But if I using haproxy
$ mysql -h172.30.0.xxx -uha_read -ppassword -e "show variables like 'server_id'"
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
i got that error
I already increase connect_timeout
if I check
SHOW GLOBAL STATUS LIKE 'Aborted_connects';
it's keep increasing
===============
This article solve my problem
CUSTOM CONFIGURATION OF AMAZON RDS INSTANCES
by default if you did not change the security group settings when launch RDS, only your IP will be authorized to reach your databases. In your case you need to authorize your haproxy node to reach your databases as well.
Go to RDS, select your instance, then security group, edit, add a new rule to enable either the security group of your HAproxy (best practice) or HAproxy IP (still good enough if this is an elastic IP) to access the database on port 3306.
Hope this is clear enough :)
EDIT: I understand that you solved your issue, but for people reading later (or even for you if you want to enhance security) I add a little information about what I said:
the RDS hostname will be resolved to private IP when the DNS query is made from an instance in the same VPC to the Amazon provided DNS server in that VPC. Thus in your security group, in that case, you would have to allow either the subnet of you haproxy or its private IP (not public one).
(I'm posting this question after the fact because of the time it took to find the root cause and solution. There's also a good chance other people will run into the same problem)
I have an RDS instance (in a VPC) that I'm trying to connect to from an application running on a classic EC2 instance, connected via ClassicLink. Security groups and DNS aren't an issue.
I am able to establish socket connections to the RDS instance, but cannot connect with CLI tools (psql, mysql, etc.) or DB GUI tools like toad or mysql workbench.
Direct socket connections with telnet or nc result in TCP connections in the "ESTABLISHED" state (output from netstat).
Connections from DB CLI, GUI tools, or applications result in timeouts and TCP connections that are stuck in the "SYN" state.
UPDATE: The root cause in my case was a problem with MTU size and EC2 ClassicLink. I've posted some general troubleshooting information below in an answer in case other people run into similar RDS connectivity issues.
Additional information for people who might run into similar issues trying to connect to RDS or RedShift:
1) Check security groups
Verify the security group for the RDS instance allows access from the security group your source server belongs to (or its IP added directly if external to AWS). The security group you should be looking at is the one specified in the RDS instance attributes from the RDS console UI (named "security group").
NOTE: Database security groups might be different from AWS EC2 security groups. If your RDS instance is in classic/public EC2, you should check in the "database security group" section of the RDS UI. For VPC users, the security group will be a normal VPC security group (the name sg-xxx will be listed in the RDS instance's attributes).
2) Confirm DNS isn't an issue.
Amazon uses split DNS, so a DNS lookup external to AWS will return the public IP while a lookup internal to AWS will return a private IP. If you suspect it is a DNS issue, have you confirmed different IPs are returned from different availability zones? If different AZs get different IPs, you will need to contact AWS support.
3) Confirm network connectivity by establishing a socket connection.
Tools like tracepath and traceroute likely won't help since RDS currently drops ICMP traffic.
Test port connectivity by trying to establish a socket connection to the RDS instance on port 3306 (mysql, or 5432 for postgres). Start by finding the IP of the RDS instance and using either telnet or nc (be sure to use the internal/private IP if connecting from within AWS):
telnet x.x.x.x 3306
nc -vz x.x.x.x 3306
a) If your connection attempt isn't successful and immediately fails, the port is likely blocked or the remote host isn't running a service on that port. you may need to engage AWS support to troubleshoot further. If connecting from outside of AWS, try to connect from another instance inside AWS first (as your firewall might be blocking those connections).
b) If your connection isn't successful and you get a timeout, packets are probably being dropped/ignored by a firewall or packets are returning on a different network path. You can confirm this by running netstat -an | grep SYN (from a different ssh session while waiting for the telnet/nc command to timeout).
Connections in the SYN state mean that you've sent a connection request, but haven't received anything back (SYN_ACK or reject/block). Usually this means a firewall or security group is ignoring or dropping packets.
It can also be a problem with NAT routing or multiple paths from multiple interfaces. Check to make sure you're not using iptables or a NAT gateway between your host and the RDS instance. If you're in a VPC, also make sure you allow egress/outbound traffic from the source host.
c) If your socket connection test was successful, but you can't connect with a mysql client (CLI, workbench, app, etc.), take a look at the output of netstat to see what state the connection is in (replace x.x.x.x with the actual IP address of the RDS instance):
netstat -an | grep x.x.x.x
If you were getting a connection established when using telnet or NC, but you see the 'SYN' state when using a mysql client, you might be running into an MTU issue.
RDS, at the time this is written, may not support ICMP packets used for PMTUD (https://en.wikipedia.org/wiki/Path_MTU_Discovery#Problems_with_PMTUD). This can be a problem if you're trying to access RDS or RedShift that's in a VPC from a classic ec2 instance via ClassicLink. Try lowering the MTU with the following, then testing again:
sudo ip link show
# take note of the current MTU (likely 1500 or 9001)
sudo ip link set dev eth0 mtu 1400
If the lower MTU worked, be sure to follow up with AWS customer support for help and mention that you are seeing an MTU issue while trying to connect to your RDS instance. This can happen if TCP packets are wrapped with encapsulation for tunneling, resulting in a lower usable MTU for packet data / payload. Lowering the MTU on the source server allows the wrapped packets to still fit under the MTU limit while passing through the tunneling gateway.
If it didn't work, set your MTU back to it's default and engage AWS support for further troubleshooting.
First of all I apologize for my English is not perfect.
I'm trying to connect to mysql database remotely.
I have already done the basic steps such as, for example, comment the bind address in my.cnf.
I gave permissions to the appropriate users in MySQL.
I tried with my friend to do it connect to the database, and it worked but we were in the same room with the same router.
Now I'm home and I tried to connect my friend, but I always get error:
ERROR 2003 (HY000): Can not connect to MySQL server ....
Maybe it's a problem of my home router? I can not even do the telnet.
P.S. I've also taken steps to set the firewall.
Some reasons for that,
mysqld is not running on the local host. Check your operating
system's process list to ensure the mysqld process is present.
You're running a MySQL server on Windows with many TCP/IP connections
to it. If you're experiencing that quite often your clients get that
error, you can find a workaround here: Section B.5.2.2.1, “Connection
to MySQL Server Failing on Windows”.
Check whether the server is running on that host by executing telnet
some_host 3306 and pressing the Enter key a couple of times. (3306 is
the default MySQL port number. Change the value if your server is
listening to a different port.) If there is a MySQL server running
and listening to the port, you should get a response that includes
the server's version number. If you get an error such as telnet:
Unable to connect to remote host: Connection refused, then there is
no server running on the given port.
Look at this mySQL full documentation, it will be helpful.
I am working on windows and having a remote desktop connection of another machine. I am trying to connect to the mysql running on the remote machine through my MySql workbench, installed on my machine.
When I specify the details to connect to mysql i.e.
IP- a.b.c.d
port-3306
username=root
password=
But everytime it shows me an error saying-
Your connection attempt to connect to user='root'failed from your host to server at a.b.c.d:3306
Is it possible that I can connect to mysql on a different machine?
Certainly that is possible, mysql is a network transparent service. However obviously the normal access authorization rules apply.
do you have network access to the mysql port, this might be blocked by a firewall. You can check that by using telnet ip-address-of-server 3306. Does the server answer or do you get a timeout or are blocked? (the answer would look cryptic, soomething like 5.5.33-MariaDB[*[n7p~g!�iXccI$r9``Y{$mysql_native_password or similar )
the mysql server can be configured to not listen to remote connections. This can add security to the setup, but would obviously block any remote connection attempts. You will have to check the configuration files of mysql for that.
mysql itself implements an authorization level. So check if that 'root' user actually is allowed to connect from the outside. Those authorizations are stored in the internal "mysql" database.
Also it is not clear from the question what your network topology is: is the mysql server running on that system you have a "remote connection" to? Is the mysql-workbench running on that remote system or on your local system? This might affect the ip address you have to use inside the workbench.