I'm running EC2 with MySQL RDS to serve dynamic websites' content.
The server was down due to 'too many connection' error on RDS database.
As it was urgent, I restart database server straight away and the problem gone.
However, I'm unable to see what queries produce those connection (as I didn't run SHOW PROCESS LISTS before reboot RDS).
CloudWatch show 250+ connections during period of issue which is obviously huge distinction from normal operating on other days.
I try to address the issue by see log in RDS, but there is quite a minimal message there.
The error message
2014-05-03 06:10:08 3628 [Warning] IP address '173.244.206.19' has been resolved to the host name '173.244.206.19.static.midphase.com', which resembles IPv4-address itself.
From above, 173.244.206.19 is not in our IP list both public and private. (but connection open to 0.0.0.0 secured with password which I'm going to limit IP remote in security group soon)
Questions
Is 173.244.206.19.static.midphase.com is something to do with RDS by default. I think this is obviously an attack sign but just would like to confirm.
What does 'resembles IPv4-address itself' mean? As this is RDS database server only, why server does need to resolve DNS?
Are there any way to digging into this for further detail (e.g. to see specific query).
I'm going to prevent this by only limit the IP access along with CloudWatch alarm setting for 10+ connections. Anything else I should do.
Thank you for reading through this guys. I'm the only developer in company start-up which take care for all front-end/backend/application/network. Therefore, apology if there are dummy questions out here.
However, your help would be really appreciated and will save a bit of my life writing the report.
You are maybe the target of a DDOS attack or a brute-force password discovery attempt.
I would report this to AWS support - as they can help to mitigate the effect of the attack.
As a best practice, we do not recommend to use 0.0.0.0/0 as source IP address for incoming connection rule in Security Group.
Try to restrict which IP addresses are authorised to connect to your database.
If you are accessing from on prem network, specify only your on-prem address range.
If you're accessing your database from an app server installed on EC2, use the ID of the App Server Security Group (sg-xxxx) as source authorised to connect to your database.
Related
I am trying to connect to a MySQL database from Data Fusion, but I am getting the following error. Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. The database is accessed through public IP through port 3306, from my machine I can connect perfectly, but from Data Fusion I cannot.
As John Hanley pointed out in his comment, it's probably due to a connectivity issue with your SQL instance.
A possible reason would be that you have not enabled your instance to be connected via its public IP. If that's the case, go to your SQL instance and edit its configuration, adding a network (if you haven't done so previously) and providing an IP range to include your Data Fusion instance. Keep in mind that if you configure your instance to accept connections using its public IP address, also configure it to use SSL to keep your data secure. If that was the issue, now you should be able to connect properly.
Also, be sure to check that the Dataproc cluster that your data Fusion instance is using under the hood has the proper configuration (you shouldn't worry about this if you haven't changed anything about the Dataproc cluster).
This is the best advice I can give without further details. If this doesn't work for you, we're going to need more information.
My two cents:
I use one windows pc with several linux box at home.
My main station is the Windows one.
I'v been using Sqlyog just to prob tables on the remote db.
I use my class C addr. to address the server.
It works well. By design I want to access the db server
from anywhere in my home...
60% to 75%ishhh problems of that kind would be related
to the .cnt file configuration.
Regards
Steph
Right now I am connecting to a cluster endpoint that I have set up for an Aurora DB-MySQL compatible cluster, and after I do a "failover" from the AWS console, my web application is unable to properly connect to the DB that should be writable.
My setup is like this:
Java Web App (tomcat8) with HikariCP as the connection pool, with ConnecterJ as the driver for MySQL. I am evaluating Aurora-MySQL to see if it will satisfy some of the needs the application has. The web app sits in an EC2 instance that is in the same VPC and SG as the Aurora-MySQL cluster. I am connecting through the cluster endpoint to get to the database.
After a failover, I would expect HikariCP to break connections (it does), and then attempt to reconnect (it does), however, the application must be connecting to the wrong server, because anytime a write is hit to the database, a SQL Exception is thrown that says:
The MySQL server is running with the --read-only option so it cannot execute this statement
What is the solution here? Should I rework my code to flush DNS after all connections go down, or after I start receiving this error, and then try to re-initiate connections after that? That doesn't seem right...
I don't know why I keep asking questions if I just answer them (I should really be more patient), but here's an answer in case anyone stumbles upon this in a Google search:
RDS uses DNS changes when working with the cluster endpoint to make it looks "seamless". Since the IP behind the hostname can change, if there is any sort of caching going on, then you can see pretty quickly how a change won't be reflected. Here's a page from AWS' docs that go into it a bit more: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-jvm-ttl.html
To resolve my issue, I went into the jvm's security file and then changed it to be 0 just to verify if what was happening was correct. Seems correct. Now I just need to figure out how to do it properly...
I have a hosted MySql server with many databases each with many views. I access the server remotely from my office using HeidiSql.
Once in a while the IP address of my office changes. When this happens, I have to add the new office IP address to the server using cPanel and the "Remote MySql" tool so that I can remotely connect again.
However, all of the views have definer: USER#OLD_IP_ADDRESS. If I need to change a view, I get Access Denied. Up to now, I have been deleting the view (yes, i can delete the view) and recreating it, which makes the view's definer USER#NEW_IP_ADDRESS, and then I can edit the view -- until the Ip address changes again.
So, my question is: What is the best practice is an environment like this. Is there a way to define the views once and, without causing a security risk, be able to edit the views after an IP Address change.
Thanks for any guidance.
You could loosen the host in the MySQL user account a bit, according to the IP range of your internet provider. That way, you don't need to change it every time you get a slightly different IP:
user#123.456.789.%
user#123.456.%
This of course loosens the security in this account, but if your password is be a good one, it should not be too risky.
A better way is to make the MySQL server only accessible via SSH. In that case, your MySQL server can be set up using the --skip-networking option, as you always connecting from localhost. Your user account can be user#localhost or user#127.0.0.1, which then would solve your above mentioned problem for all times.
HeidiSQL also supports SSH tunneled MySQL connections, see here for some tutorial.
I went through these instructions:
http://docs.amazonwebservices.com/AmazonRDS/latest/GettingStartedGuide/
But still can't connect to the database using mysql from the command line.
I even tried:
telnet xxxxxxx.clpkcufglfdn.us-east-1.rds.amazonaws.com 3306
but that doesn't connect either.
I did set up the DB security group using the CIDR suggested by the console (I hope it give me the right value).
Update: Firewalls are disabled on my end. It would be odd if amazon had firewalls on their end and wouldn't mention it in their documentation.
Update2: I'm behind a wireless router which had assigned my machine the IP of 10.0.0.2 so I tried adding that to the security groups but still can't connect.
Update: Firewalls are disabled on my end. It would be odd if amazon had firewalls on their end and wouldn't mention it in their documentation.
Amazon's security groups are a firewall.
I did set up the DB security group using the CIDR suggested by the console (I hope it give me the right value).
If it's not the IP you're connecting from, it's not the right value.
Sounds like you have a firewall that is dropping your connection. Either it is on your end or Amazon's end; probably need to talk to your IT group and if that isn't the issue then Amazon's group to see if you missed a step.
If you are trying to access a RDS instance from within the same AWS datacenter you need to specify a local internal IP address (i.e. 10.x.x.x) and not the public dns/ip as your connection simply doesn't go out of the lan
You can find the local IP address of your EC2 box by typing: ifconfig OR checking your login name (i.e. ubuntu#10.x.x.x)
I didn't find this anywhere and wasted a lot of time. I hope this helps!
I've got a trio of Windows servers (data1, data2 and datawitness) that aren't part of any domain and don't use AD. I'm trying to set up mirroring based on the instructions at http://alan328.com/SQL2005_Database_Mirroring_Tutorial.aspx. I've had success right up until the final set of instructions where I tell data1 to use datawitness as the witness server. That step fails with the following message:
alter database MyDatabase set witness = 'TCP://datawitness.somedomain.com:7024'
The ALTER DATABASE command could not be sent to the remote server instance 'TCP://datawitness.somedomain.com:7024'. The database mirroring configuration was not changed. Verify that the server is connected, and try again.
I've tested both port 7024 as well as 1433 using telnet and both servers can indeed connect with each other. I'm also able to add a connection to the witness server from SQL Server Manager on the primary server. I've used the Configuration Manager on both servers to enabled Named Pipes and verify that IP traffic is enabled and using port 1433 by default.
What else could it be? Do I need any additional ports open for this to work? (The firewall rules are very restrictive, but I know traffic on the previously mentioned ports is explicitly allowed)
Caveats that are worth mentioning here:
Each server is in a different network segment
The servers don't use AD and aren't part of a domain
There is no DNS server configured for these servers, so I'm using the HOSTS file to map domain names to IP addresses (verified using telnet, ping, etc).
The firewall rules are very restrictive and I don't have direct access to tweak them, though I can call in a change if needed
Data1 and Data2 are using SQL Server 2008, Datawitness is using SQL Express 2005. All of them use the default instance (i.e. none of them are named instances)
After combing through blogs and KB articles and forum posts and reinstalling and reconfiguring and rebooting and profiling, etc, etc, etc, I finally found the key to the puzzle - an entry in the event log on the witness server reported this error:
Database mirroring connection error 2 'DNS lookup failed with error: '11001(No such host is known.)'.' for 'TCP://ABC-WEB01:7024'.
I had used a hosts file to map mock domain names for all three servers in the form of datax.mydomain.com. However, it is now apparent that the witness was trying to comunicate back using the name of the primary server, which I did not have a hosts entry for. Simply adding another entry for ABC-WEB01 pointing to the primary web server did the trick. No errors and the mirroring is finally complete.
Hope this saves someone else a billion hours.
I'd like to add one more sub answer to this specific question, as my comment on Chris' answer shows, my mirror was showing up as disconnected (to the witness) Apperently you need to reboot (or in my case i just restarded the service) the witness server.
As soon as i did this the mirror showed the Witness connection as Connected!
See: http://www.bigresource.com/Tracker/Track-ms_sql-cBsxsUSH/