we have mysql-server(5.5.47)that hosted on physical server. It listen external internet interface(with restrict user access), mysql server intensively used from different places(we use different libraries to communicate with mysql). But sometimes whole mysql server(or network) stuck and stop accept connection, and a clients failed with etimedout(connect)/timeout(recv), even direct connection from server to mysql with mysql cli not working(stuck without any response — seems to be try to establish connections).
First thought was that it is related to tcp backlog, so mysql backlog was increased — but this not help at all.
Issue not repeatable, so last time when this issue happened we sniff traffic, and what we get:
http://grab.by/STwq — screenshot
*.*.27.65 — it is client
*.*.20.80 — it is mysql server
From session we can assume that tcp connection established, but server retransmit SYN/ACK to client(from dump we see that server receive ACK, why retransmit ?), but in normal case mysql must generate init packet and send to client, after connection was established.
It is only screen from 1 session, but all other sessions mostly same, SYN -> SYN/ACK -> ACK -> and server retransmit SYN/ACK up to retries_count.
After restart mysql all get normal immediately after restart. So not sure it is related to network or mysql.
Any thoughts would be appropriate.
Thank you!
Related
We use mediasoup to create our products. However, I am having problems with the transport connection.
The client transport connection state goes disconnected a few eights seconds after connection.
The following log will be output in the chrome console.
mediasoup-client:Transport connection state changed to connected
However, the following log will be output in the chrome console a few eights seconds later
mediasoup-client:Transport connection state changed to disconnected
If the NewProducer is present before the disconnection, the above will not happen.
Do you know the possible causes?
Resolved. I changed the AWS security policy according to the topic below and it worked.
You’ll also need to configure your AWS Security Group to allow TCP/UDP on whatever port range you’re using.
https://mediasoup.discourse.group/t/docker-setup-with-listenips/2557/4
I'm running EC2 with MySQL RDS to serve dynamic websites' content.
The server was down due to 'too many connection' error on RDS database.
As it was urgent, I restart database server straight away and the problem gone.
However, I'm unable to see what queries produce those connection (as I didn't run SHOW PROCESS LISTS before reboot RDS).
CloudWatch show 250+ connections during period of issue which is obviously huge distinction from normal operating on other days.
I try to address the issue by see log in RDS, but there is quite a minimal message there.
The error message
2014-05-03 06:10:08 3628 [Warning] IP address '173.244.206.19' has been resolved to the host name '173.244.206.19.static.midphase.com', which resembles IPv4-address itself.
From above, 173.244.206.19 is not in our IP list both public and private. (but connection open to 0.0.0.0 secured with password which I'm going to limit IP remote in security group soon)
Questions
Is 173.244.206.19.static.midphase.com is something to do with RDS by default. I think this is obviously an attack sign but just would like to confirm.
What does 'resembles IPv4-address itself' mean? As this is RDS database server only, why server does need to resolve DNS?
Are there any way to digging into this for further detail (e.g. to see specific query).
I'm going to prevent this by only limit the IP access along with CloudWatch alarm setting for 10+ connections. Anything else I should do.
Thank you for reading through this guys. I'm the only developer in company start-up which take care for all front-end/backend/application/network. Therefore, apology if there are dummy questions out here.
However, your help would be really appreciated and will save a bit of my life writing the report.
You are maybe the target of a DDOS attack or a brute-force password discovery attempt.
I would report this to AWS support - as they can help to mitigate the effect of the attack.
As a best practice, we do not recommend to use 0.0.0.0/0 as source IP address for incoming connection rule in Security Group.
Try to restrict which IP addresses are authorised to connect to your database.
If you are accessing from on prem network, specify only your on-prem address range.
If you're accessing your database from an app server installed on EC2, use the ID of the App Server Security Group (sg-xxxx) as source authorised to connect to your database.
I am continuously getting below error while connecting to mysql-
Can\'t connect to MySQL server on /ip/address
but mysql is not generating any log for this error. I have below entries in my.cnf file
log=/var/log/mysql_err.log
log-error=/var/log/mysql/mysql_error.log
I am using correct credentials. No issue of conections.
Now how will I determine the exact issue If I'll not get any log ?
Any Idea, how to solve this ?
This is not surprising:
the server can only log an incident it is aware of. However your client does not even reach the server at all! So how should the server know some client has attempted to contact it?
The error message you get clearly indicates that you do have a connection issue.
You can easily make a test to check the most common problems: just open a telnet connection from the system trying to connect to the system the sql server runs on, connect to the mysql port: telnet <ip-of-mysql-server> mysql
On typical unixoid systems "mysql" will be substituted by the "well known port number of mysql, which is 3306. otherwise you have to specify it manually. Do you get a connection at all? I would guess not. This means either the mysql server is not listening where expected (not running or configured otherwise) or the connection is blocked on network level (firewall).
I am trying to connect remotely to SQL Server 2008 R2 Express on a server running Windows Server 2008 R2. I receive the following error:
A network-related or instance-specific error occurred while
establishing a connection to SQL Server. The server was not found or
was not accessible. Verify that the instance name is correct and that
SQL Server is configured to allow remote connections. (provider: TCP
Provider, error: 0 - A connection attempt failed because the connected
party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond.)
I think the problem might be related to my connection string, because I am not sure exactly what it should be. I understand the connection string should take the form:
Data Source=123.123.123.123\InstanceName;Initial Catalog=MyDBName;user id=MyUserName;password=MyPassword;
The instance name of my SQL Server installation takes the form ComputerName\SQLEXPRESS. This is what I see when I log in to Management Studio on the server. So should the connection string begin with
Data Source=123.123.123.123\ComputerName\SQLEXPRESS
This looks wrong because of all the slashes. I have tried to omit the computer name thus:
Data Source=123.123.123.123\SQLEXPRESS
I also read somewhere that you can specify a port so I have also tried
Data Source=123.123.123.123,1433\SQLEXPRESS
because I understand that 1433 is the TCP port used by SQL Server and
Data Source=123.123.123.123,1434\SQLEXPRESS
because I understand that 1434 is the UDP port used by SQL Server browser.
I have also tried every combination of these and they all give the same error.
These are the steps I have taken to enable remote browsing on the server:
In Management Studio, right click the instance, go to Properties, Security and check SQL Server and Windows Authentication Mode. On the connections tab I have checked "Allow remote connections to this computer".
In SQL Server Configuration Manager I have enabled all four connection options (Shared Memory, Named Pipes, TCPIP and Via) under every node they occur. Under protocols for SQL Express I have tried specifying port 1433 and also leaving it blank with TCP Dynamic Ports set to 0 (which I think is meant to enable dynamic ports).
I have created firewall exceptions for TCP Port 1433, UDP Port 1434, and program exceptions for sqlservr.exe and sqlbrowser.exe.
If someone could tell me which of the four versions of the DataSource part of the connection string I should be using it would be a great help, even if it did not solve the problem. It would mean that everything I try subsequently I would only have to test once instead of four times.
Thanks in advance for any help!
Where did you enter the firewall exceptions, on your machine, on the server, or both?
The second connection string is the only really valid one. Can you telnet to the server on port 1433? Can you connect to the instance from Management Studio remotely and the problem is only from your code where the connection string is? Did you try running the same code on the server? Did you try forcing TCP/IP (vs. named pipes/shared memory etc) adding the following parameter to your connection string:
Network=DBMSSOCN;
This question comes up quite a lot and I'm sure you're hitting something that others have hit before. Have you gone through this article, "How to troubleshoot connecting to the SQL Server Database Engine"? Also there are 20+ questions on here that mention this error message and Express, according to this search result. I suggest double-checking your settings against some of the items in those answers that solved the issue for other users.
I've got a trio of Windows servers (data1, data2 and datawitness) that aren't part of any domain and don't use AD. I'm trying to set up mirroring based on the instructions at http://alan328.com/SQL2005_Database_Mirroring_Tutorial.aspx. I've had success right up until the final set of instructions where I tell data1 to use datawitness as the witness server. That step fails with the following message:
alter database MyDatabase set witness = 'TCP://datawitness.somedomain.com:7024'
The ALTER DATABASE command could not be sent to the remote server instance 'TCP://datawitness.somedomain.com:7024'. The database mirroring configuration was not changed. Verify that the server is connected, and try again.
I've tested both port 7024 as well as 1433 using telnet and both servers can indeed connect with each other. I'm also able to add a connection to the witness server from SQL Server Manager on the primary server. I've used the Configuration Manager on both servers to enabled Named Pipes and verify that IP traffic is enabled and using port 1433 by default.
What else could it be? Do I need any additional ports open for this to work? (The firewall rules are very restrictive, but I know traffic on the previously mentioned ports is explicitly allowed)
Caveats that are worth mentioning here:
Each server is in a different network segment
The servers don't use AD and aren't part of a domain
There is no DNS server configured for these servers, so I'm using the HOSTS file to map domain names to IP addresses (verified using telnet, ping, etc).
The firewall rules are very restrictive and I don't have direct access to tweak them, though I can call in a change if needed
Data1 and Data2 are using SQL Server 2008, Datawitness is using SQL Express 2005. All of them use the default instance (i.e. none of them are named instances)
After combing through blogs and KB articles and forum posts and reinstalling and reconfiguring and rebooting and profiling, etc, etc, etc, I finally found the key to the puzzle - an entry in the event log on the witness server reported this error:
Database mirroring connection error 2 'DNS lookup failed with error: '11001(No such host is known.)'.' for 'TCP://ABC-WEB01:7024'.
I had used a hosts file to map mock domain names for all three servers in the form of datax.mydomain.com. However, it is now apparent that the witness was trying to comunicate back using the name of the primary server, which I did not have a hosts entry for. Simply adding another entry for ABC-WEB01 pointing to the primary web server did the trick. No errors and the mirroring is finally complete.
Hope this saves someone else a billion hours.
I'd like to add one more sub answer to this specific question, as my comment on Chris' answer shows, my mirror was showing up as disconnected (to the witness) Apperently you need to reboot (or in my case i just restarded the service) the witness server.
As soon as i did this the mirror showed the Witness connection as Connected!
See: http://www.bigresource.com/Tracker/Track-ms_sql-cBsxsUSH/