I see frequently when I run mysqladmin proc or when I review the MySQL Server process list a user marked with: unauthenticated user trying to connect.
+-----+----------------------+--------------+-----------------+---------+------+------------------+------------------+-----------+---------------+-----------+
| Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined | Rows_read |
+-----+----------------------+--------------+-----------------+---------+------+------------------+------------------+-----------+---------------+-----------+
| 40 | unauthenticated user | x.x.x.x:xxxx | | Connect | | Reading from net | | 0 | 0 | 0 |
What may causes such thing?
Is that normal, or should I investigate my system for any vulnerability or security breach?
Thanks
unauthenticated user is the user connected and not yet sent authentication credentials. Doesn't look like a hack attempt to me.
Related
Consider following list of connections:
+----------+---------+------+------------------------+
| ID | COMMAND | TIME | STATE |
+----------+---------+------+------------------------+
| 87997796 | Sleep | 15 | cleaned up |
| 90850182 | Sleep | 105 | cleaned up |
| 88009697 | Sleep | 38 | delayed commit ok done |
| 88000267 | Sleep | 6 | delayed commit ok done |
| 88009819 | Sleep | 38 | delayed commit ok done |
| 90634882 | Sleep | 21 | cleaned up |
| 90634878 | Sleep | 21 | cleaned up |
| 90634884 | Sleep | 21 | cleaned up |
| 90634875 | Sleep | 21 | cleaned up |
+----------+---------+------+------------------------+
After some short time under minute:
+----------+---------+------+------------------------+
| ID | COMMAND | TIME | STATE |
+----------+---------+------+------------------------+
| 87997796 | Sleep | 9 | cleaned up |
| 88009697 | Sleep | 32 | delayed commit ok done |
| 88000267 | Sleep | 9 | delayed commit ok done |
| 88009819 | Sleep | 31 | delayed commit ok done |
| 90634882 | Sleep | 14 | cleaned up |
| 90634878 | Sleep | 14 | cleaned up |
| 90634884 | Sleep | 14 | cleaned up |
| 90634875 | Sleep | 14 | cleaned up |
+----------+---------+------+------------------------+
8 rows in set (0.02 sec)
enter code here
After I finished writing this stackoverflow post:
+----------+---------+------+------------------------+
| ID | COMMAND | TIME | STATE |
+----------+---------+------+------------------------+
| 87997796 | Sleep | 0 | cleaned up |
| 88009697 | Sleep | 53 | delayed commit ok done |
| 88000267 | Sleep | 0 | delayed commit ok done |
| 88009819 | Sleep | 52 | delayed commit ok done |
| 90634882 | Sleep | 5 | cleaned up |
| 90634878 | Sleep | 5 | cleaned up |
| 90634884 | Sleep | 5 | cleaned up |
| 90634875 | Sleep | 5 | cleaned up |
+----------+---------+------+------------------------+
Context:
This is some 3rd vendor app opening connections (source code isn't available to us, so we don't know details). We know that their connection management is awful , they know it as well. It is awful because connections leak which you can see in first table - 90850182. If others have their timers reset, then this one starts to age infinitely. In older versions of the app it would stay forever. In newer version it is eventually captured by a "patch" which vendor introduced , which effectively cleans connections after the x seconds you specify. So it's "a leak healing patch".
The problem:
We are hosting hundreds of such vendor apps and most of them have much more than 8 connections as they have more traffic. That results in disgusting number(talking thousands) of connections we have to maintain. About 80% of connections sit in "cleaned up" state and under 120 seconds (cleaned eventually by aforementioned configurable app parameter).
This is all handled by Aurora RDS and AWS engineers told us that if the app doesn't close properly connections the standard "wait_timeout" isn't going to work. Well, "wait_timeout" becomes useless decoration in AWS Aurora, but let us take it with Jeff in other thread/topic.
So regardless, we have this magic configurable parameter from third party vendor set on this obscure app which controls eviction of stale connections and it works.
The questions:
Is it safe to evict connections which are in "cleaned up" state immediately?
At the moment this happens after 120 seconds which results in huge number of such connections. Yet in the tables above you can see that the timers are reset meaning that something is happening to these connections and they are not entirely stale. I.e. connection pooling of the app "touches" them for further re-use?
I don't posses knowledge of connection pools inner guts as how they are seen from within database. Are all reserved connections of a connection pool by default are "sleeping" in "cleaned up" state?
So say if you start cleaning too much, you will fight connection pool aggressively creating more to replenish?
Or reserved connections have some different state?
Even if you don't fully understand the context I'd expect veteran DBA or connection pool library maintainer to help with such questions. Otherwise will get my hands dirty and answer this myself eventually, would try apache connection pool, hikari, observe them and try to kill their idle connections (simulating magic parameter) and try this 3rd party app connection with 0 seconds magic parameter, see if it still works.
Appreciate your time :bow:.
The Answer
Yes, from AWS forum (https://forums.aws.amazon.com/thread.jspa?messageID=708499)
In Aurora the 'cleaned up' state is the final state of a connection
whose work is complete but which has not been closed from the client
side. In MySQL this field is left blank (no State) in the same
circumstance.
Also from the same post:
Ultimately, explicitly closing the connection in code is the best
solution here
From my personal experience as a MySQL DBA, and knowing that "cleaned up" represents a blank state, I'd definitely kill those connections.
USE mysql;
DROP PROCEDURE IF EXISTS ShowUsers;
DELIMITER $
CREATE PROCEDURE `ShowUsers`(IN KnownUsers varchar(500), IN KnownHosts varchar(500))
BEGIN
SELECT
user,host
FROM
user
WHERE
NOT FIND_IN_SET(host, KnownHosts)
AND
NOT FIND_IN_SET(user, KnownUsers)
ORDER BY user, host ASC;
END $
DELIMITER ;
Example complete data to work with:
+-------------+-------------+
| user | host |
+-------------+-------------+
| knownuser1 | 192.168.1.5 |
| knownuser2 | 192.168.1.5 |
| unknownuser | 192.168.1.5 | # I want this result to show
| someuser1 | 192.168.1.6 |
| someuser2 | 192.168.1.6 |
| someuser3 | 192.168.1.6 |
| root | localhost |
+-------------+-------------+
I have marked the result I would want to show from running the procedure, basically the two IN parameters are known users, and known hosts those that should be have a user record on this database.
Calling the function like this
# users and hostnames(ips) to match for exclusion from results.
SET #Usernames = 'knownuser1,knownuser2';
SET #Hostnames = '192.168.1.5';
CALL ShowUsers(#Usernames, #Hostnames);
Expected Result:
+-------------+-------------+
| user | host |
+-------------+-------------+
| unknownuser | 192.168.1.5 | # I want this result to show
| someuser1 | 192.168.1.6 |
| someuser2 | 192.168.1.6 |
| someuser3 | 192.168.1.6 |
| root | localhost |
+-------------+-------------+
Actual Result:
+-------------+-------------+
| user | host |
+-------------+-------------+
| someuser1 | 192.168.1.6 |
| someuser2 | 192.168.1.6 |
| someuser3 | 192.168.1.6 |
| root | localhost |
+-------------+-------------+
Explanation (off this topic but I think I should clarify) The reason I want this procedure to work, I have a master server with multiple remote slaves, the slaves need to have access to the masters database which means they also have to have "root" access, they can create/reconfigure their own access credentials. The problem with this is if one of those servers were ever compromised it would leave open the chance to have a new user added with credentials to basically all of the database. Wide open and free to take.
I could lock the slaves out after initial configuration and manually open up the door, run an update and then lock it again which would be pretty laborious for the application and make the application virtually useless.
The idea I'm going with right now is to run this procedure via cron run script and check for unknown users/hosts and lock that slave server out of the database until I accept or reject the user from the main application.
The condition in the WHERE clause is:
NOT FIND_IN_SET(host, KnownHosts) AND NOT FIND_IN_SET(user, KnownUsers)
which is equivalent to:
NOT (FIND_IN_SET(host, KnownHosts) OR FIND_IN_SET(user, KnownUsers))
which means that you want to exclude the rows for which:
host is included in KnownHosts or user is included in KnownUsers.
So for your sample data, the row:
unknownuser | 192.168.1.5
will not be returned, because host = '192.168.1.5' and it is included in KnownHosts (= '192.168.1.5').
Maybe change the logical operator to OR, if this is the logic that you want to apply:
NOT FIND_IN_SET(host, KnownHosts) OR NOT FIND_IN_SET(user, KnownUsers)
how to send mail alert for MySQL?
can we send alerts when the MySQL has large number of connections, or MySQL is not responding properly. Can someone help me to solve this prolem ?
You can do this in a number of ways. SHOW FULL PROCESSLIST; query would give you information about the number of connections as well as the queries being executed by each connection(thread). A sample result is as follows.
mysql> SHOW FULL PROCESSLIST;
+------+------+--------------------+------+---------+------+-------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+------+--------------------+------+---------+------+-------+-----------------------+
| 1298 | root | 192.168.1.76:37648 | NULL | Sleep | 0 | | NULL |
| 1491 | root | localhost | NULL | Query | 0 | init | show full processlist |
+------+------+--------------------+------+---------+------+-------+-----------------------+
If you are only concerned with the number of current connection(threads) you can use the following query.
mysql> SHOW STATUS WHERE `variable_name` = 'Threads_connected';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| Threads_connected | 2 |
+-------------------+-------+
Now about the mail alerts, you can setup a cron job(shell script) to fire a mail alert as soon as the number of current connections exceed a certain limit. mail command can be used for this.
$ echo "Max MySQL Connections reached"| mail -s "your subject" your#email.com
Also, I came across a great MySQL Monitoring tool- MONyog. It would let you setup mail alerts for any of the MySQL variable.
I have a problem with the iax clients when I try to used them with dynamic realtime, I can stablish a call with an extension
I'am using Centos 6, asterisk 11 and mysql
The only visible problem is when a iax client try to register asterisk log this:
CLI> ...WARNING: chan_iax2.c:4452 realtime_peer: Failed to parse sockaddr '(null)' for ipaddr of realtime peer 'ipaddr'
I use this table for iax clients:
+------+----------+--------+---------+--------+---------+----------+-------+
| name | username | type | host | secret | context | disallow | allow |
+------+----------+--------+---------+--------+---------+----------+-------+
| 1001 | 1001 | friend | dynamic | 1001 | agents | all | ulaw |
| 1002 | 1002 | friend | dynamic | 1002 | agents | all | ulaw |
+------+----------+--------+---------+--------+---------+----------+-------+
regards
The warning message stop when is establish the parameter rtcachefriends in the general section of iax.con file:
/etc/asterisk/iax.conf
[general]
rtcachefriends=yes
The call could not be establish because a mistake in the extconfig.conf:
Different from 'sip' for 'iax' one have to explicitly specify 'peers' and 'users'
/etc/asterisk/extconfig.conf:
sippeers => mysql,asterisk,sippeers
extensions => mysql,asterisk,extensions
iaxpeers => mysql,asterisk,iaxfriends
iaxusers => mysql,asterisk,iaxfriends
I found that my mysql sever have many of connection who is sleep. i want to delete them all.
so how i can configure my mysql server than then delete or dispose the connection who is in sleep not currently in process.
are this possible to delete this thing in mysql tell me how i can do following
a connection allow only one time datareader open and destroy the connection [process] after giving resposnse of query.
If you want to do it manually you can do like this:
login to Mysql as admin:
mysql -uroot -ppassword;
And than run command:
mysql> show processlist;
You will get something like below :
+----+-------------+--------------------+----------+---------+------+-------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-------------+--------------------+----------+---------+------+-------+------------------+
| 49 | application | 192.168.44.1:51718 | XXXXXXXX | Sleep | 183 | | NULL ||
| 55 | application | 192.168.44.1:51769 | XXXXXXXX | Sleep | 148 | | NULL |
| 56 | application | 192.168.44.1:51770 | XXXXXXXX | Sleep | 148 | | NULL |
| 57 | application | 192.168.44.1:51771 | XXXXXXXX | Sleep | 148 | | NULL |
| 58 | application | 192.168.44.1:51968 | XXXXXXXX | Sleep | 11 | | NULL |
| 59 | root | localhost | NULL | Query | 0 | NULL | show processlist |
+----+-------------+--------------------+----------+---------+------+-------+------------------+
You will see complete details of different connections. Now you can kill the sleeping connection as below:
mysql> kill 52;
Query OK, 0 rows affected (0.00 sec)
Why would you want to delete a sleeping thread? MySQL creates threads for connection requests, and when the client disconnects the thread is put back into the cache and waits for another connection.
This reduces a lot of overhead of creating threads 'on-demand', and it's nothing to worry about. A sleeping thread uses about 256k of memory.
you can find all working process execute the sql:
show process;
and you will find the sleep process, if you want terminate it, please remember the processid and excute this sql:
kill processid
but actually you can set a timeout variable in my.cnf:
wait_timeout=15
connect_timeout=10
interactive_timeout=100
for me with MySql server on windows,
I update the file (because cannot set variable with sql request due privileges):
D:\MySQL\mysql-5.6.48-winx64\my.ini
add the lines:
wait_timeout=61
interactive_timeout=61
restart service, and acknowledge new values with:
SHOW VARIABLES LIKE '%_timeout';
==> i do a connection tests and after 1 minutes all 10+ connections in sleep are disapeared!