We have problem to run php script via crontab. There are around 70 lines that run scripts installed in different virtual servers. When few lines are executed everything is working but when we have more some of them are not executed.
First we thought that this is crontab issue, but then we figure out that we are getting error from the script 'Error msg: Unable to connect to the database'. Just to let you now this script allow applications, that we have installed on our VPS to execut of the system's scheduled sending script (so you we can schedule our emails to be sent at a later date) on our server.
By getting this error we know that problem exist in the mysql engine. We are not mysql experts so maybe someone knows what to do with this. Here are few outputs from our mysql.
Uptime: 80832 Threads: 60 Questions: 4116373
Slow queries: 2067 Opens: 69721 Flush tables: 2
Open tables: 400 Queries per second avg: 50.925
and
+--------------------------+--------+
| Variable_name | Value |
+--------------------------+--------+
| Aborted_connects | 64 |
| Connections | 671995 |
| Max_used_connections | 215 |
| Ssl_client_connects | 0 |
| Ssl_connect_renegotiates | 0 |
| Ssl_finished_connects | 0 |
| Threads_connected | 60 |
+--------------------------+--------+
Do you see something suspicious here? Connections are opened only on localhost. In /etc/my.cnf there is a setup
bind-address = 127.0.0.1
Related
I am writing a small web app in Go, which uses mysql to store data.
I got intermittent mysql error if the web sever didn't get any request after some amount of time(> 8 hours):
[mysql] 2017/02/08 16:31:56 packets.go:33: unexpected EOF
[mysql] 2017/02/08 16:31:56 packets.go:130: write tcp 127.0.0.1:49188->127.0.0.1:3306: write: broken pipe
I found some related discussion on github(issue 529, issue 257 and issue 446). From what I understand, mysql db would close the connection if timeout is reached.
I tried to set SetMaxOpenConns to 9 and SetMaxIdleConns to 0 as some people recommended. However, this threw exception immediately. (But if I set SetMaxIdleConns larger than 0, there was no immediate exception thrown)
I also tried to set SetConnMaxLifetime to 5 mins. This threw exception too after 5 mins.
Now I am trying the code below:
db.SetConnMaxLifetime(0)
db.SetMaxOpenConns(10)
db.SetMaxIdleConns(5)
It has been running for 20 mins. It's still too early to tell.
(UPDATE: this doesn't work either)
Here is configuration:
driver: go-sql-driver V1.3.
go version: go1.7.1 darwin/amd64
mysql: latest from docker hub
rkt version: 1.18
CoreOS: 1284.0.0
Perhaps you can start a heartbeat Goroutine to avoid timeout.
you can check your mysql time_wait variable:
mysql> show global variables like 'wait_timeout':
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| wait_timeout | 300 |
+---------------+-------+
1 row in set (0.00 sec)
then use db.SetConnMaxLifetime(120*time.Second), which mean when db connection is idle over than 120s, sql.db will reopen or get a new connection from db pool by db.Open. If you not set connection max life time, you maybe use a closed connection and got the error.
watching the mysql process list，mysql> show processlist;，if connection sleep over than 300s，it's recycled by mysql:
mysql> show processlist;
+-------+-----------------+------------------+-------------+---------+---------+------------------------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-------+-----------------+------------------+-------------+---------+---------+------------------------+------------------+
| 4 | event_scheduler | localhost | NULL | Daemon | 1363480 | Waiting on empty queue | NULL |
| 26539 | root | 172.17.0.1:48732 | NULL | Query | 0 | starting | show processlist |
| 26575 | auditcenter | 172.17.0.1:51714 | obs_gb_test | Sleep | 51 | | NULL |
+-------+-----------------+------------------+-------------+---------+---------+------------------------+------------------+
3 rows in set (0.00 sec)
SetMaxOpenConns and SetMaxIdleConns is used for setting connection resource, see enter link description here
My issue is, my website open too many connections in mysql server this is hanged my website and also generate many requests on apache. i have installed the apache module mod-evasive and mod-security i have also empty the iptables rules basically i am using amazon ec2 so it gives security ip i have blocked all outbound and inbound i have opened only 443,80 and ssh 22 port only but still when i take netstat it shows
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
4 --0.0.0.0
4 ---119.159.195.199
25 ---54.69.254.252
374 ---
on above my question is why my server 374 apache connection on ::1:80 how can block this or reduce this
mysql connection stat are
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| Connections | 5208 |
| Threads_cached | 0 |
| Threads_connected | 54 |
| Threads_created | 5207 |
| Threads_running | 54 |
+-------------------+-------+
5 rows in set (0.00 sec)
my second question is why my sql connections is increasing.
Please any one help me i will really appreciated.
Can anyone tell me how can I kill all the sleeping processes?
I searched for it and I found that we can do it by command
mk-kill --match-command Sleep --kill --victims all --interval 10
I connected the DB server(Linux) but I find the message that command not found.
I tried to connect via MYSQL administrator and it doesn't say that command not found but also doesn't executes the query , just says you have an SQl error
login to Mysql as admin:
mysql -uroot -ppassword;
And than run command:
mysql> show processlist;
You will get something like below :
+----+-------------+--------------------+----------+---------+------+-------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-------------+--------------------+----------+---------+------+-------+------------------+
| 49 | application | 192.168.44.1:51718 | XXXXXXXX | Sleep | 183 | | NULL ||
| 55 | application | 192.168.44.1:51769 | XXXXXXXX | Sleep | 148 | | NULL |
| 56 | application | 192.168.44.1:51770 | XXXXXXXX | Sleep | 148 | | NULL |
| 57 | application | 192.168.44.1:51771 | XXXXXXXX | Sleep | 148 | | NULL |
| 58 | application | 192.168.44.1:51968 | XXXXXXXX | Sleep | 11 | | NULL |
| 59 | root | localhost | NULL | Query | 0 | NULL | show processlist |
+----+-------------+--------------------+----------+---------+------+-------+------------------+
You will see complete details of different connections. Now you can kill the sleeping connection as below:
mysql> kill 55;
Query OK, 0 rows affected (0.00 sec)
kill $queryID; is helpful but if there is only one query causing an issue;
Having a lot of MySQL sleeping processes can cause a huge spike in your CPU load or IO
Here is a simple one-line command (if behind the MySQL server is linux) which would kill all of the current sleeping MySQL processes:
for i in `mysql -e "show processlist" | awk '/Sleep/ {print $1}'` ; do mysql -e "KILL $i;"; done
This is only a temporary repair; I strongly advise identifying and addressing the problem's main cause.
For instance, you may set the wait timeout variable to the amount of time you want MySQL to hold open connections before shutting them.
But if the issue still persists and you have to investigate the DB queries that cause the problem there is another way. In screen session, you can use another while cycle to continuously kill the sleeping queries. (while there is an output of the mysql show processlit | grep -i sleep | awk id column and kill it.) If you are using MySQL replication between different hosts this will help them to catch up. So when using show slave status\G; Seconds_behind_master will be going to catch up.
Of course, you should investigate the root cause again.
My debian-based is booting so slow after I installed MySQL and imported some databases on it. Looking for some statement, I found this one during boot:
mysql> show full processlist;
+----+------------------+-----------+------+---------+------+----------------+----------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+------------------+-----------+------+---------+------+----------------+----------------------------------------------------------------------+
| 9 | debian-sys-maint | localhost | NULL | Query | 12 | Opening tables | select count(*) into #discard from `information_schema`.`PARTITIONS` |
| 10 | root | localhost | NULL | Query | 0 | NULL | show full processlist |
+----+------------------+-----------+------+---------+------+----------------+----------------------------------------------------------------------+
2 rows in set (0.00 sec)
Here the statement that causing trouble:
select count(*) into #discard from `information_schema`.`PARTITIONS`
I have +-10 databases totaling over 8gb of data.
Is there any configuration to disable this query on system booting ? If yes, why run it during boot ?
Information
I have a standard MySQL installation without custom configs.
Best regards.
It seems Debian, whose Linux Mint is based upon, have scripts that get executed when the mysql server is started or restarted, to check for corrupted tables and make an alert for that.
In my Debian server, the culprit seems to be /etc/mysql/debian-start bash script, which in turn calls /usr/share/mysql/debian-start.inc.sh , so check both scripts and comment out the function that is iterating all your tables, from a quick look it seems the following:
check_for_crashed_tables;
which is called from the debian-start script I mentioned above.
I found that my mysql sever have many of connection who is sleep. i want to delete them all.
so how i can configure my mysql server than then delete or dispose the connection who is in sleep not currently in process.
are this possible to delete this thing in mysql tell me how i can do following
a connection allow only one time datareader open and destroy the connection [process] after giving resposnse of query.
If you want to do it manually you can do like this:
login to Mysql as admin:
mysql -uroot -ppassword;
And than run command:
mysql> show processlist;
You will get something like below :
+----+-------------+--------------------+----------+---------+------+-------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-------------+--------------------+----------+---------+------+-------+------------------+
| 49 | application | 192.168.44.1:51718 | XXXXXXXX | Sleep | 183 | | NULL ||
| 55 | application | 192.168.44.1:51769 | XXXXXXXX | Sleep | 148 | | NULL |
| 56 | application | 192.168.44.1:51770 | XXXXXXXX | Sleep | 148 | | NULL |
| 57 | application | 192.168.44.1:51771 | XXXXXXXX | Sleep | 148 | | NULL |
| 58 | application | 192.168.44.1:51968 | XXXXXXXX | Sleep | 11 | | NULL |
| 59 | root | localhost | NULL | Query | 0 | NULL | show processlist |
+----+-------------+--------------------+----------+---------+------+-------+------------------+
You will see complete details of different connections. Now you can kill the sleeping connection as below:
mysql> kill 52;
Query OK, 0 rows affected (0.00 sec)
Why would you want to delete a sleeping thread? MySQL creates threads for connection requests, and when the client disconnects the thread is put back into the cache and waits for another connection.
This reduces a lot of overhead of creating threads 'on-demand', and it's nothing to worry about. A sleeping thread uses about 256k of memory.
you can find all working process execute the sql:
show process;
and you will find the sleep process, if you want terminate it, please remember the processid and excute this sql:
kill processid
but actually you can set a timeout variable in my.cnf:
wait_timeout=15
connect_timeout=10
interactive_timeout=100
for me with MySql server on windows,
I update the file (because cannot set variable with sql request due privileges):
D:\MySQL\mysql-5.6.48-winx64\my.ini
add the lines:
wait_timeout=61
interactive_timeout=61
restart service, and acknowledge new values with:
SHOW VARIABLES LIKE '%_timeout';
==> i do a connection tests and after 1 minutes all 10+ connections in sleep are disapeared!