I have a shell script how execute some query on a big loop, but sometimes the script doesn't respond.
I want to add a timeout to Mysql query to prevent this bug.
$sql=`mysql --protocol=tcp --user=**** --password=\'****\' --host="vl*****1" --database="C***K" -e "UPDATE C****k SET current_status =\'$status\',last_state_change=\'$lastchange\' WHERE MC_internal_name=\'$InternalName\';"
Tkanks !
Use option:
--connect-timeout=#
Replace dash with number of seconds before connection timeout.
Related
I am running shell scripts which have multiple mysql -u <> -e "select * from tbl" statements. Do i need to explicitly close mysql connection through my shell script? If yes, can you guide me on how to do that?
I am seeing many sleep connections in mysql. Even if i kill them, they are created again in short period an i am suspecting that shell script execution is causing this.
Thanks
If you didn't solve the root cause of the sleep queries it will be restarted and will consume the resources after each time you killed the connection, so you have to check your code and modify it to close the connection after executing the query.
Any way, you can kill any connection by it's ID which can be determined by:
mysql -e 'show processlist;'
The ID will be in the first column and then you can kill them by the following:
mysql -e 'kill $ID'
And to kill them all in one time:
for i in $(mysql -e "show processlist" | awk '/Sleep/ {print $1}'); do mysql -e "KILL $i;"; done
I'm trying to run following query inside a bash script.
When it is executed from mysql command promt, execution time was 0.06sec.
mysql> delete from assign_history where offer_id not in
->('7','8','9','10','11','12','13','14','32','157','211','240','241','242','273',
->'274','275','310','312','313','314','326','328','329','333','334','335','336',
->'337','342','343','355','362','374','375','376','378','379','383','384','409','411')
->and date(action_date) < "2015-06-25" order by id limit 1000;
Query OK, 1000 rows affected (0.06 sec)
But when run it inside a bash script, it takes more than 2 minutes.
[root#localhost umap]# cat ./history_del.sh
#! /bin/bash
echo $(date)
mysql -uroot -ppassword db_offers -e "delete from assign_history where offer_id not in ('7','8','9','10','11','12','13','14','32','157','211','240','241','242','273','274','275','310','312','313','314','326','328','329','333','334','335','336','337','342','343','355','362','374','375','376','378','379','383','384','409','411') and date(action_date) < "2015-06-25" limit 1000;"
echo $(date)
[root#localhost umap]# ./history_del.sh
Wed Aug 26 19:08:45 IST 2015
Wed Aug 26 19:10:48 IST 2015
I also tried with "mysql -Bse" options. No improvement. Any ideas?
Any ideas?
First, you need to escape double-quotes inside the query string: \"2015-06-25\" (try to output your query with echo and you'll see, why ). I dont know, how your request works without properly specified quotes...
Second, it is better and preferred to place your long-line-request in the file, so your command-line will look like this:
mysql -uroot -ppassword db_offers <YOUR_FILE
Request in YOUR_FILE will be the same as in the mysql prompt (of course, you dont need to escape double-quotes here).
And yes, when you call mysql utility - it can take unpredictably long time to connect to MySQL server, so 2 minutes include this time (but 0.06 sec in mysql prompt doesnt!), so you cant say, how much time does it take to connect to server and how much - to send and execute your query.
To know, how much time does it take to connect to mysql server, try to execute (wait several seconds after previous run of the mysql utility) any empty query, such as:
time mysql -u user -ppassword -Bs <<<'select null'
Is there some configuration that can be done at MySQL side to automatically kill or timeout queries that are extremely slow, say 100 seconds.
You can list all your MySQL queries by the following command:
$ mysqladmin processlist
so you can run some script which will parse that list and it'll kill the specific query.
In example, you can run some script in any language via cron to periodically check for the long queries, e.g.:
$result = mysql_query("SHOW FULL PROCESSLIST");
while ($row=mysql_fetch_array($result)) {
$process_id=$row["Id"];
if ($row["Time"] > 200 ) {
$sql="KILL $process_id";
mysql_query($sql);
}
}
Another example:
mysql> select concat('KILL ',id,';') from information_schema.processlist
where user='root' and time > 200 into outfile '/tmp/a.txt';
mysql> source /tmp/a.txt;
Related:
How do I kill all the processes in Mysql "show processlist"?
Read more:
http://www.mysqlperformanceblog.com/
Starting with MySQL 5.1 you can create a stored procedure to query the information_schmea.PROCESSLIST table for all queries that match your criteria ("long running time" in your case). Upon querying processlist table, you can simply iterate over a cursor to kill the running query processes that exceeds your timeout criteria.
Take a look at the following example: Procedure to find and terminate all non-SUPER and "system account" queries running longer than N seconds
http://forge.mysql.com/tools/tool.php?id=106
You should checkout the command pt-kill of percona toolkit
Is it possible to timeout a query in MySQL?
That is, if any query exceeds the time I specify, it will be killed by MySQL and it will return an error instead of waiting for eternity.
There is a nice Perl script on CPAN to do just this:
http://search.cpan.org/~rsoliv/mysql-genocide-0.03/mysql-genocide
One only needs to schedule it to run with the proper parameters. Create a CRONtab file /etc/cron.d/mysql_query_timeout to schedule it to run every minute:
* * * * * root /path/to/mysql-genocide -t 7200 -s -K
Where 7200 is the maxiumum allowed execution time in seconds. The -s switch filters out all except SELECT queries. The -K switch instructs the script to kill the matching processes.
The root user should be able to run local mysql tools without authentication otherwise you will need to provide credentials on the command line.
I just set up the following bash script as a cron job to accomplish this with MySQL 5.0 (kills any query that has been executing for more than 30 seconds). Sharing it here in case it proves useful to anyone (apologies if my bash scripting style is inefficient or atrocious, it is not my primary development language):
#!/bin/bash
linecount=0
processes=$(echo "show processlist" | mysql -uroot -ppassword)
oldIfs=$IFS
IFS='
'
echo "Checking for slow MySQL queries..."
for line in $processes
do
if [ "$linecount" -gt 0 ]
then
pid=$(echo "$line" | cut -f1)
length=$(echo "$line" | cut -f6)
query=$(echo "$line" | cut -f8)
#Id User Host db Command Time State Info
if [ "$length" -gt 30 ]
then
#echo "$pid = $length"
echo "WARNING: Killing query with pid=$pid with total execution time of $length seconds! (query=$query)"
killoutput=$(echo "kill query $pid" | mysql -uroot -ppassword)
echo "Result of killing $pid: $killoutput"
fi
fi
linecount=`expr $linecount + 1`
done
IFS=$oldIfs
I thought it has been around a little longer, but according to this,
MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements.
SELECT
MAX_STATEMENT_TIME = 1000 --in milliseconds
*
FROM table;
Note that this only works for read-only SELECT statements.
Starting with MySQL 5.1 you can create a stored procedure to query the information_schmea.PROCESSLIST table for all queries that match your criteria for "long running" then iterate over a cursor to kill them. Then setup that procedure to execute on a recurring basis in the event scheduler.
See: http://forge.mysql.com/tools/tool.php?id=106
The MySQL forum has some threads about this.
This post details how to set up timeouts on the server using innodb_lock_wait_timeout.
Here's a way to do it programmatically, assuming you're using JDBC.
I think this old question needs an updated answer.
You can set a GLOBAL timeout for all your read-only SELECT queries like this:
SET GLOBAL MAX_EXECUTION_TIME=1000;
The time specified is in milliseconds.
If you want the timeout only for a specific query, you can set it inline like this:
SELECT /*+ MAX_EXECUTION_TIME(1000) */ my_column FROM my_table WHERE ...
MySQL returns an error instead of waiting for eternity.
Note that this method only works for read-only SELECTs. If a SELECT statement is determined not to be read-only, then any timer set for it is cancelled and the following NOTE message is reported to the user:
Note 1908 Select is not a read only statement, disabling timer
For statements with subqueries, it limits the top SELECT only. It does not apply to SELECT statements within stored programs. Using the MAX_EXECUTION_TIME hint in SELECT statements within a stored program will be ignored.
I don't think the egrep above would find "2000".
Why not try just selecting the id as well, and avoiding all of that posh shell stuff:
mysql -e 'select id from information_schema.processlist where info is not null and time > 30;'
Since MySQL 5.7.8 there is max_execution_time option that defines the execution timeout for SELECT statements.
Here is my script :
mysql -e 'show processlist\G' |\
egrep -b5 'Time: [6-9]{3,10}' |\
grep 'Id:' |\
cut -d':' -f2 |\
grep -v '155' |\ ## Binary Log PID
sed 's/^ //' |\
while read id
do
mysql -e "kill $id;"
done
I am trying to create a batch script that would connect to a mySQL database and issue a delete command:
#echo off
echo Resetting all assessments...
mysql -hlocalhost -urdfdev -p%1 rdf_feedback
delete from competency_question_answer;
I will run this script providing the password as a command-line argument, but all this script does is, connects to the database, and the mysql> prompt will be shown. After I exit from mysql, the rest of the batch commands get to execute (and fail, no surprise).
How can I pass the SQL commands from the batch script to the mysql console? Is this even possible?
You need to use command line tools. I don't know if there exists any for MySQL but for SQL there is SQLCMD and for Oracle there is OSQL.
What you can also do is something like this.
mysql -uuser -ppass < foo.sql
Where foo.sql is the commands you want to execute.
You may need to connect multiple times:
#echo off
echo Resetting all assessments...
mysql -hlocalhost -urdfdev -p%1 rdf_feedback -e delete from competency_question_answer;
Alternatively, you should be able to put all your commands in a separate file such as input.sql and use:
mysql -hlocalhost -urdfdev -p%1 rdf_feedback <input.sql
echo "delete from competency_question_answer;" | mysql -hlocalhost -ur... etc.
Putting multiple sets of commands into .sql batch files works best, and you can execute multiples of these in the .bat file.