Calling a MySQL query from a shell script after some time interval - mysql

I wanted to execute a MySQL query from a shell script after some time. I have written the script but the output is not as expected. It is printing the same value again from the database. Can someone please help.
SCRIPT- demo.sh
#!/bin/bash
echo "Shell Script to Query a Database!"
echo "--Querying from the Database starts--"
cd "C:\Program Files\MySQL\MySQL Server 5.7\bin"
for i in {1..5};
do
./mysql -u root -proot#123 << EOF
use catalyst_latest;
select id from ci_master limit 1;
EOF
echo "Wait for 2 seconds for the next ID."
sleep 2;
done
echo "--Query Stopped!--"
OUTPUT
$ ./demo.sh
Shell Script to Query a Database!
--Querying from the Database starts--
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
--Query Stopped!--
Can you see that 282 getting returned every time? So i want the next ID from the database after 2 seconds. Please tell me how to do it.
Thank you in advance.

You need to remember the returned id value into the bash variable and then reusing that variable in WHERE clause of the query like this:
select id from ci_master where id > $id limit 1
To write output of the command into variable use something like this:
id=$(mysql <options> -Nsqe "<query>")
Option N to skip column names (like id), option s for more silent output, option q for not caching result, option e for the command to execute.
Initially the $id variable should be set to some value like 1 or the minimum of existing values in the table.

#!/bin/bash
echo "Shell Script to Query a Database!"
echo "--Querying from the Database starts--"
cd "C:\Program Files\MySQL\MySQL Server 5.7\bin"
for i in {1..10};
do
temp=`./mysql -u root -proot#123 << EOF
use catalyst_latest;
select id from ci_master limit $i;
EOF`
result=''
for j in $temp
do
result=$j
done
#date;
echo "$result"
echo "Wait for 1 seconds for the next ID."
sleep 1;
done
echo "--Query Stopped!--"

Related

Getting MySQL query result without error messages (suppress error messages)

I need to suppress all error messages that the mysql command prints to stdout. I saw many other similar questions but all answers suggest workarounds to avoid these messages (check database/table/column exists before executing query). But I need the mysql command to return a failure exit code on error and don't print anything to output except the data explicitly requested in a query in a successful run. The -s key doesn't help in hiding error messages.
My task is to execute a MySQL query in a script and get either the requested data (printed with the -s key) or a non-zero exit code. I don't want to check each and every table/column/etc existence before executing a target query. How can I achieve this?
UPD: I tried this but it didn't help:
mysql ... db -s -N -e "SELECT config_id FROM core_config_data LIMIT 1;" 2> /dev/null
To sum it up:
You want the mysql command to:
not print any error
exit with a non-success code when a query fails
Then I've got good news for you! The output of errors will always be on stderr. Therefore you can just redirect the output to null or whatever you like.
root#icarus ~/so # mariadb -Dmaio290sql1 -e 'SELECT * FROM wp_users' -s 2> bla.txt
[actual content]
root#icarus ~/so # echo $?
0
root#icarus ~/so # mariadb -Dmaio290sql1 -e 'SELECT * FROM nope' -s 2> bla.txt
root#icarus ~/so # echo $?
1
The last query is throwing an error and therefore the exit code is not 0.
This was tested on MariaDB though: 10.3.27-MariaDB-0+deb10u1 Debian 10.

Bash mysql error handling locked tables

i have a shell script that is called with a few parameters very frequently. it is supposed to build a query and execute the statement. in case an error occures, it should write the arguments seperated into a file so the error-handling can take place by calling that script again.
everything works but
the problem is, i catch connection refused error etc but if the statement cannot be executed because the table is locked and i do not want to wait for the timeout.
my code:
...
mysql -u ${username} -p${password} -h ${database} -P ${port} --connect-timeout=1 --skip-reconnect -e "$NQUERY"
mysqlstatus=$?
if [ $mysqlstatus -ne 0 ]; then
echo "[ERROR:QUERY COULD NOT BE EXECUTED:$mysqlstatus: QUERY WRITTEN TO LOG]" >> ${GENLOG}
#echo ${NQUERY} >> ${FQUER}
for i in "$#"; do
ARGS="$ARGS $i|"
done
echo "${ARGS}" >> ${ARGLOG}
else
echo "[OK] $NQUERY" >> ${GENLOG}
fi
...
but when a table is locked, the executing is not canceled and it runs like forever..
its not a solution for me to set the Max_statement_time_set or anything on the mysql server, since im not the only one using the db
You can use the timeout command along with mysql
timeout 3 mysql -u ...
This will wait 3 seconds for the mysql command to return, if the command runs longer then 3 seconds timeout will return exit status 124 to the shell. If you don't have timeout you can use job control with something like this.
#background the process
mysql -u ... &
#get pid of background process
bg_pid=$!
sleep 3
#check if your pid is still running
#using string matching incase pid was re assigned
if [[ $(ps -p $bg_pid -o args --no-headers) =~ "mysql" ]]
then
echo "running to long"
else
echo "OK"
fi

way to connect mysql which executes only once inside the for loop in bash script?

I have a script which will execute Insert Query n times - for that i have used FOR loop , but the problem is the command which connects to remote mysql also executes n times. Here is the script for the better idea for my problem.
#!/bin/bash -X
#fields: id|alias|booking_time|contact_no|deleted|grace|number_in_queue|pax|seated_time|status|walk_in_time|queue_id|user_id
echo "Bash version ${BASH_VERSION}..."
for i in {1..5..1}
do
_alias="Name$i"
_contact_no=$(cat /dev/urandom | tr -dc '1-9' | fold -w 10 | head -n 1)
_deleted="FALSE"
_number_in_queue=$i
_pax=$(( $RANDOM % 10 + 20 ))
_status="waiting"
_queue_id=424
_user_id=550
mysql -u root -p restbucks << EOF #Want this to execute only One time
INSERT INTO queue_item VALUES ('','$_alias',now(),'$_contact_no','$_deleted',NULL,'$_number_in_queue','$_pax',now(),'$_status',now(),'$_queue_id','$_user_id');
EOF
done
Everytime i try to run the script , it will ask me for the password. What i want is that only once the connection made.
You can move the mysql connect before the for loop.
mysql -u root -p restbucks << EOF #this execute only One time
for i in {1..5..1}
do
.....
done
EOF
Also it is recommended that you can write your queries into a file and then finally execute the file using single connection.
You can refer bulk-mysql-query
That is because it is placed inside the for loop. Every time the control passes to the loop it gets executed. You don't need to connect to your server once you are connected. Try placing the statement before the FOR statement.

mysql connection terminates when i logoff from the remote server

I had written a script that connects to the local mysql server every 6 seconds and checks if there is any data in the table .if there is data it runs some php commands and then deletes that data from the table. I logged into my remote server(Shared hosting) through ssh and then copied the script and executed it using command "nohup ./script.sh 0<&- &>alert.log &" so that it runs in background and writes all the output to alert.log file. my problem is that when i log in to the server through SSH and execute the script it runs perfectly , but when i log out from server its not running . when i check the alert.log file after it is showing error "cannot connect to local mysql server". any solutions ??
this is the code
while true
do
res=($(mysql -u root -p123456 --skip-column-names -Dtest -e "select id from temptab"))
if [[ "$res" > 0 ]];then
del=`mysql -u root -p123456 -Dtest -e "delete from temptab;" `
now="$(date +'%d/%m/%Y:%H.%M.%S')"
for ((i=0; i < ${#res[#]}; i++))
do
php -n /var/lib/mysql/trigger.php ${res[$i]}
echo "[$now]:Trigger called with videoid ${res[$i]}"
done
fi
sleep 6
done
and this is the sample output
cat nohup.out
X-Powered-By: PHP/5.4.20
Content-type: text/html
{"multicast_id":8864856209398719411,"success":2,"failure":1,"canonical_ids":0,"results":[{"message_id":"0:1385797766832904%4f0c6467f9fd7ecd"},{"error":"InvalidRegistration"},{"message_id":"0:1385797766832901%4f0c6467f9fd7ecd"}]}81Inserted police info
[30/11/2013:00.49.26]:Trigger called with videoid 65
/etc/bashrc: line 14: whoami: command not found
/etc/bashrc: line 20: grep: command not found
/etc/bashrc: line 59: dircolors: command not found
./alert.sh: line 15: php: command not found
[30/11/2013:07.50.27]:Trigger called with videoid 70
./alert.sh: line 15: /ramdisk/php/54/bin/php54: No such file or directory
[30/11/2013:09.09.52]:Trigger called with videoid 71
screen is what you need. There are plenty of tutorials on google on screen usage.
I suggest to move your code into a crontab even that will run every X minutes (5 minutes, or anything else you like) rather than have your user run it during a live session.
Just place the PHP script inside a call to cron, login, and run crontab -e then add:
*/5 * * * * /home/username/phpscript.php
You could try to run your script like:
/path/to/script.sh </dev/null &>/home/yourname/alert.log &
disown
I
finally i got the solution .... i was using /usr/bin/php to call my php files but .....when i edited it to /usr/bin/php.orig it started working........... but what is that php.orig...?? #all thanks

How can I stop a MySQL query if it takes too long?

Is it possible to timeout a query in MySQL?
That is, if any query exceeds the time I specify, it will be killed by MySQL and it will return an error instead of waiting for eternity.
There is a nice Perl script on CPAN to do just this:
http://search.cpan.org/~rsoliv/mysql-genocide-0.03/mysql-genocide
One only needs to schedule it to run with the proper parameters. Create a CRONtab file /etc/cron.d/mysql_query_timeout to schedule it to run every minute:
* * * * * root /path/to/mysql-genocide -t 7200 -s -K
Where 7200 is the maxiumum allowed execution time in seconds. The -s switch filters out all except SELECT queries. The -K switch instructs the script to kill the matching processes.
The root user should be able to run local mysql tools without authentication otherwise you will need to provide credentials on the command line.
I just set up the following bash script as a cron job to accomplish this with MySQL 5.0 (kills any query that has been executing for more than 30 seconds). Sharing it here in case it proves useful to anyone (apologies if my bash scripting style is inefficient or atrocious, it is not my primary development language):
#!/bin/bash
linecount=0
processes=$(echo "show processlist" | mysql -uroot -ppassword)
oldIfs=$IFS
IFS='
'
echo "Checking for slow MySQL queries..."
for line in $processes
do
if [ "$linecount" -gt 0 ]
then
pid=$(echo "$line" | cut -f1)
length=$(echo "$line" | cut -f6)
query=$(echo "$line" | cut -f8)
#Id User Host db Command Time State Info
if [ "$length" -gt 30 ]
then
#echo "$pid = $length"
echo "WARNING: Killing query with pid=$pid with total execution time of $length seconds! (query=$query)"
killoutput=$(echo "kill query $pid" | mysql -uroot -ppassword)
echo "Result of killing $pid: $killoutput"
fi
fi
linecount=`expr $linecount + 1`
done
IFS=$oldIfs
I thought it has been around a little longer, but according to this,
MySQL 5.7.4 introduces the ability to set server side execution time limits, specified in milliseconds, for top level read-only SELECT statements.
SELECT
MAX_STATEMENT_TIME = 1000 --in milliseconds
*
FROM table;
Note that this only works for read-only SELECT statements.
Starting with MySQL 5.1 you can create a stored procedure to query the information_schmea.PROCESSLIST table for all queries that match your criteria for "long running" then iterate over a cursor to kill them. Then setup that procedure to execute on a recurring basis in the event scheduler.
See: http://forge.mysql.com/tools/tool.php?id=106
The MySQL forum has some threads about this.
This post details how to set up timeouts on the server using innodb_lock_wait_timeout.
Here's a way to do it programmatically, assuming you're using JDBC.
I think this old question needs an updated answer.
You can set a GLOBAL timeout for all your read-only SELECT queries like this:
SET GLOBAL MAX_EXECUTION_TIME=1000;
The time specified is in milliseconds.
If you want the timeout only for a specific query, you can set it inline like this:
SELECT /*+ MAX_EXECUTION_TIME(1000) */ my_column FROM my_table WHERE ...
MySQL returns an error instead of waiting for eternity.
Note that this method only works for read-only SELECTs. If a SELECT statement is determined not to be read-only, then any timer set for it is cancelled and the following NOTE message is reported to the user:
Note 1908 Select is not a read only statement, disabling timer
For statements with subqueries, it limits the top SELECT only. It does not apply to SELECT statements within stored programs. Using the MAX_EXECUTION_TIME hint in SELECT statements within a stored program will be ignored.
I don't think the egrep above would find "2000".
Why not try just selecting the id as well, and avoiding all of that posh shell stuff:
mysql -e 'select id from information_schema.processlist where info is not null and time > 30;'
Since MySQL 5.7.8 there is max_execution_time option that defines the execution timeout for SELECT statements.
Here is my script :
mysql -e 'show processlist\G' |\
egrep -b5 'Time: [6-9]{3,10}' |\
grep 'Id:' |\
cut -d':' -f2 |\
grep -v '155' |\ ## Binary Log PID
sed 's/^ //' |\
while read id
do
mysql -e "kill $id;"
done