Closing mysql connection when executed from shell script - mysql

Using shell script to run mysql insert query, while running the query i am seeing connections are not being closed from shell script.
Server Response after running the shell script
$ date
Tue Feb 20 15:43:58
$ netstat -alnp | grep 3306 | wc -l
26
Where above 26 counts were like
tcp6 0 0 192.168.10.169:31503 192.168.10.170:3306 ESTABLISHED 11603/java
$ netstat -alnp | grep 3306 | wc -l
50
Where above 50 counts were like (TIME_WAIT - 22) and (ESTABLISHED - 28)
tcp6 0 0 192.168.10.169:48308 192.168.10.170:3306 ESTABLISHED 12603/java
tcp6 0 0 192.168.10.169:48990 192.168.10.170:3306 TIME_WAIT
$ date
Tue Feb 20 15:46:49
Does mysql connection ran through shell script doesn't get closed by self
What is greater impact if shell script with mysql insert command ran via cron job at every 30 minutes
Script
#!bin/bash
query="insert into table_name values ('foo', 'bar' , 123, NOW() )where column_name is NOT NUll"
mysql -u username -p password mysql <<EOF
$query;
EOF
What will be impact on mysql maximum connections, While ran to my system i got more than 100 connections ESTABLISHED

What STATUS value corresponds to connections ESTABLISHED?
What was the value of
`SHOW GLOBAL STATUS LIKE 'Max_used_connections';
I expect a small number like 1 or 2.
For
`SHOW GLOBAL STATUS LIKE 'Connections';
I expect over 100.
The commandline tool mysql will create a connection (bumping Connections and possibly increasing the "high water mark" for Max_used_connections), do the action, then close the connection (without decreasing any STATUS). Threads_running is also incremented and decremented.
Your cron job should not be threatening any limitations.

Related

MySQL remote connection takes more than 3 minutes to display result

I am trying to connect from a local server to my remote server with mysql command...
Result after long delay:
[root#local ~] mysqlshow -u test -p*** -h XXX.XXX.XXX.XXX
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading authorization packet', system error: 0
(The connection is done through my second remote server without any error or delay)
Update 1
Retry connecting from local server after add connect_timeout=500 to the remote server's /etc/my.cnf:
[root#local ~] time mysqlshow -u test -p*** -h XXX.XXX.XXX.XXX
+---------------------+
| Databases |
+---------------------+
| foo |
| bar |
+---------------------+
real 3m21.174s <======== (3m!)
user 0m0.004s
sys 0m0.015s
Successful connection but with more than 3 minutes delay to display the result!
The interesting thing is that when the connection is established, other mysql requests (e.g. mysql> SELECT) take effect immediately!
Update 2
Same result after add skip-host-cache and skip-name-resolve to remote server's /etc/my.cnf...
Update 3
Same request through my second remote server: (Everything seems good)
[root#remote2 ~] time mysqlshow -u test -p*** -h XXX.XXX.XXX.XXX
+---------------------+
| Databases |
+---------------------+
| foo |
| bar |
+---------------------+
real 0m0.016s <======== (0.016s)
user 0m0.007s
sys 0m0.002s
Update 4
Check MySQL Connection from local server with telnet: (Everything seems good!)
[root#local ~] time echo X | telnet -e X XXX.XXX.XXX.XXX 3306
Telnet escape character is 'X'.
Trying XXX.XXX.XXX.XXX...
Connected to XXX.XXX.XXX.XXX.
Escape character is 'X'.
telnet> Connection closed.
real 0m0.136s <======== (0.136s)
user 0m0.000s
sys 0m0.005s
Update 5
Try to connect through PHP mysqli from local server:
<?php
$servername = "XXX.XXX.XXX.XXX";
$username = "test";
$password = "***";
$dbname = "dbname";
$conn = new mysqli( $servername, $username, $password, $dbname );
if( $conn->connect_error ) {
die($conn->connect_error);
}
?>
Result:
Warning: mysqli::__construct(): (HY000/2002): Permission denied in /var/www/html/test.php on line 7
Permission denied
(The PHP mysqli connection is done through the second remote server without any error or delay)
Update 6
Retry with disabled SELinux on the local server solves the PHP connection error but the connection still takes more than 3 minutes!
When the connection is established, other requests (e.g. SELECT) take effect immediately.
Update 7
After trying to connect from local server, On the server's phpMyAdmin (or mysqladmin proc command), new connection added with unauthenticated user...
Update 8
Connection try through XAMPP on windows (from different PC and network) to the remote server:
Same result (connecting with unauthenticated user and more than 3 minutes delay to display results) ...
I'm about to go crazy!!

Losing connection to Cloud SQL while performing a mysqldump

Recently, I've started seeing issues performing a mysqldump on a Cloud SQL instance. It runs for about 20 minutes, then afterwards it fails with the following error:
mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table "table3" at row: 3518748
Then, if I look in the MySQL error log on the CloudSQL console, I see the following:
"2018-06-29T14:46:11.143774Z 4320729 [Note] Aborted connection 4320729 to db: 'bobs_db' user: 'bob' host: '1.2.3.4' (Got timeout writing communication packets)"
Here is the command I run:
mysqldump -h 1.1.1.1 --port=3306 -u bob -pbobs_pwd --net_buffer_length=16m --compatible=ansi --skip-extended-insert --compact --single-transaction --skip-triggers --where="created_at < '2018-06-29 00:00:00' OR updated_at < '2018-06-29 00:00:00'" bobs_db "table1 table2 table3 table4"
Seems everything I read around this issue points to making the net_read_timeout and net_write_timeout bigger but CloudSQL doesn't provide access to those variables from what I can tell.
I'm running with the default database flags except for the following one:
max_allowed_packet=1073741824

Calling a MySQL query from a shell script after some time interval

I wanted to execute a MySQL query from a shell script after some time. I have written the script but the output is not as expected. It is printing the same value again from the database. Can someone please help.
SCRIPT- demo.sh
#!/bin/bash
echo "Shell Script to Query a Database!"
echo "--Querying from the Database starts--"
cd "C:\Program Files\MySQL\MySQL Server 5.7\bin"
for i in {1..5};
do
./mysql -u root -proot#123 << EOF
use catalyst_latest;
select id from ci_master limit 1;
EOF
echo "Wait for 2 seconds for the next ID."
sleep 2;
done
echo "--Query Stopped!--"
OUTPUT
$ ./demo.sh
Shell Script to Query a Database!
--Querying from the Database starts--
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
mysql: [Warning] Using a password on the command line interface can be insecure.
id
282
Wait for 2 seconds for the next ID.
--Query Stopped!--
Can you see that 282 getting returned every time? So i want the next ID from the database after 2 seconds. Please tell me how to do it.
Thank you in advance.
You need to remember the returned id value into the bash variable and then reusing that variable in WHERE clause of the query like this:
select id from ci_master where id > $id limit 1
To write output of the command into variable use something like this:
id=$(mysql <options> -Nsqe "<query>")
Option N to skip column names (like id), option s for more silent output, option q for not caching result, option e for the command to execute.
Initially the $id variable should be set to some value like 1 or the minimum of existing values in the table.
#!/bin/bash
echo "Shell Script to Query a Database!"
echo "--Querying from the Database starts--"
cd "C:\Program Files\MySQL\MySQL Server 5.7\bin"
for i in {1..10};
do
temp=`./mysql -u root -proot#123 << EOF
use catalyst_latest;
select id from ci_master limit $i;
EOF`
result=''
for j in $temp
do
result=$j
done
#date;
echo "$result"
echo "Wait for 1 seconds for the next ID."
sleep 1;
done
echo "--Query Stopped!--"

Delete query execution is very slow when run inside bash script

I'm trying to run following query inside a bash script.
When it is executed from mysql command promt, execution time was 0.06sec.
mysql> delete from assign_history where offer_id not in
->('7','8','9','10','11','12','13','14','32','157','211','240','241','242','273',
->'274','275','310','312','313','314','326','328','329','333','334','335','336',
->'337','342','343','355','362','374','375','376','378','379','383','384','409','411')
->and date(action_date) < "2015-06-25" order by id limit 1000;
Query OK, 1000 rows affected (0.06 sec)
But when run it inside a bash script, it takes more than 2 minutes.
[root#localhost umap]# cat ./history_del.sh
#! /bin/bash
echo $(date)
mysql -uroot -ppassword db_offers -e "delete from assign_history where offer_id not in ('7','8','9','10','11','12','13','14','32','157','211','240','241','242','273','274','275','310','312','313','314','326','328','329','333','334','335','336','337','342','343','355','362','374','375','376','378','379','383','384','409','411') and date(action_date) < "2015-06-25" limit 1000;"
echo $(date)
[root#localhost umap]# ./history_del.sh
Wed Aug 26 19:08:45 IST 2015
Wed Aug 26 19:10:48 IST 2015
I also tried with "mysql -Bse" options. No improvement. Any ideas?
Any ideas?
First, you need to escape double-quotes inside the query string: \"2015-06-25\" (try to output your query with echo and you'll see, why ). I dont know, how your request works without properly specified quotes...
Second, it is better and preferred to place your long-line-request in the file, so your command-line will look like this:
mysql -uroot -ppassword db_offers <YOUR_FILE
Request in YOUR_FILE will be the same as in the mysql prompt (of course, you dont need to escape double-quotes here).
And yes, when you call mysql utility - it can take unpredictably long time to connect to MySQL server, so 2 minutes include this time (but 0.06 sec in mysql prompt doesnt!), so you cant say, how much time does it take to connect to server and how much - to send and execute your query.
To know, how much time does it take to connect to mysql server, try to execute (wait several seconds after previous run of the mysql utility) any empty query, such as:
time mysql -u user -ppassword -Bs <<<'select null'

Login to MySql using Linux terminal not responding

Im trying to login to MySQL in localhost using the root username (default) with no password entering the following command:
mysql -u root -h localhost
I dont get the 'mysql>' and it does not fail so im left with the terminal letting me type but it does not execute any commands (inside or outside of mysql). I also tried using -p and leaving the password blank and I get the same result
Check the following (taken from http://dev.mysql.com/doc/mysql-linuxunix-excerpt/5.5/en/unix-postinstallation.html )
Use mysqladmin to verify that the server is running. The following commands provide simple tests to check whether the server is up and responding to connections:
shell> bin/mysqladmin version
shell> bin/mysqladmin variables
The output from mysqladmin version varies slightly depending on your platform and version of MySQL, but should be similar to that shown here:
shell> bin/mysqladmin version
mysqladmin Ver 14.12 Distrib 5.5.29, for pc-linux-gnu on i686
...
Server version 5.5.29
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 14 days 5 hours 5 min 21 sec
Threads: 1 Questions: 366 Slow queries: 0
Opens: 0 Flush tables: 1 Open tables: 19
Queries per second avg: 0.000
If you are unable to run that command then your installation is either not complete (follow the instructions starting at the top of the page ) or you do not have permission as the user to execute the mysql program.