Subshell MySQL-query in bash - mysql

I'm trying to set variable in the cycle in Ubuntu bash, which is getting recordset from database, but this variable is setting to its previous value.
Here is a code:
#!/bin/bash
PREV_FILE_PATH="127"
while true
do
echo "$PREV_FILE_PATH"
mysql -h$DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME --skip-column-names --default-character-set=UTF8 -e "here is a query" | while read "here is getting variables from recordset";
do
PREV_FILE_PATH="777"
done
done
And this code prints every time:
127
127
127
But whe I replaced this block-:
mysql -h$DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME --skip-column-names --default-character-set=UTF8 -e "here is a query" | while read "here is getting variables from recordset";
with just while true and break at the end of cycle it works fine and prints:
127
777
777
777
Script creates some subshell and running that MySQL query in that subshell. So what should I do to make script change that variable?

As you noted the issue is due to the creation of a subshell which is being caused by piping the output of the mysql command to the while loop. A simple example:
PREV_FILE_PATH=127
echo test | while read _; do
PREV_FILE_PATH=777
done
echo $PREV_FILE_PATH
# output: 127
Since you're using BASH you can move the mysql command from being a pipe to a substituted process fed to the while loop via STDIN redirection. Using the previous simple example:
PREV_FILE_PATH=127
while read _; do
PREV_FILE_PATH=777
done < <(echo test)
echo $PREV_FILE_PATH
# output: 777
So to fix your code, you will want to move your mysql command in the same fashion that I moved the echo command above:
while read "here is getting variables from recordset"
do
PREV_FILE_PATH="777"
done < <(mysql -h$DB_HOST -u $DB_USER [..remaining options..])
Note that process substitution via <() is a BASH-ism and isn't POSIX compliant.

Related

How can I detect if mysqldump fails in a bash script?

I have a small down and dirty script to dump one of the tables all of a client's databases nightly:
#!/bin/bash
DB_BACKUP="/backups/mysql_backup/`date +%Y-%m-%d`"
DB_USER="dbuser"
DB_PASSWD="dbpass"
# Create the backup directory
mkdir -p $DB_BACKUP
# Remove backups older than 10 days
find /backups/mysql_backup/ -maxdepth 1 -type d -mtime +10 -exec rm -rf {} \;
# Backup each database on the system
for db in $(mysql --user=$DB_USER --password=$DB_PASSWD -e 'show databases' -s --skip-column-names|grep -viE '(staging|performance_schema|information_schema)');
do echo "dumping $db-uploads"; mysqldump --user=$DB_USER --password=$DB_PASSWD --events --opt --single-transaction $db uploads > "$DB_BACKUP/mysqldump-$db-uploads-$(date +%Y-%m-%d).sql";
done
Recently we've had some issues where some of the tables get corrupted, and mysqldump fails with the following message:
mysqldump: Got error: 145: Table './myDBname/myTable1' is marked as crashed and should be repaired when using LOCK TABLES
Is there a way for me to check if this happens in the bash script, and log the errors if so?
Also, as written would such an error halt the script, or would it continue to backup the rest of the databases normally? If it would halt execution is there a way around that?
Every program has an exit status. The exit status of each program is assigned to the $? builtin bash variable. By convention, this is 0 if the command was successful, or some other value 1-255 if the command was not successful. The exact value depends on the code in that program.
You can see the exit codes that mysqldump might issue here: https://github.com/mysql/mysql-server/blob/8.0/client/mysqldump.cc#L65-L72
You can check for this, and log it, output an error message of you choosing, exit the bash script, whatever you want.
mysqldump ...
if [[ $? != 0 ]] ; then
...do something...
fi
You can alternatively write this which does the same thing:
mysqldump ... || {
...do something...
}
The || means to execute the following statement or code block if the exit status of the preceding command is nonzero.
By default, commands that return errors do not cause the bash script to exit. You can optionally make that the behavior of the script by using this statement, and all following commands will cause the script to exit if they fail:
set -e

Expand selected variables within here-document while invoking Bash through SSH

I don't have remote access to a MySQL server, so I am trying to do it via an SSH session.
It partly works, but not correctly.
sshpass -p $password ssh user#$IP /bin/bash << EOF
mysql -N -uroot -ppassword test -e "select id from client where user ='$user'"
EOF
This will show the result of the select statement, but I'd like to be able to use that result in another echo statement.
eg:
The user ID is: xxxxx
I tried to assign the output to a variable using:
sshpass -p $password ssh user#$IP /bin/bash << EOF
res=$(mysql -N -uroot -ppassword test -e "select id from client where user ='$user'")
echo $res
EOF
But that results in:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/run/mysql/mysql.sock' (2)
If I quote EOF like 'EOF' then I can get the result into res but I lose the ability to use $user
Is there anyway to do this so I can use my variable in the heredoc and get the result of the MySQL query to a new variable ?
If I quote EOF like 'EOF' then I can get the result in to res but I lose the ability to use $user
You can pass $user to bash as a positional parameter and still have the quoted EOF and its advantages. E.g:
sshpass -p "$password" ssh "user#$IP" /bin/bash -s "$user" << 'EOF'
res=$(mysql -N -uroot -ppassword test -e "select id from client where user ='$1'")
echo $res
EOF
Bash manual describes the -s option as follows.
If the -s option is present, or if no arguments remain after option processing, then commands are read from the standard input. This option allows the positional parameters to be set when invoking an interactive shell or when reading input through a pipe.

How to verify if all the queries inside a test.sql file are executed successfully through shell script?

I have a q.sql file that has queries like
SET SQL_SAFE_UPDATES = 0;
UPDATE student SET gender = 'f' WHERE gender = 'm';
.
.
UPDATE student SET rollno = '03' WHERE rollno = '003';
This .sql file is executed through a shellscript:
mysql -uuser -ppass DB < q.sql
The command is executed even when one of the queries in q.sql file has failed. Now I want to verify if all the queries are updated successfully.
I tried to echo $? but it always prints 0, i.e command successful, even if the one of the queries in q.sql has failed.
mysql -uuser -ppass DB < q.sql
echo $?
If query fails I want it to print "failed" or stop the further execution of the shellscript.
If you use bash, you can use the set -e in your script and execute each line of your mysql script using -e option.
#!/bin/bash
set -e
while read line; do
mysql -uuser -ppass DB -e "$line"
done < q.sql
For information set --help shows:
-e Exit immediately if a command exits with a non-zero status.
and the man mysql page:
--execute=statement, -e statement
Execute the statement and quit. The default output format is like that produced with --batch. See Section 4.2.3.1, "Using Options on the Command Line", for some examples. With this option, mysql does not use the history file.
You can catch the output in a file for further processing:
mysql -uuser -ppass DB < q.sql > mysql.out
If you have a query that produces a lot of output, you can run the output through a pager rather than watching it scroll off the top of your screen:
mysql -uuser -ppass DB < q.sql | more
If you want to get the interactive output format in batch mode, use mysql -t. To echo to the output the statements that are executed, use mysql -v.
https://dev.mysql.com/doc/refman/8.0/en/batch-mode.html

Why is this bash variable blank when taking output from mysql?

I am trying to take the output from a MySQL query in bash and use it in a bash variable, but it keeps coming up blank when used in the script, but works perfectly from the terminal. What's wrong here?
I've tried changing the way the statement is written and changing the name of the variable just in case it was somehow reserved. I've also done a significant amount of searching but it turns out if you but 'bash', 'blank', and 'variable' in the search it usually comes up with some version of how to test for blank variables which I already know how to do.
tempo=$(mysql -u "$dbuser" -p"$dbpass" -D "$database" -t -s -r -N -B -e "select user from example where user='$temp' > 0;")
printf "the output should be: $tempo" # This is a test statement
The end result should be that the $tempo variable should either contain a user name from the database or be blank if there isn't one.
I think there is some error with your sql statement at user = '$temp' > 0.
But to get the result from MySql you have to redirect the standard error (stderr) to the standard output (stdout), you should use 2>&1.
Most probably you will run into MySql error but try running this on terminal.
tempo=$((mysql -u "$dbuser" -p"$dbpass" -D "$database" -t -s -r -N -B -e "select user from example where user='$temp' > 0;") 2>&1)
The solution was to echo the result of the sql query like this:
tempo=$(echo $(mysql -u "$dbuser" -p"$dbpass" -D "$database" -s -N -B -e "select user from example where user='$username' > 0;"))
Now I'm left with logic issues but I think I can handle that.

connect to mysql db and execute query and export result to variable - bash script

I want to connect to mysql databse and execute some queries and export its result to a varibale, and do all of these need to be done entirely by bash script
I have a snippet code but does not work.
#!/bin/bash
BASEDIR=$(dirname $0)
cd $BASEDIR
mysqlUser=n_userdb
mysqlPass=d2FVR0NA3
mysqlDb=n_datadb
result=$(mysql -u $mysqlUser -p$mysqlPass -D $mysqlDb -e "select * from confs limit 1")
echo "${result}" >> a.txt
whats the problem ?
The issue was resolved in the chat by using the correct password.
If you further want to get only the data, use mysql with -NB (or --skip-column-names and --batch).
Also, the script needs to quote the variable expansions, or there will be issues with usernames/passwords containing characters that are special to the shell. Additionally, uppercase variable names are usually reserved for system variables.
#!/bin/sh
basedir=$(dirname "$0")
mysqlUser='n_userdb'
mysqlPass='d2FVR0NA3'
mysqlDb='n_datadb'
cd "$basedir" &&
mysql -NB -u "$mysqlUser" -p"$mysqlPass" -D "$mysqlDb" \
-e 'select * from confs limit 1' >a.txt 2>a-err.txt
Ideally though, you'd use a my.cnf file to configure the username and password.
See e.g.
MySQL Utilities - ~/.my.cnf option file
mysql .my.cnf not reading credentials properly?
Do this:
result=$(mysql -u $mysqlUser -p$mysqlPass -D $mysqlDb -e "select * from confs limit 1" | grep '^\|' | tail -1)
The $() statement of Bash has trouble handling variables which contain multiple lines so the above hack greps only the interesting part: the data