How to embed BASH variable in MySQL query in shell script file - mysql

#!/bin/bash
fut=$(date -d "+14 days" +'%Y-%m-%d')
echo $fut
mysql -u user -ppassword base1 << "EOF"
INSERT INTO tableB SELECT * FROM tableA WHERE startdate < $fut;
....many lines of MySQL....
EOF
Mysql engine doesn't understand $fut as a variable. What syntax should be applied to make this work?

I have no problem with using a here-document, but I would like to suggest another possibility which is slightly more verbose, but has advantages when you may have to do more than just substituting variables : piping the output of a function.
#!/bin/bash
sql_statement()
{
local fut=$(date -d "+14 days" +'%Y-%m-%d')
echo "INSERT INTO tableB SELECT * FROM tableA WHERE startdate < $fut;"
echo "....many lines of MySQL...."
}
mysql -u user -ppassword base1 < <(sql_statement)
This is not required in this specific case, but allows the function to have additional logic (e.g. if blocks, loops...) to generate a more complex or variable SQL statement. As a general solution, it has nice advantages.

You need to remove the " from the EOF:
#!/bin/bash
fut=$(date -d "+14 days" +'%Y-%m-%d')
echo $fut
mysql -u user -ppassword base1 << EOF
INSERT INTO tableB SELECT * FROM tableA WHERE startdate < $fut;
....many lines of MySQL....
EOF
From bash docs:
No parameter substitution when the "limit string" is quoted or escaped.

Related

Can't access variables outside loop fetching mysql data in bash

I am currently trying to access variables outside of a loop which fetches 2 columns of a MySQL table in Bash:
mysql dbnameplaceholder -N -uusernameplaceholder -ppasswordplaceholder -
h127.0.0.1 -se "SELECT value1,value2 FROM table1 WHERE
value1='placeholdervalue'" | while read -r value1 value2;
do
var_value1="$value1"
var_value2="$value2"
done
echo "$var_value1 $var_value2" >> /tmp/script_debug
but the output is empty.
I did read up on the scope of variables here http://mywiki.wooledge.org/BashFAQ/024 and changed my original code to the following:
var_value1="NULL"
var_value1="NULL"
while read -r value1 value2;
do
var_value1="$value1"
var_value2="$value2"
done < <(mysql dbnameplaceholder -N -uusernameplaceholder -ppasswordplaceholder -
h127.0.0.1 -se "SELECT value1,value2 FROM table1 WHERE
value1='placeholdervalue'")
echo "$var_value1 $var_value2" >> /tmp/script_debug
but that did not work either.
By the way if i echo inside the loop i get the expected results so there is no MySQL query issue. Im no Bash expert so im kind of lost here.

BASH: Assigning MySQL SELECT result elements to different variables

Perhaps I am overcomplicating but ,I'm have built the following statement:
query="SELECT url, users FROM result as r INNER JOIN search as s ON s.UID = r.SearchID WHERE s.isProcessed = 0
The next step is to loop though the result but I need to be able to call urls and users separately. That's how I am attempting it:
while read -r link users; do
output+=("$link", "$users")
done < <(mysql -h $host -u $user -p$pwd -s -N -D $db -e "$query")
for i in ${!output[#]} ; do
echo "${output[i]}"
Any hint how to echo users only, please?
The above will return both urls and users :(
The users are in the odd-numbered elements of $output, so start at 1 and increment the array index by 2.
for ((i=1; i < ${#output[#]}; i+=2)); do
echo "${output[$i]}"
done

Store mysql result in a bash array variable

I am trying to store MySQL result into a global bash array variable but I don't know how to do it.
Should I save the MySQL command result in a file and read the file line by line in my for loop for my other treatment?
Example:
user password
Pierre aaa
Paul bbb
Command:
$results = $( mysql –uroot –ppwd –se « SELECT * from users );
I want that results contains the two rows.
Mapfile for containing whole table into one bash variable
You could try this:
mapfile result < <(mysql –uroot –ppwd –se "SELECT * from users;")
Than
echo ${result[0]%$'\t'*}
echo ${result[0]#*$'\t'}
or
for row in "${result[#]}";do
echo Name: ${row%$'\t'*} pass: ${row#*$'\t'}
done
Nota This will work fine while there is only 2 fields by row. More is possible but become tricky
Read for reading table row by row
while IFS=$'\t' read name pass ;do
echo name:$name pass:$pass
done < <(mysql -uroot –ppwd –se "SELECT * from users;")
Read and loop to hold whole table into many variables:
i=0
while IFS=$'\t' read name[i] pass[i++];do
:;done < <(mysql -uroot –ppwd –se "SELECT * from users;")
echo ${name[0]} ${pass[0]}
echo ${name[1]} ${pass[1]}
New (feb 2018) shell connector
There is a little tool (on github) or on my own site: (shellConnector.sh you could use:
Some preparation:
cd /tmp/
wget -q http://f-hauri.ch/vrac/shell_connector.sh
. shell_connector.sh
newSqlConnector /usr/bin/mysql '–uroot –ppwd'
Following is just for demo, skip until test for quick run
Thats all. Now, creating temporary table for demo:
echo $SQLIN
3
cat >&3 <<eof
CREATE TEMPORARY TABLE users (
id bigint(20) unsigned NOT NULL PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(30), date DATE)
eof
myMysql myarray ';'
declare -p myarray
bash: declare: myarray: not found
The command myMysql myarray ';' will send ; then execute inline command,
but as mysql wont anwer anything, variable $myarray wont exist.
cat >&3 <<eof
INSERT INTO users VALUES (1,'alice','2015-06-09 22:15:01'),
(2,'bob','2016-08-10 04:13:21'),(3,'charlie','2017-10-21 16:12:11')
eof
myMysql myarray ';'
declare -p myarray
bash: declare: myarray: not found
Operational Test:
Ok, then now:
myMysql myarray "SELECT * from users;"
printf "%s\n" "${myarray[#]}"
1 alice 2015-06-09
2 bob 2016-08-10
3 charlie 2017-10-21
declare -p myarray
declare -a myarray=([0]=$'1\talice\t2015-06-09' [1]=$'2\tbob\t2016-08-10' [2]=$'3\tcharlie\t2017-10-21')
This tool are in early step of built... You have to manually clear your variable before re-using them:
unset myarray
myMysql myarray "SELECT name from users where id=2;"
echo $myarray
bob
declare -p myarray
declare -a myarray=([0]="bob")
If you're looking to get a global variable inside your script you can simply assign a value to a varname:
VARNAME=('var' 'name') # no space between the variable name and value
Doing this you'll be able to access VARNAME's value anywhere in your script after you initialize it.
If you want your variable to be shared between multiple scripts you have to use export:
script1.sh:
export VARNAME=('var' 'name')
echo ${VARNAME[0]} # will echo 'var'
script2.sh
echo ${VARNAME[1]} # will echo 'name', provided that
# script1.sh was executed prior to this one
NOTE that export will work only when running scripts in the same shell instance. If you want it to work cross-instance you would have to put the export variable code somewhere in .bashrc or .bash_profile
The answer from #F. Hauri seems really complicated.
https://stackoverflow.com/a/38052768/470749 helped me realize that I needed to use parentheses () wrapped around the query result to treat is as an array.
#You can ignore this function since you'll do something different.
function showTbl {
echo $1;
}
MOST_TABLES=$(ssh -vvv -t -i ~/.ssh/myKey ${SERVER_USER_AND_IP} "cd /app/ && docker exec laradock_mysql_1 mysql -u ${DB} -p${REMOTE_PW} -e 'SELECT table_name FROM information_schema.tables WHERE table_schema = \"${DB}\" AND table_name NOT LIKE \"pma_%\" AND table_name NOT IN (\"mail_webhooks\");'")
#Do some string replacement to get rid of the query result header and warning. https://stackoverflow.com/questions/13210880/replace-one-substring-for-another-string-in-shell-script
warningToIgnore="mysql\: \[Warning\] Using a password on the command line interface can be insecure\."
MOST_TABLES=${MOST_TABLES/$warningToIgnore/""}
headerToIgnore="table_name"
MOST_TABLES=${MOST_TABLES/$headerToIgnore/""}
#HERE WAS THE LINE THAT I NEEDED TO ADD! Convert the string to array:
MOST_TABLES=($MOST_TABLES)
for i in ${MOST_TABLES[#]}; do
if [[ $i = *[![:space:]]* ]]
then
#Remove whitespace from value https://stackoverflow.com/a/3232433/470749
i="$(echo -e "${i}" | tr -d '[:space:]')"
TBL_ARR+=("$i")
fi
done
for t in ${TBL_ARR[#]}; do
showTbl $t
done
This successfully shows me that ${TBL_ARR[#]} has all the values from the query result.
results=($( mysql –uroot –ppwd –se "SELECT * from users" ))
if [ "$?" -ne 0 ]
then
echo fail
exit
fi

How to write hive script for return result in shell variable in an Oozie coordinator?

Then i coordinate my script.sh in an Oozie there is nothing in a variables S.
Here is shell script
S=$(hive -S -hiveconf MY_VAR1=$DB -hiveconf MY_VAR2=$avgpay -hiveconf MY_VAR3=$Date_LastDay -hiveconf MY_VAR4=$Date_LastNmonth -f bpxp.hql)
echo $S "S"
S1=( $( for k in $S ; do echo $k ; done ) )
cntn=${#S1[#]}
for (( p=0 ; p<$cntn; p=p+5 ))
do
`mysql -h$mysqlhost -u$mysqluser -p$mysqlpass $mysqldb -e "INSERT INTO weekstat (timeshift, partnerid, avg_value, processdate, weekday) VALUES ('${S1[p]}', '${S1[p+1]}', '${S1[p+2]}', '${S1[p+3]}', '${S1[p+4]}');"`
done
Mysql commands works fine with another variables
Here is bpxp.hql
...
hive -e "select * from ${hiveconf:MY_VAR1}.weekstat;"
Then i run script from shell it works fine. I try to use bpxp.hql without this line hive -e "select * from ${hiveconf:MY_VAR1}.weekstat;" and write it in shell S=hive -e "select * from $DB.weekstat;" but nothing change.
Where is my mistake?
Whatever you done is an efficient way, but minor change as mentioned below:
S=$(hive -S -hiveconf MY_VAR1=$DB -hiveconf MY_VAR2=$avgpay -hiveconf MY_VAR3=$Date_LastDay -hiveconf MY_VAR4=$Date_LastNmonth -f bpxp.hql)
In bpxp.hql keep the query
select * from $DB.weekstat;
You don't need to mention hive -e in the hql script, the command above will automatically executes and gives you the output, and stores into S.
Please let me correct if I'm wrong.

Bash script: Mysql query field to variable

I'm trying to obtain a recordset and read a few fields from it. I'm not able to figure it out how put the fields into variables. The script is:
#!/bin/bash
sqlQuery="$(mysql -h host -u user -ppass -D oberonsaas_v2 -s -N -e
'select ventas.id_venta,
ventas_entradas.id_ventas_entradas,
ventas.id_evento,
id_tarifa,
DATE_FORMAT(fecha_evento,"%Y%m%d") as fecha,
TIME_FORMAT(pase,"%H%i") as pase
from pases,ventas,ventas_entradas,recintos
where ventas.id_recinto = recintos.id_recinto
and ventas.id_pase = pases.id_pase
and ventas.id_venta = ventas_entradas.id_venta
and recintos.id_cliente = 32
and ventas.estado="Pagada"
and date(fecha_venta) = date_add(date(CURRENT_TIMESTAMP),INTERVAL -1 day)')"
echo $sqlQuery
I get all the recordset in $sqlQuery, but i want to do a loop and concat the fields.
Well I have invented some output
#!/bin/bash
Result="21,336,purchase,tarif_exspensive,Jose"
(IFS=","
for i in $Result
do
echo "I Have: " $i
done
)
Ouput is
$ bash t5.sh
I Have: 21
I Have: 336
I Have: purchase
I Have: tarif_exspensive
I Have: Jose
Collapse this into the simplest possible construct.
Put returned recordset into a bash array (for example):
sqlQuery=( pases_value ventas_value ventas_entradas_value recintos_value )
note: yours could look like:
sqlQuery=( $(mysql -h host ... )
new lines between the array operators "(" and ")" are ok
each will be in a distinct sqlQuery array location.
for example return: recintos_value
echo ${sqlQuery[3]}
concat all:
echo ${sqlQuery[*]}
if delimiters are of concern, use the mysql --delimiter switch to define a desired character, then
set concatValue=${sqlQuery[*]}
then to remove the delimiter character ("," for example):
set finalValue=${concatValue//,}