Bash read line from variables - mysql

Hi & thanks in advance.
I'm trying to update a column(version) on an MySQL table from a Bash script.
I've populated a variable with the version numbers, but it fails after applying the first version in the list.
CODE:
UP_VER=`seq ${DB_VER} ${LT_VER} | sed '1d'`
UP_DB=`echo "UPDATE client SET current_db_vers='${UP_VER}' WHERE client_name='${CLIENT}'" | ${MYSQL_ID}`
while read -r line
do
${UP_DB}
if [[ "${OUT}" -eq "0" ]]; then
echo "Database upgraded.."
else
echo "Failed to upgrade.."
exit 1
fi
done < "${UP_VER}"
Thanks
Hopefully solved... My $UP_VER is in a a row not a column.

You're misunderstanding what several shell constructs do:
var=`command` # This executes the command immediately, and stores
# its result (NOT the command itself) in the variable
... < "${UP_VER}" # Treats the contents of $UP_VER as a filename, and tries
# to use that file as input
if [[ "${OUT}" -eq "0" ]]; then # $OUT is not defined anywhere
... current_db_vers='${UP_VER}' ... # this sets current_db_vers to the entire
# list of versions at once
Also, in the shell it's best to use lowercase (or mixed-case) variable names to avoid conflicts with the variables that have special meanings (which are all uppercase).
To fix the first problem, my recommendation is don't try to store shell commands in variables, it doesn't work right. (See BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!.) Either use a function, or just write the command directly where it's going to be executed. In this case I'd vote for just putting it directly where it's going to be executed. BTW, you're making the same mistake with ${MYSQL_ID}, so I'd recommend fixing that as well.
For the second problem, you can use <<< "${UP_VER}" to feed a variable's contents as input (although this is a bashism, and not available in generic posix shells). But in this case I'd just use a for loop:
for ((ver=db_ver+1; ver<=lt_ver; ver++)); do
For the third problem, the simplest way to test the success of a command is to put it directly in the if:
if somecommand; then
echo "Database upgraded.."
else # ... etc
So, here's my take at a rewrite:
mysql_id() {
# appropriate function definition goes here...
}
for ((ver=db_ver+1; ver<=lt_ver; ver++)); do
if echo "UPDATE client SET current_db_vers='${ver}' WHERE client_name='${client}'" | mysql_id; then
echo "Database upgraded.."
else
echo "Failed to upgrade.."
exit 1
fi
done
... but I'm not sure I understand what it's supposed to do. It seems to be updating current_db_vers one number at a time until it reaches $ver_lt... but why not set it directly to $ver_lt in a single UPDATE?

try something like :
done <<< "${UP_VER}"

Related

Bash case statement doesnt work with json string value using jq

I working on a function which extract the choosen track from a media container (mkv,mp4...etc). One of its major feature will be the "auto output file extension assigner".
the process will be the following...
step 1) when i give the script the number of the track, which i want to extract, it automatically inspect the source file with mediainfo and output the results in JSON format.
step 2) With JQ, i query the value of the "track" key from the selected track, and save it to the "mediaFormat" variable.
step 3) put this variable in a switch statement and compare with a predefined list of switches. If there is a match, then it will initialize the "mediaExtension" variable
with the appropriate value, which will be used as a extension of the ouput file.
For now i just want echo the "mediaExtension" variable, to see if it works. And it DIDN'T WORK.
The problem is step 1-2 works as expected, but somehow the switch statement (step 3) doesn't work. Only the (*) switch will be executed, which means it doesn't recognize the "AVC" switch.
#!/bin/bash
# INCLUDES
# mediainfo binary
PATH=/cygdrive/c/build_suite/local64/bin-video:$PATH;
# jq binary
PATH=/cygdrive/c/build_suite/local64/bin-global:$PATH;
# BASH SETTINGS
set -x;
# FUNCTION PARAMETER
function inspectExtension () {
mediaFormat=$(mediainfo "$1" --Output=JSON | jq ".media.track[$2].Format");
case $mediaFormat in
"AVC") mediaExtension="264";;
*) echo "ERROR";;
esac
set "$mediaExtension";
echo "$mediaExtension";
}
inspectExtension "test.mp4" "1";
read -p "Press enter to continue...";
And as you can see, in this script i activated tracing (set -x), and this is what i see in the console window (i use cygwin on windows 10).
+ inspectExtension test.mp4 1
++ mediainfo test.mp4 --Output=JSON
++ jq '.media.track[1].Format'
' mediaFormat='"AVC"
+ case $mediaFormat in
+ echo ERROR
ERROR
+ set ''
+ echo ''
+ read -p 'Press enter to continue...'
Press enter to continue...
Any ideas? Or is something what i miss here?
Thx for the help!
Maybe the only thing you miss is using the --raw-output option of jq like so:
mediaFormat=$(mediainfo "$1" --Output=JSON | jq --raw-output ".media.track[$2].Format");
Whenever you use jq to access some string values, it will be best to use the --raw-output option because it get's rid of the enclosing quotes.
Assuming you want mediaFormat to be a JSON value (i.e., assuming the invocation of jq is the way you have it), "AVC" in the case statement should be quoted:
'"AVC"' ) ...
In addition, it would probably be safer to quote the argument of case.

If in Loop in bash

This should be an incredibly easy question but I am not very familiar with bash and I am taking way longer than I should to figure it out.
declare -a ids=( 1 2 3 )
for i in "${ids[#]}";
do
re= $(mysql -h .... "SELECT col_A FROM DBA WHERE id=$i")
if [ $re -eq 0 ]; then
echo sucess
fi
done
This is an example of what I am trying to do, I have an id array and I want to send a query to my db so I can get a flag in the row with a certain id and then do something based on that. But I keep getting unexpected token errors and I am not entirely sure why
Edit: While copying the code and deleting some private information somehow I deleted the then, it was present in the code I was testing.
Based on what you described and the partial script, I am not certain I can completely create what you are trying to do but the token error messages you are experiencing usually have to do with the way bash handles whitespace as a delimiter. A few comments based on what you posted:
You need to remove the space around the equal sign in declaring an variable, so the space after the equal sign in re= needs to removed.
Because bash will is sensitive to whitespace, you need to quote variables declarations that might contain a space. To be safe, quotes need to be around the sub-shell $( )
You were missing the then in the if statement
It is important that variables in the test brackets, that is single [ ]s, must be quoted. Using an unquoted string with -eq, or even just the unquoted string alone within test brackets normally works, however, this is an unsafe practice and can give unpredictable results.
So, taking into account the items noted, the updated script would look something like:
declare -a ids=( 1 2 3 )
for i in "${ids[#]}";
do
re="$(mysql -h .... "SELECT col_A FROM DBA WHERE id=$i")"
if [ "$re" -eq "0" ]; then
echo "success"
fi
done
Can you try working the edits mentioned into your script and see if you are able to get it working? Remember, it will be helpful for you to use a site like ShellCheck to learn more about potential pitfalls or the uniquenesses of bash syntax. This will help to ensure you are working toward a solution to your specific need rather then getting trapped by some tricky syntax.
After you have worked through those edits, can you report back your experience?
EDIT
Based on your comments there is a good chance you are not running your script with bash despite the including #!/bin/bash at the top of your script. When you run the script as sh scriptname.sh you are forcing the script to be run by sh not bash. Try running your script like this /bin/bash scriptname.sh then report back on your experience.
For more information on the differences between various shells, see Unix/Linux : Difference between sh , csh , ksh and bash Shell
Your problem with your if statement is that you do not have the then keyword. A simple fix is:
declare -a ids=( 1 2 3 )
for i in "${ids[#]}";
do
re= $(mysql -h .... "SELECT col_A FROM DBA WHERE id=$i")
if [ $re -eq 0 ]; then
echo sucess
fi
done
Also here is a great reference on if statements in bash

bash: return code from redirected input command

I have a bash script I am using to loop through and process the results from a SQL query like such.
while read field1 field2 field3 field4
do
{...something here...}
done < <(mysql -h $HOST -u $USER -p"$PASS" $DB << EOF
{...multi-line select query here...}
EOF)
The problem I do not know how to solve is how do I refactor this so I can get the return code, error out and skip the loop if something goes wrong querying the database?
Edit:
I have tried to using a named-pipe with the following.
mkfifo /tmp/mypipe
mysql -h $HOST -u $USER -p"$PASS" $DB << EOF >> /tmp/mypipe
{...multi-line select query here...}
EOF
echo $?
{...loop here...}
This did not seem to work because the mysql command sits and waits for the pipe to be read before continuing. So unless I have something reading the pipe mysql does not exit to have a return code.
I have tried storing the query results into a variable first with the following.
DATADUMP=$(mysql -h $HOST -u $USER -p"$PASS" $DB -e \
'select stuff from place \
join table 1 on record ..... \
')
The issue I ran into with this is the read loop would only read the first four "words" from the DATADUMP variable and would ignore the rest.
At this point, unless someone comes back with a great idea, I'm going to mktemp a temp file to hold the query results. I was hoping to keep from constantly reading and writing to the disk, but my deadline is approaching very quickly.
There is no convenient way to do this, other than to capture the entire output of the mysql command before processing it. You could capture it into a variable, if you thought it wasn't going to be gigabytes of data, or into a temporary file. That's the only way to guarantee that no partial result is processed in the case of an error.
If you knew that all errors produced empty output, then it would be reasonably simple, since the while loop wouldn't execute at all. You wouldn't be able to distinguish between an error and an empty search, but you could work around that by producing an error indication:
got_data=0
while read field1 field2; do
got_data=1
# ...
done < <( mysql ... || printf "%s %d" "ERROR" $?) <<EOF
# ...
EOF
)
if ((!got_data)); then
if [[ $field1 == "ERROR" ]]; then
# error code is in $field2
else
# query returned no result
fi
However, the premise is likely to be false. In particular, a network failure in the middle of the query return will probably produce both some output and an error indication. And in that case, the above code would process the partial return.
If you don't actually mind processing a partial result, as long as the error is eventually detected, then you could use a simple variation on the above code. The code guarantees that an unsuccessful exit of mysql will eventually produce a line which starts with the ERROR token. (I don't know if mysql guarantees output of a trailing newline if the network transmission is interrupted. You'd need to play some games to guarantee that the error line output by printf always appeared on a line by itself.)
There is a little trick in the above: The error line (ERROR 3, for example) is output with printf without a trailing newline. The absence of the trailing newline will cause read to exit with a failure code even though it will successfully split the line into the variables field1 and field2. This makes it unnecessary to try to distinguish the error indication from normal data inside the while loop, although you still need a clear indication for the test after the while loop.

GNU make call function with multiple arguments and multiple commands

I am trying to write a GNU make call function (example below) which has multiple shell commands to execute, such that it can be called with different arguments.
shell_commands = $(shell echo $(1); ls -ltr $(2))
try:
$(call shell_commands,$(FILE1),$(FILE2))
1) Is above the correct way to write a call function with multiple commands? By using a semi-colon to separate them? To make it readable, I write my targets as shown below. Is there a similar way to write a call function?
shell_commands:
echo $(1)
ls -ltr $(2)
2) I get this error from make when I execute make -B try. It looks like it is trying to execute /home/user/file1. But why?
make: execvp: /home/user/file1: Permission denied
make: *** [try] Error 127
3) Is it possible to pass variable number of parameters to a call function? Like pass in just the second parameter and not the first one.
$(call shell_commands,,$(FILE2))
I tried google-ing, searching on SO, and looking on gnu.org but I did not get any solutions. Will appreciate any answers or pointers to any resources which document call function with multiple optional arguments and commands.
Question 1: No, this is not right. The shell make function should NEVER be used inside a recipe: the recipe is already running in the shell, so why would you run another shell? It's just confusing. Second, it's generally recommended to use && between multiple commands in a recipe, so that if the first command fails the entire command will immediately fail, rather than continuing on and perhaps succeeding. Of course, that is not always correct either, it depends on what you're trying to do.
Question 2: This happens because the shell make function is like backticks in the shell: it expands to the output printed by the shell command it runs. Your shell command that make runs is:
echo $(1); ls -ltr $(2)
(where, one assumes, $1 expands to /home/user/file1) which prints the string /home/user/file1. After the expansion, that string is substituted into the recipe and make tries to run that recipe, giving the error you see above.
You want this, most likely:
shell_commands = echo $(1) && ls -ltr $(2)
try:
$(call shell_commands,$(FILE1),$(FILE2))
Now the call expands to the actual text, not an invocation of make's shell function, and then that text is run as the recipe.
Question 3: Sure, just using empty parameters means that the variable $1 (in this case) expands to the empty string.

How to get variable from other while loop?

#!/bin/bash
read()
{
count=0
cat localfile |
while read line
do
FILE[$((count += 1))]="$line"
done
}
read
for((i=0;i<${#FILE[#]});i++)
do
echo ${FILE[i]}
done
The result of echo is whole blank. Is there any way to get the FILE array?
You posted this under ash, the Almquist shell, but you are using bash, the Bourne-Again SHell.
One problem is the pipe. When you run a pipe in bash, each side runs in its own sub-shell, and any variables are local to it. The correct mechanism is to redirect into the loop.
Another problem is that your function is called read, which is a shell-builtin (you use it!). A good idea is to use a naming convention, like an f_ prefix, so you don't get these name collisions.
Another issue you have is that the syntax of your second for loop is wrong in several ways. Here is a corrected version:
#!/bin/bash
f_read()
{
count=0
while read line
do
FILE[$((count += 1))]="$line"
done < localfile
}
f_read
for ((i=0;i<${#FILE[#]};i++))
do
echo ${FILE[i]}
done