So I need to read in a bunch of ID numbers from a file and do a mysql query on each of them. I started off with this:
#!/bin/bash
file="eidlist.txt"
while IFS= read -r line
do
mysql --host <redacted> --user <redacted> --password=<redacted> -N -e "use netops;select m_mailname from footprints where m_empno=$line;"
done <"$file"
This works and produces the expected output. Now I need to do the same thing but with a different list of IDs, querying a different field. Since the field I'm now querying on contains alphanumeric values (whereas the previous one was entirely numberic), I need to surround the value in quotes in the mysql query string, which I escape with \" like so:
#!/bin/bash
file="pidlist0.txt"
while IFS= read -r line
do
mysql --host <redacted> --user <redacted> --password=<redacted> -N -e "use netops;select m_mailname from footprints where m_stuid=\"$line\";"
done <"$file"
This doesn't work - the script produces no output. What am I doing wrong?
When I test the mysql command at the command line (with the $line variable pre-populated to one of the values from the file), it works, but when run from inside the script it produces no output. What's going on?
Just use m_stuid='$line';" ? Or change the double to single.
Related
I am trying to take the output from a MySQL query in bash and use it in a bash variable, but it keeps coming up blank when used in the script, but works perfectly from the terminal. What's wrong here?
I've tried changing the way the statement is written and changing the name of the variable just in case it was somehow reserved. I've also done a significant amount of searching but it turns out if you but 'bash', 'blank', and 'variable' in the search it usually comes up with some version of how to test for blank variables which I already know how to do.
tempo=$(mysql -u "$dbuser" -p"$dbpass" -D "$database" -t -s -r -N -B -e "select user from example where user='$temp' > 0;")
printf "the output should be: $tempo" # This is a test statement
The end result should be that the $tempo variable should either contain a user name from the database or be blank if there isn't one.
I think there is some error with your sql statement at user = '$temp' > 0.
But to get the result from MySql you have to redirect the standard error (stderr) to the standard output (stdout), you should use 2>&1.
Most probably you will run into MySql error but try running this on terminal.
tempo=$((mysql -u "$dbuser" -p"$dbpass" -D "$database" -t -s -r -N -B -e "select user from example where user='$temp' > 0;") 2>&1)
The solution was to echo the result of the sql query like this:
tempo=$(echo $(mysql -u "$dbuser" -p"$dbpass" -D "$database" -s -N -B -e "select user from example where user='$username' > 0;"))
Now I'm left with logic issues but I think I can handle that.
I am writing a batch script that needs to get some log in tab separated format.
Problem is that some of the items in the log contains line breaks (\n) and this make lines that don't adhere to the number of columns I expect. is there any way I can replace or ignore or force mysql to write special chars as \x0a etc?
#!bash
mysql --defaults-extra-file=my.cnf --database db -e "SELECT * FROM db.Log ORDER BY ID DESC LIMIT 1,10;" -r -s -N 2>/dev/null
I am trying to execute an insert statement from linux shell where one of the columns has '$2a$10$zKjqmgld1gDYB/qkDuAS' in the value. When I see the inserted data the value is truncated and I get only 'aKjqmgld1gDYB/qkDuAS' as any digit followed by dollar is treated by linux as a parameter passed to the script.
This is how I am executing the script
mysql -u user --password=password -e "insert into users(id,name,password) values(1,'Some Name','\$2a\$10\$zKjqmgld1gDYB/qkDuAS')"
I have even tried escaping the $ like \$2a\$10\$zKjqmgld1gDYB/qkDuAS, but still it yields the same truncated data , however when I do echo '\$2a\$10\$zKjqmgld1gDYB/qkDuAS', I get the entire thing back.
Sameer
Single quotes don't nest in shell (do they anywhere else, anyway?). So, your string is effectively outside the quotes. Combine with double quotes and backslashes:
mysql -u user --password=password -e \
"insert into users(id,name,password) values(1,'Some Name','\$2a$10\$zKjqmgld1gDYB/qkDuAS')"
it was already in double quotes (sorry, for writing the wrong query). That didn't work either, however I found a workaround
echo "insert into users(id,name,password) values(1,'Some Name','\$2a\$10\$zKjqmgld1gDYB/qkDuAS')" >> temp.sql
mysql -u user --password=password < temp.sql
This finally worked.
I am new to bash /mysql however I have found tons of help reading threw examples and other people's problems ...but have ran into one of my own .
I am attempting to insert a row into an MySQL database table every time a file is added to a specific directory (for this i am using inotifywait ) anyways here is my bash script
#!/bin/bash
while true; do
filename= "false"
filename= inotifywait --format "%f" -e create /var/www/media2net/torrent
date_field= date +"%F":"%T"
mysql --host=localhost --user=root --password=admin Media2net << EOF
insert into video (title, description, url_video, upload_date)
values('testing','default_description','$filename', '$date_feild');
EOF
echo $filename
done
From this I have verified with echo the variable $filename is properly held at end of bash script however when i look at entry in the table the column url_video has it's default value and not the string represented by $filename
From what i can conclude the variable $filename does not get passed through EOF
i have tried as indicate here http://www.dba-oracle.com/t_access_mysql_with_bash_shell_script.htm
as well as this
Using shell script to insert data into remote MYSQL database
any help of where i can find how to pass variable into query would be greatly appreciated
In your example, filename is set to the empty string (mind the spaces after the = sign!). You need
filename=$(inotifywait --format "%f" -e create /var/www/media2net/torrent)
Similarly,
date_field=$(date +"%F:%T")
and be careful, you have a typo in your mysql command (date_field and note date_feild):
mysql --host=localhost --user=root --password=admin Media2net <<EOF
insert into video (title, description, url_video, upload_date)
values('testing','default_description','$filename', '$date_field');
EOF
Now I hope that you're controlling the filenames. Imagine a filename that contains a single quote e.g., hello'howdy. You'll have a problem in your query. Worse, an evil user who puts a file named whatever','something'); evil_mysql_command; whatever, you'll have the evil command performed! One possibility is to sanitize the filename using printf thus:
printf -v filename '%q' "$(inotifywait --format "%f" -e create /var/www/media2net/torrent)"
This will at least escape the single quotes that could appear in a filename. See Gordon Davisson's comment: the printf trick will not prevent from all the possible attacks, so I really hope you absolutely control the name of the files!
All these suggestions yield the following script:
#!/bin/bash
while true; do
printf -v filename '%q' "$(inotifywait --format "%f" -e create /var/www/media2net/torrent)"
date_field=$(date +"%F:%T")
mysql --host=localhost --user=root --password=admin Media2net <<EOF
insert into video (title, description, url_video, upload_date)
values('testing','default_description','$filename', '$date_field');
EOF
echo "$filename"
done
Edit.
To answer your question in the comment:
why did the script properly echo $filename to my terminal but not send it properly to MySQL, does that have to do with string starting with a space? or something else completely?
That's because when you do something like:
whatever= command
then the variable whatever is set to the empty string, and the command command is executed (with the whatever variable set as environment variable). E.g.,
$ IFS='c' read a b c <<< "AcBcC"
$ echo "$a $b $c"
A B C
$ echo $IFS
$
In your script, the variable filename was in fact never globally set. You can check it by doing this:
$ filename= "false"
$ echo "$filename"
$
What happens is that the environment variable filename is set to empty string then the command false (which happens to exist) is launched using that environment variable and we're done.
When you do this:
filename= inotifywait --format "%f" -e create /var/www/media2net/torrent
the variable filename is set to the empty string, and then the command inotifywait ... is executed with filename as an environment variable (but inotifywait doesn't really care about it). And that's what you saw on your terminal! it was the output of this command. Then you probably saw an empty line, that was the output of
echo $filename
which was equivalent to
echo
since the variable filename expanded to an empty string.
Hope this helps.
I've been trying for a few hours now to store row data into a BASH var then use that to launch some PHP scripts.
So far, I've only been able to echo out the whole result set. I have since broken that script, so I can't even paste it here. This is what I have, that's not working at all.
#! bin/bash
query=`echo "SELECT id FROM searches WHERE running=1 AND id_user=2" | mysql -u root`
I've never used BASH before, so I'm completely lost.
What I'm asking for is some resources that can show me how I can connect to mysql, then loop through a result set using the data from that row.
Thanks.
You can read mysql --silent output into Bash vars with while read:
>sql="SELECT id FROM searches WHERE running=1 AND id_user=2"
>mysql --silent -u ctrahey test -e "$sql" |while read myid; do
php -f my_processor.php $myid;
done
>
Notes:
sql="... sets a Bash variable of your sql (not actually necessary, just aides readability)
mysql -e the -e option allows you to pass in a query, so you don't need STDIN
mysql --silent --silent suppresses the extra formatting mysql usually does. in my testing, this was actually also unnecessary (the pipe chars in the output did not mess up Bash)
php -f .. this executes a php file, passing the current id as an arg.
Bash is not a good language for interacting with mysql, it only works for very very simple cases. Use php instead (since you mentioned php), it has an api for interacting with mysql databases sanely. And yes, you can run a php script as you would a bash script.
If you want to run a single command with MySQL, use mysql -e or mysql --execute. They are the same command, but the second version is more memorable. If you add this flag to the above, the rows will be stored in the variable.
However, as mentioned by geirha, BASH may not be the best language to get row data. But if you choose to parse it wish BASH, mysql -e is the right command to execute queries.