I have a query that sends the results to an email. I would like not to send an email if the query has NO results. How can i do that ?
heres the code
mysql -umy -hmysql1.com -P2 -pmysq <<<" Select * from Data.data "| mail -aFrom:test#test.com -s 'test' test#gmail.com
Not every task can be done easily in a single command pipeline. Trying to force it into a one-liner can make it hard to code and hard to maintain.
Feel free to write some statements in a script:
result=`mysql -umy -hmysql1.com -P2 -pmysq -e " Select * from Data.data "`
if [ -n "$result" ]
then
echo "$result" | mail -aFrom:test#test.com -s 'test' test#gmail.com
fi
The -n test is for strings being nonzero length. Read http://linuxcommand.org/lc3_man_pages/testh.html for more details on that.
Re your comment:
The statements I showed above are things you could type at the command-line in bash. Bash supports variables and "if/then/else" constructs and a lot more.
Writing a bash script is easy. Anything you can type at the command-line can be in a file. Open a text editor and write the lines I showed above. Save the file. For example it could be called "mailmyquery.sh" (the .sh extension is only customary, it's not required).
Exit the text editor. Then run:
bash mailmyquery.sh
And it runs the statements in the file as if you had written them yourself at the command-line.
Voilà! You are now a shell script programmer!
Related
Hello I am having trouble with a very specific line in a bash script.
Here is the code:
ssh $SOURCEIP "/usr/bin/time -f \"%e\" bash -c \"seq $ITER | parallel -n0 \"mysql --silent -h $TARGET -uroot -ppass -e 'SELECT * FROM dbname.tablename WHERE size = $SIZE;' >> out.txt\""
The problem is I ran out of quotes. The opening and escaped double quotes at the beginning of "mysql" are closing those from "bash -c". I have to put the mysql statement in double quotes and the query in single quotes, otherwise i get an error and I can't figure out how to proceed. I know that I should not pass the password like that and it will be changed later, I get this warning "$ITER"-times everytime i test this because --silent doesn't suppress this.
The problematic code is part of a small shell script that is supposed to just perform this data transfer.
I want to change to the other machine with ssh first and not via parallel because of consistency with other scripts.
So basically I need the double quotes around the bash -c command to get this whole parallel operation to work, which are already escaped because of the opening ssh doublequotes and also I need to put the mysql command inside quotes as well but they are closing each other somehow.
Any help will be greatly appreciated.
Thanks in advance.
Largio
Edit: (SOLUTION)
As suggested by #ole-tange the following command worked for me.
parallel --shellquote | parallel --shellquote
After invoking in a shell, i pasted my string in question into the prompt and got the masked string back. I still had troubles with finding out what exactly to paste but in the end it is just logical.
What exactly i pasted into the quoter was:
sql mysql://root:pass#$TARGET/ 'SELECT data FROM db_name.tablename WHERE size = ${SIZE};' >> out.txt
But still i had some problems with my variables inside my query. The problem here was that i had to de-mask the masking of the 2 variables $TARGET and $SIZE after everything got masked by the parallel quoter. Maybe my thinking has a too laborious manner but i could not get it to work in another way. Also note that i did not put quotes around the whole sql statement, as my plan was before, because now the quoter compensated for that. For consistency reasons i paste the final string that i got working in the end (with my changes afterwards):
ssh $SOURCEIP "/usr/bin/time -f \"%e\" bash -c \"seq $ITER | parallel -n0 sql\\\ mysql://root:pass#$TARGET/ \\\'SELECT\\\ data\\\ FROM\\\ db_name.tablename\\\ WHERE\\\ size\\\ =\\\ ${SIZE}\\\;\\\'\\\ \\\>\\\>\\\ out.txt\""
GNU Parallel has a quoter:
$ parallel --shellquote
"*\`$
[CTRL-D]
\"\*\\\`\$
And you can do it twice:
$ parallel --shellquote | parallel --shellquote
"*\`$
[CTRL-D]
\\\"\\\*\\\\\\\`\\\$
So just paste the string you want quoted.
But you might want to consider using functions and use env_parallel to copy the function:
myfunc() {
size=$1
target=$2
sql mysql://root:pass#$target/ "SELECT data FROM db_name.tablename WHERE size = $size;" >> out.txt
}
env_parallel --env myfunc -S $SOURCEIP --nonall myfunc $SIZE $TARGET
Also: Instead of mysql try sql mysql://root:pass#/ 'SELECT * FROM dbname.tablename WHERE size = $SIZE;'
This should be an incredibly easy question but I am not very familiar with bash and I am taking way longer than I should to figure it out.
declare -a ids=( 1 2 3 )
for i in "${ids[#]}";
do
re= $(mysql -h .... "SELECT col_A FROM DBA WHERE id=$i")
if [ $re -eq 0 ]; then
echo sucess
fi
done
This is an example of what I am trying to do, I have an id array and I want to send a query to my db so I can get a flag in the row with a certain id and then do something based on that. But I keep getting unexpected token errors and I am not entirely sure why
Edit: While copying the code and deleting some private information somehow I deleted the then, it was present in the code I was testing.
Based on what you described and the partial script, I am not certain I can completely create what you are trying to do but the token error messages you are experiencing usually have to do with the way bash handles whitespace as a delimiter. A few comments based on what you posted:
You need to remove the space around the equal sign in declaring an variable, so the space after the equal sign in re= needs to removed.
Because bash will is sensitive to whitespace, you need to quote variables declarations that might contain a space. To be safe, quotes need to be around the sub-shell $( )
You were missing the then in the if statement
It is important that variables in the test brackets, that is single [ ]s, must be quoted. Using an unquoted string with -eq, or even just the unquoted string alone within test brackets normally works, however, this is an unsafe practice and can give unpredictable results.
So, taking into account the items noted, the updated script would look something like:
declare -a ids=( 1 2 3 )
for i in "${ids[#]}";
do
re="$(mysql -h .... "SELECT col_A FROM DBA WHERE id=$i")"
if [ "$re" -eq "0" ]; then
echo "success"
fi
done
Can you try working the edits mentioned into your script and see if you are able to get it working? Remember, it will be helpful for you to use a site like ShellCheck to learn more about potential pitfalls or the uniquenesses of bash syntax. This will help to ensure you are working toward a solution to your specific need rather then getting trapped by some tricky syntax.
After you have worked through those edits, can you report back your experience?
EDIT
Based on your comments there is a good chance you are not running your script with bash despite the including #!/bin/bash at the top of your script. When you run the script as sh scriptname.sh you are forcing the script to be run by sh not bash. Try running your script like this /bin/bash scriptname.sh then report back on your experience.
For more information on the differences between various shells, see Unix/Linux : Difference between sh , csh , ksh and bash Shell
Your problem with your if statement is that you do not have the then keyword. A simple fix is:
declare -a ids=( 1 2 3 )
for i in "${ids[#]}";
do
re= $(mysql -h .... "SELECT col_A FROM DBA WHERE id=$i")
if [ $re -eq 0 ]; then
echo sucess
fi
done
Also here is a great reference on if statements in bash
For a data mining project I need to convert 80 tab delimited files(100 MB each) to CSV files. Anybody is aware of some tools that can be handy in this case.
Download python: https://www.python.org/downloads/
Install it.
And run a script similar to the following.
Save the following as convert_tsv_to_csv.py Or anything ending in .py:
import csv
with open('C:\\path\to\file','r') as f:
tab_file = csv.reader(f, dialect=csv.excel_tab)
with open('C:\path\to\outfile.csv','w') as g:
comma_file = csv.writer(g, dialect=csv.excel)
for row in tab_file:
comma_file.writerow(row)
Change the paths and run it like: python convert_tsv_to_csv.py
The basic idea:
If the files are big, read them line by line.
Learn your basic tools.
On any UNIX/Linux/OSX system, the following commands each should do the trick:
sed -i -e 's/\t/,/g' *.csv
perl -i -p -e 's/\t/,/g' *.csv
These perform the basic tab to comma substitution. They won't take care of things like quoting and escaping if your data contains columns with a tabular or comma, or chaning the file name for you! Note that the syntax of sed and perl are very similar... -i is inplace editing, -e is execute a command, s/// is the syntax for regular expression substitutions. Etc.
Either way, your basic unix tools for this job are
extremely fast (the "stream editor" sed is well optimized, low-level C code)
handy (just some 10 keypresses!)
easy to use, once you've learned the basics (i.e. read the manual)
My script need to store in a structure the result of a query:
#!/bin/bash
user="..."
psw="..."
database="..."
query="select name, mail from t"
customStructure=$(mysql -u$user -p$psw $database -e "$query";)
I've no idea how store the array of {name, mail} from query result..
I need structure like this:
array=[ [name1,mail1] , [name2,mail2], ....., [nameN, mailN] ]
Is there a way to do this in bash?
Bash arrays are initialized like so:
myarray=("hi" 1 "2");
To capture the individual portions of output of a command into an array, we must loop through the output, adding it's results to the array. That can be done like so:
for i in `echo "1 2 3 4"`
do
myarray+=($i)
done
In your example, it looks like you wish to get the output of a MySQL command and store the parts of it's output lines into subarrays. I will show you how to capture lines into arrays, and given that, you should be able to figure out how to put subarrays into that yourself.
while read line
do
myarray+=("$line")
done < <(mysql -u${user} -p${psw} ${database} -e "${query}")
It's also worth mentioning that for this kind of MySQL operation, where you don't need output metadata (such as pretty formatting and table names), you can use MySQL's -B option to do 'batch output'.
Field level record can be accessed via read -a command and IFS is set to the empty string to prevent read from stripping leading and trailing whitespace from the line.
#!/bin/bash
user="..."
psw="..."
database="..."
query="select name, mail from t"
OIFS="$IFS" ; IFS=$'\n' ; oset="$-" ; set -f
while IFS="$OIFS" read -a line
do
echo ${line[0]}
echo ${line[1]}
done < <(mysql -u${user} -p${psw} ${database} -e "${query}")
Hello and thank you for any help you can provide
I have my Apache2 web server set up so that when I go to a specific link, it will run and display the output of a shell script stored on my server. I need to output the results of an SVN command (svn log). If I simply put the command 'svn log -q' (-q for quiet), I get the output of:
(of course not blurred), and with exactly 72 dashes in between each line. I need to be able to take these dashes, and turn them into an html line break, like so:
Basically I need the shell script to take the output of the 'svn log -q' command, search and replace every chunk of 72 dashes with an html line break, and then echo the output.
Is this at all possible?
I'm somewhat a noob at shell scripting, so please excuse any mess-ups.
Thank you so much for your help.
svn log -q | sed -e 's,-{72},<br/>,'
If you want to write it in the script this might help:
${string//substring/replacement}
Replace all matches of $substring with $replacement.
stringZ=abcABC123ABCabc
echo ${stringZ/abc/xyz} # xyzABC123ABCabc
# Replaces first match of 'abc' with 'xyz'.
echo ${stringZ//abc/xyz} # xyzABC123ABCxyz
# Replaces all matches of 'abc' with # 'xyz'.