I'm going to be running a shell script containing a CP command using a scheduled cron job. I would like to include in the script something to output to a log file whether the copy was successful or failed.
Appreciate any advice in advance.
Thanks
You can check the return code of cp. From the cp man page:
EXIT STATUS
The cp utility exits 0 on success, and >0 if an error occurs.
The exit code of the last operation is stored in the special variable $?, so you can do something like this:
cp .. ..
echo $? >> outputfile
Most likely, you'll want to have some sort of "custom" error message. For that purpose, you can check the value of $?
cp .. ..
if [ $? -ne 0 ]
then
echo "there was an error" >> outputfile
fi
I hope that gets you started.
Related
I write a .sh script that firstly downloads the source code of a page and secondly executes a Rscript only if the source code downloaded is different from the latter. The page is updated once a day and the URL ends with the actual date. This is all on a server and a cron job would run the .sh every 15 min. So I do this :
#!/bin/bash
lwp-download "https://geodes.santepubliquefrance.fr/GC_indic.php?lang=fr&prodhash=de1751e6&indic=type_hospit&dataset=covid_hosp_type&view=map2&filters=sexe=0,jour="$(date '+%Y-%M-%d') download.html
md5 page.html > last_md5
diff previous_md5 last_md5
if[ "$?" = "!" ] ; then
Rscript myscript.R
fi
mv last_md5 previous_md5
rm page.html
First problem, it carries on running the R script even though download.html is downloaded and unchanged.
Plus, I hit an error after the R script has run "Syntax error: "fi" unexpected"
Some issues:
You need to put a space between if and [ - or you could just do if command; then.
You calculate the MD5 sum on the wrong file.
You remove the wrong file.
Since you're probably not interested in seeing the actual diff in the MD5 sums, I suggest that you use cmp -s instead of diff.
Also note that I quoted the $(date ...) command too. It's not necessary in this particular case, but it makes linters happy.
#!/bin/bash
lwp-download "https://geodes.santepubliquefrance.fr/GC_indic.php?lang=fr&prodhash=de1751e6&indic=type_hospit&dataset=covid_hosp_type&view=map2&filters=sexe=0,jour=$(date '+%Y-%M-%d')" download.html
md5 download.html > last_md5
if ! cmp -s previous_md5 last_md5; then
Rscript myscript.R
mv last_md5 previous_md5
else
rm last_md5
fi
rm download.html
You should leave a space between if and [.
#!/bin/bash
lwp-download "https://geodes.santepubliquefrance.fr/GC_indic.php?lang=fr&prodhash=de1751e6&indic=type_hospit&dataset=covid_hosp_type&view=map2&filters=sexe=0,jour="$(date '+%Y-%M-%d') download.html
md5 page.html > last_md5
diff previous_md5 last_md5
if [[ "$?" = "!" ]] ; then
Rscript myscript.R
fi
mv last_md5 previous_md5
rm page.html
Also i'd recommend if you dont see any error to use any online lint to guide you in whats wrong
https://www.shellcheck.net/
Is there any way to avoid the first eval for code below? I tried to play with ${LASTPIPE[0]} and such things but it was way too complicated for me. For instance when I tried to inject code with semicolon ($MYSQLCOMMAND; if ...) it broke my output to the pipe. I spent more than 8 hours wandering StackOverflow just to give up and write one more SQL command without pipe. Don't want to publish bad written code.
MYSQLQUERY="select * from $TABLENAME"
MYSQLTOPTIONS="--defaults-extra-file=$EXTRA -h $HOSTNAME -D $DBNAME -N -e"
MYSQLCOMMAND='mysql $MYSQLTOPTIONS "$MYSQLQUERY"'
# Exit immediately if something wrong with MySQL command
if eval "$MYSQLCOMMAND > /dev/null" ; then : ; else \
printf "Script returned non-zero exit code: %s\n" "$?" ; exit $?; fi
# Count rows for valid JSON output
ROWS=0
eval $MYSQLCOMMAND | \
while read ; \
do
((ROWS++))
done
(rest of the code generates JSON with calling the same
eval ... while read... and verified by https://jsonlint.com/)
Also I'd like to hear any your comment on the code since I'm not an experienced bash coder.
I have the shell script:
#!/bin/bash
export LD=$(lsb_release -sd | sed 's/"//g')
export ARCH=$(uname -m)
export VER=$(lsb_release -sr)
# Load the test function
/bin/bash -c "lib/test.sh"
echo $VER
DISTROS=('Arch'
'CentOS'
'Debian'
'Fedora'
'Gentoo')
for I in "${DISTROS[#]}"
do
i=$(echo $I | tr '[:upper:]' '[:lower:]') # convert distro string to lowercase
if [[ $LD == "$I"* ]]; then
./$ARCH/${i}.sh
fi
done
As you can see it should run a shell script, depending on which architecture and OS it is run on. It should first run the script lib/test.sh before it runs this architecture and OS-specific script. This is lib/test.sh:
#!/bin/bash
function comex {
which $1 >/dev/null 2>&1
}
and when I run it on x86_64 Arch Linux with this x86_64/arch.sh script:
#!/bin/bash
if comex atom; then
printf "Atom is already installed!"
elif comex git; then
printf "Git is installed!"
fi
it returned the output:
rolling
./x86_64/arch.sh: line 3: comex: command not found
./x86_64/arch.sh: line 5: comex: command not found
so clearly the comex shell function is not correctly loaded by the time the x86_64/arch.sh script is run. Hence I am confused and wondering what I need to do in order to correctly define the comex function such that it is correctly loaded in this architecture- and OS-dependent final script.
I have already tried using . "lib/test.sh" instead of /bin/bash -c "lib/test.sh" and I received the exact same error. I have also tried adding . "lib/test.sh" to the loop, just before the ./$ARCH/${i}.sh line. This too failed, returning the same error.
Brief answer: you need to import your functions using . or source instead of bash -c:
# Load the test function
source "lib/test.sh"
Longer answer: when you call script with bash -c, a child process is created. This child process sees all exported variables (including functions) from parent process. But not vice versa. So, your script will never see comex function. Instead you need to include script code directly in current script and you do so by using . or source commands.
Part 2. After you "sourced" lib/test.sh, your main script is able to use comex function. But arch scripts won't see this function because it is not exported to them. Your need to export -f comex:
#!/bin/bash
function comex {
which $1 >/dev/null 2>&1
}
export -f comex
I'm building a shell script that will run nightly with the help of crontab and my script keeps bombing out and giving me the error "Syntax error: word unexpected (expected "do").
The script itself is creating a new directory for each MySQL file it finds in a directory and then copies the files to a created directory from the respective mysqldump directory. Please let me know if I am on the right track with this code and what could be causing my end of file error. I am new to shell scripting and would accept any advice given. If there is a better way to write my code then feel free to help in that respect as well.
CDPATH="/backups/mysql"
#
now=$(date +"%Y_%m_%d")
#
# Find Directory Name
#
for file in */; do
dir=${file%/}
if [[ -e "$dir"]]
then
echo "Directory Exists!"
else
echo "Directory doesn't exist."
fi
done
#
# Copy MySQL Databases
#
while [ -d $dir ]; do # Check existing Dirs
cp -upf /dbase/files/*.sql /backups/mysql/$dir/$now # If Dir exists, create copy
if [ ! -d $dir ]; then # If Dir nonexistant create
mkdir -p $dir
cp -upf /dbase/files/*.sql /backups/mysql/$dir/$now
else # If all else fails just create a copy in /mysql
cp -upf /dbase/files/*.sql /backups/mysql/$dir/$now
fi
done
Thanks for the help in advance!
1. do not use "then" after "else"
2. The second line may does not what you want. It just set the variable dir to the string "dir (/backups/mysql/*)"
... obsolete since question has changed.
I assume, files in the following form ...
/dbase/files/db1.sql
/dbase/files/db2.sql
...
... should be backed up to the following destination:
/backups/mysql/db1/2014_11_18/db1.sql
#!/bin/bash
BACKUP_DIR="/backups/mysql/"
FILE_DIR=/dbase/files/
now=$(date +"%Y_%m_%d")
# setting the input field seperator to newline
IFS=$'\n'
# find db backups and loop over
for file in $(find ${FILE_DIR} -maxdepth 1 -name "*.sql" -type f -exec basename {} \;); do
# create backup directory:
mkdir -p "${BACKUP_DIR}${file%.sql}/${now}"
# copy file over
cp "${FILE_DIR}${file}" "${BACKUP_DIR}${file%.sql}/${now}/"
done
Convert the script to unix encoding:
dos2unix script.sh
I have written a shell script which starts MySQL when its killed/terminated. I am running this shell script using a crontab.
My cron looks for the script file named mysql.sh under /root/mysql.sh
sh /root/mysql.sh
mysql.sh:
cd /root/validate-mysql-status
sh /root/validate-mysql-status/validate-mysql-status.sh
validate-mysql-status.sh:
# mysql root/admin username
MUSER="xxxx"
# mysql admin/root password
MPASS="xxxxxx"
# mysql server hostname
MHOST="localhost"
MSTART="/etc/init.d/mysql start"
# path mysqladmin
MADMIN="$(which mysqladmin)"
# see if MySQL server is alive or not
# 2&1 could be better but i would like to keep it simple
$MADMIN -h $MHOST -u $MUSER -p${MPASS} ping 2>/dev/null 1>/dev/null
if [ $? -ne 0 ]; then
# MySQL's status log file
MYSQL_STATUS_LOG=/root/validate-mysql-status/mysql-status.log
# If log file not exist, create a new file
if [ ! -f $MYSQL_STATUS_LOG ]; then
cat "Creating MySQL status log file.." > $MYSQL_STATUS_LOG
now="$(date)"
echo [$now] error : MySQL not running >> $MYSQL_STATUS_LOG
else
now="$(date)"
echo [$now] error : MySQL not running >> $MYSQL_STATUS_LOG
fi
# Restarting MySQL
/etc/init.d/mysql start
now1="$(date)"
echo [$now1] info : MySQL started >> $MYSQL_STATUS_LOG
cat $MYSQL_STATUS_LOG
fi
When I run the above mysql shell script manually using webmin's crontab, MySQL started successfully (when its killed).
However, when I schedule it using a cron job, MySQL doesn't starts. The logs are printed properly (it means my cron runs the scheduled script successfully, however MySQL is not restarting).
crontab -l displays:
* * * * * sh /root/mysql.sh
I found from URL's that we should give absolute path to restart MySQL through schedulers like cron. However, it haven't worked for me.
Can anyone please help me!
Thank You.
First, crontab normaly looks like this:
* * * * * /root/mysql.sh
So remove the surplus sh and put it at the beginning of the script - #!/bin/bash I suppose (why are you referring to sh instead of bash?) and don't forget to have an execute permission on the file (chmod +x /root/mysql.sh)
Second, running scripts within crontab is tricky, because the environment is different! You have to set it manually. We start with PATH: go to console and do echo $PATH, and then copy-paste the result into export PATH=<your path> to your cron script:
mysql.sh:
#!/bin/bash
export PATH=.:/bin:/usr/local/bin:/usr/bin:/opt/bin:/usr/games:./:/sbin:/usr/sbin:/usr/local/sbin
{
cd /root/validate-mysql-status
/root/validate-mysql-status/validate-mysql-status.sh
} >> OUT 2>> ERR
Note that I also redirected all the output to files so that you don't receive emails from cron.
Problem is how to know which other variables (besides PATH) matter. Try to go through set | less and try to figure out which variables might be important to set in the cron script too. If there are any MYSQL related variables, you must set them! You may also examine the cron script environment by putting set > cron.env to the cron script and then diff-ing it against console environment to look for significant differences.