I have a database backup command that takes a mysql dump and then uploads that dump file to AWS S3, when I run the command as a normal user it works perfectly but when I use the same command in a cron job it fails.
I have checked the syslog and there is no error message saying there was a problem after the job. There is only a line saying the job is run and then goes on to run the next cron job.
The command is as follows, I have removed the sensitive parts:
mysqldump -u {{ db_user }} -p{{ db_password }} {{ db_name }} > /home/db_backup.sql | aws s3 cp /home/db_backup.sql s3://{{ s3_url }}/$(date --iso-8601=seconds)_db.sql --profile backupprofile
When this command is run by a normal user there is a warning output not to use the the mysql password in command line but this is essential for the command to work without interaction. There is also a second line ofor the S3 to say that the upload worked. Could these outputs be effecting the cronjob is someway?
You will need to have full paths on your cronjobs, I see you missed them out on mysqldump and also your aws for the connection URL. I would do whereis mysqldump and whereis aws to find which full path you need to run it.
Try checking the the environment variable, cron passes a minimal set of environment variables to your jobs. You can set the PATH easily inside crontab
PATH=/usr/local/bin:/usr/sbin
Also many cron execute command using sh, and you might be using another shell in your script . You can tell cron to run all commands in bash by setting the shell at the top of your crontab:
SHELL=/bin/bash
Cron tries to interpret % symbol, you need to escape it if you have that somewhr in your command.
If the output that comes first after running your command is interactive that is if this asks you to hit enter or something like that thn this is the issue otherwise thr shouldn't be any problem with this.
Related
I created a bat file where I asked it to load a putty session and passed another bat file using -m to execute all the commands necessary. In the second bat file, I have some repeated mysql commands to be executed. So I created a for loop, and wanted to pass the list of DB names as parameters to it, (which should be given along with the file I am asking to execute in the first bat file). I am not able to pass the list of DB names inside the file name using % symbol.
What is the solution for this.
SET file=%1
SET DB=%2
Putty.exe -load xyz -m %file% %DB% -t
This is my first bat file, I want to pass the DB name as list along with file. Can someone please help me achieve this?
I have some MySQL commands that are repeated for different DB's. Can I achieve that using for loop inside a bat file?
FOR %%q IN (%DB%) DO mysql -h xyzdb -e 'drop database %%q;'
This is giving a syntax error, can someone suggest where I went wrong?
The script file that you pass to putty is not a batch file. It's a list of commands for remote shell. So:
You have to use remote shell constructs and commands, not batch file commands.
You cannot refer to local environment variables.
But you can generate the file using the batch file
FOR %%q IN (%DB%) DO echo mysql -h xyzdb -e 'drop database %%q;' > script.txt
Putty.exe -load xyz -m script.txt
Also, for automation, you should use plink, not putty.
I'm not sure what you wanted to achieve by %DB% in the putty commandline.
I've been searching to understanding the following MySQL command:
framework_mariadb.sql | mysql -u username -p password -h 127.0.0.1 -P 26257 target
My guess was the sql statements within the sql file get executed by mysql for the given target/database. But then I came across the source command in MySQL, ie
\bin\mysql -u root -p testdatabase < C:\Users\Juan\Desktop\databasebackup.sql
So my question is, does the first command and the second command essentially do the same thing? My apologies if this has already been asked, I haven't been able to find details for the first SQL command.
This is more about Linux shell capabilities than it is about MySQL.
The second form runs the mysql client, and uses the < symbol to tell it to take its input from the specified file.
The first form does essentially the same thing, but uses the pipe character | to indicate that the output of the first command should be sent to the input of the second command.
However, for the first form I'd expect the line to start with cat (as in cat framework_mariadb.sql | mysql ...) because the SQL script won't normally run as a shell command. cat is a command that reads one or more files and send s them to its output.
It is possible to set the SQL script up to run like this, but that requires a specific line (#! /bin/cat or similar) to be present at the top of the file, and the file must have the execution bit set. (At least, that's how I'd do it. There might be some other bash magic I'm not aware of. bash is not my forté)
There are many resources on the web that can teach the fundamentals of the Linux shell. You could try Microsoft's Introduction to bash, but there are many others.
I've recently upgrade a server to Debian 9 and MySQL to the latest version. I have a simple backup script that I run before performing any work on a production site but this time, when running my script, I encounter the following:
mysqldump: unknown variable 'local-infile=0'
Here is my script. What's going on?
#!/bin/bash
# [skipping commentary]
SITE=prod
# Set the directory that the Drupal root is IN, no trailing slashes
DROOT=[website_root]
# Set the directory for storing backups, no trailing slashes
BUD=/$DROOT/notes/backups
# Don't edit; End of defining variables
echo Doing a full back up...
echo Prepare to enter MySQL password...
# tar -czf $BUD/$SITE-files-$(date +'%Y%m%d%H%M%S').tgz $DROOT/docroot
mysqldump -u mysql_user -p drupal > $BUD/$SITE-drupal-$(date +'%Y%m%d%H%M%S').sql
mysqldump -u mysql_user -p civicrm > $BUD/$SITE-civicrm-$(date +'%Y%m%d%H%M%S').sql
ls -lh $BUD
pwd
echo Finished with backups...
MySQL version 10.1.37-MariaDB-0+deb9u1 Debian 9.6
Edit: When I ssh and run mysqldump with correct permissions I get the same issue. Weirdest thing, cron that runs similar process is backing up my databases as ordered.
The best way to solve this is simply to rename the variable to:
loose-local-infile=1
This will allow mysqldump to merely throw a warning, rather than a fatal error.
The suggestion to comment out the variable is not an option if you want LOAD DATA INFILE functionality out of the box, and MySQL 8+ for security reasons requires you to set this variable for both server (mysqld) and client. It is the [client] variable grouping in your config that chokes mysqldump if you don't add the "loose-" prefix to local-infile.
Seems like the new version you install is compiled without support of local-infile parameter. And because package management system (usually) keep your current configuration file you can try to find this parameter in my.ini file and comment it.
This parameter manage LOAD DATA LOCAL functionality. But seems like this have some potential security issues (more here)
Im using a bash script (sync.sh), used by cron, that is supposed to sync a file to a MySQL database. It works by copying a file from automatically uploaded location, parse it by calling SQL script which calls other MySQL internally stored scripts, and at the end emails a report text file as an attachment.
But, seems like something is not working as nothing happens to MySQL databases. All other commands are executed (first line and last line: copy initial file and e-mail sending).
MySQL command when run separately works perfectly.
Server is Ubuntu 16.04.
Cron job is run as root user and script is part of crontab for root user.
Here is the script:
#!/bin/bash
cp -u /home/admin/web/mydomain.com/public_html/dailyxchng/warehouse.txt /var/lib/mysql-files
mysql_pwd=syncit4321
cd /home/admin/web/mydomain.com/sync
mysql -u sync -p$mysql_pwd --database=database_name -e "call sp_sync_report();" > results.txt
echo "<h2>Report date $(date '+%d/%m/%Y %H:%M:%S')</h2><br/><br/> <strong>results.txt</strong> is an attached file which contains sync report." | mutt -e "set content_type=text/html" -s "Report date $(date '+%d/%m/%Y %H:%M:%S')" -a results.txt -- recipient#mydomain.com
cron will execute the script using a very stripped environment. you probably want to add the full path to the mysql command to the cron script
you can find the full path by
which mysql
at the prompt,
or you can add an expanded path to the cron invocation
1 2 * * * PATH=/usr/local/bin:$PATH scriptname
I’m looking to create a deploy script that I can run from a terminal and it automatically deploys my site from a repository. The steps I’ve identified are:
Connect to remote server via SSH
Fetch latest version of site from remote repository
Run any SQL patches
Clean up and exit
I’ve placed the SSH connection and git pull commands in my shell file, but what I’m stuck with is MySQL with it being an (interactive?) shell itself. So in my file I have:
#!/bin/bash
# connect to remote server via SSH
ssh $SSH_USER#$SSH_HOST
# update code via Git
git pull origin $GIT_BRANCH
# connect to the database
mysql --user $MYSQL_USER --password=$MYSQL_PASSWORD --database=$MYSQL_DBNAME
# run any database patches
# disconnect from the database
# TODO
exit 0
As you can see, I’m connecting to the database, but not sure how to then execute any MySQL statements.
At the moment, I have a directory containing SQL patches in numerical order. So 1.sql, 2.sql, and so on. Then in my database, I have a table that simply records the last patch to be run. So I’d need to do a SELECT statement, read the last patch to be ran, and then run any neccesary patches.
How do I issue the SELECT statement to the mysql prompt in my shell script?
Then what would be the normal flow? Close the connection and re-open it, passing a patch file as the input? Or to run all required patches in one connection?
I assume I’ll be checking the last patch file, and doing a do loop for any patches in between?
Help here would be greatly appreciated.
Assuming you want to do all the business on the remote side:
ssh $SSH_USER#$SSH_HOST << END_SSH
git pull origin $GIT_BRANCH
mysql --user $MYSQL_USER --password=$MYSQL_PASSWORD --database=$MYSQL_DBNAME << END_SQL
<sql statements go here>
END_SQL
END_SSH
You could get the output from mysql using Perl or similar. This could be used to do your control flow.
Put your mysql commands into a file as you would enter them.
Then run as: mysql -u <user> -p -h <host> < file.sqlcommands.
You can also put queries on the mysql command line using '-e'. Put your 'select max(patch) from .' and read the output in your script.
cat *.sql | mysql --user $MYSQL_USER --password=$MYSQL_PASSWORD --database=$MYSQL_DBNAME