I'm looking for a kind of remote database backup automation.
Then, I came across a scripting language which commonly used for administrative tasks, "Expect scripting" and I believe it could serve my purpose very well.
what I'd like to do is I want to perform login to a remote server using the following bash script from my local linux box. (supposed everything has been set properly, SSH authentication via generated key pair, so no password is required)
For the most important part, I'd like to send a mysqldump command to perform backup for my database on that server.
#!/usr/bin/expect
set login "root"
set addr "192.168.1.1"
spawn ssh $login#$addr
expect "#"
send "cd /tmp\r"
expect "#"
send "mysqldump -u root -ppassword my_database > my_database.sql\r"
expect "#"
send "exit\r"
The only problem I found here was after the line send "mysqldump -u root....... ".
It was never waiting until the process to finish, but immediately exit the shell with 'send "exit\r"' command line.
what do I do to make it waits until mysqldump command finish and log off the SSH properly?
I don't know the answer to your question: add exp_internal 1 to the top of the program to see what's going on.
However, since you have ssh keys set up, you don't really need expect at all:
ssh $login#$addr 'cd /tmp && mysqldump -u root -ppassword my_database > my_database.sql'
Related
You can avoid re-entering mysql command line password by putting the queries into a file.
In my case, the later queries are not determined until after the first queries have finished.
This happens in a non-interactive script so running a mysql console is not an option.
Is there any notion of a session for mysql command line interactions? Or can I set it up to listen for commands on a local unix socket (the output is required to be returned)? Or something like that?
User #smcjones mentions using the .my.cnf file or mysql_config_editor. Those are good suggestions, I give my +1 vote to him.
Another solution is to put the credentials in any file of your choosing and then specify that file when you invoke MySQL tools:
mysql --defaults-extra-file=my_special.cnf ...other arguments...
And finally, just for completeness, you can use environment variables for some options, like host and password. But strangely, not the user. See http://dev.mysql.com/doc/refman/5.7/en/environment-variables.html
export MYSQL_HOST="mydbserver"
export MYSQL_PWD="Xyzzy"
mysql ...other arguments...
I don't really recommend using an environment variable for the password, since anyone who can run ps on your client host can see the environment variables for the mysql client process.
There are a few ways to handle this in MySQL.
Put password in hidden .my.cnf in the home directory of the user the script is running as.
[client]
user=USER
password=PASSWORD
Use mysql_config_editor
mysql_config_editor set --login-path=client --host=localhost
--user=localuser --password
When prompted to enter your password, enter it like you otherwise would.
IMO this is the worst option, but I'll add it for the sake of completeness.
You could always create a function wrapper for MySQL that appends your set password.
#! /bin/bash
local_mysql_do_file() {
mysql -u localuser -h localhost -pPASSWORD_NO_SPACE < $1
}
# usage
local_mysql_do_file file.sql
I have a script that dumps data from a cloud foundry db, and it works in the following way:
cf ssh -L 33001:db.host:3306 --skip-remote-execution App &
TUNNEL_PID=$!
mysqldump --protocol TCP --port= 33001 ..... db_name > /tmp/my-db-dump.sql
kill $TUNNEL_PID
The problem is that mysqldump fails with
mysqldump: Got error: 2003: Can't connect to MySQL server on 'localhost' (61) when trying to connect
I expect that the problem is that the tunnel is not established yet. When I do sleep 5 before mysqldump, everything works. But I don't want to rely on random 5 seconds. Is it possible to wait for the tunnel to get started?
Can you run mysqldump via the ssh command, instead of opening a tunnel?
Mysqldump will write to its stdout, which will be transferred back to your client host via the ssh command.
ssh App "mysqldump db_name" > /tmp/my-db-dump.sql
Or you could even dump to a compressed file on the server, and then fetch the dump file with scp. That will help the transfer to go faster.
ssh App "mysqldump db_name | gzip -c > /tmp/my-db-dump.sql.gz"
scp App:/tmp/my-db-dump.sql.gz .
ssh App "rm /tmp/my-db-dump.sql.gz"
This is untested, but I hope it gives you some ideas to experiment with.
I'm studying MySQL, and every time I have to
Enter ssh XXX#XXX command, and enter my password to the school server.
Enter mysql -u XXX -p command, and enter MySQL password.
I want to create a Bash script for performing the steps above automatically.
I can accomplish the first step with this code:
#!/usr/bin/expect -f
set address xxx.com
set password xxx
set timeout 10
spawn ssh xxx#$address
expect { "*yes/no" { send "yes\r"; exp_continue} "*password:" { send "$password\r" } }
send clear\r
interact
But I don't know how to automatically input the next command (mysql -u xxx -p) and the password.
How can I do this?
You don't need such a complex script to just enter the MySQL console on remote machine. Use the features of the ssh tool:
ssh -tt user#host -- mysql -uuser -ppassword
The -t option forces pseudo-terminal allocation. Multiple -t force tty allocation, even if ssh has no local tty (see man ssh). Note the use of -p option. There must be no spaces between -p and password (see the manual for mysql).
Or even connect via mysql directly, if the MySQL host is accessible from your local machine:
mysql -hhost -uuser -p
Don't forget to adjust the shebang:
#!/bin/bash -
Use my.cnf to store your password securly like ssh keys.
https://easyengine.io/tutorials/mysql/mycnf-preference/
Same way ssh is also possible through ssh -i parameter and passing the private key path of the remote host.
Best of luck!
I'm trying to create a cron that daily backups my MySQL slave. The backup.sh content:
#!/bin/bash
#
# Backup mysql from slave
#
#
sudo mysql -u root -p'xxxxx' -e 'STOP SLAVE SQL_THREAD;'
sudo mysqldump -u root -p'xxxxx' ng_player | gzip > database_`date +\%Y-\%m-\%d`.sql.gz
sudo mysqladmin -u root -p'xxxxx' start-slave
I made it executable by sudo chmod +x /home/dev/backup.sh
and entered in to crontab by:
sudo crontab -e
0 12 * * * /home/dev/backup.sh
but it doesn't work, if I only run in the command line it works but not in crontab.
FIXED:
I used the script from this link: mysqldump doesn't work in crontab
Break the problem in half. First try sending only email from the cron job to see if you are getting it to even run. Put this above in a file and have your cron job point to it:
#!/bin/bash
/bin/mail -s "test subject" "yourname#yourdomain" < /dev/null
The good thing about using this tester is that it is very simple and more likely to give you some results. It does not depend on your current working directory, which can sometimes be not what you expect it to be.
Try use full link to mysql bin directory in .sh file
example :
sudo /var/lib/mysql -u root -p'xxxxx' -e 'STOP SLAVE SQL_THREAD;'
I had this same problem.
I figured out that you can't use the command sudo in a non-interactive script.
The sudo command would create a field where you would type in the password to your account (root).
If you are logged into a command prompt like ssh sudo works without typing in any passwords, but when another program runs sudo it would ask for password.
Try this instead su command doesn't require any logins and it does the same thing.
su --session-command="mysql -u root -p'xxxxx' -e 'STOP SLAVE SQL_THREAD;'" root
su --session-command="mysqldump -u root -p'xxxxx' ng_player | gzip > database_`date +\%Y-\%m-\%d`.sql.gz" root
su --session-command="mysqladmin -u root -p'xxxxx' start-slave" root
Replace root with your linux username.
EDIT:
Look at this thread for a different answer.
https://askubuntu.com/questions/173924/how-to-run-cron-job-using-sudo-command
Let's start with the silly stuff in the script.
The only command which you don't run via 'sudo' is the, spookily enough, only command which I would expect you might need to run via sudo (depending on the permissions of the target file).
Prefixing the commands in a script with sudo without a named user (i.e. running as root) serves no useful function if you are invoking the script as root.
On a typical installation, the mysql, mysqladmin and gzip programs are typically executable by any user - the authentication and authorization of the commands to the DBMS are authenticated by the DBMS using the authentication credentials passed as arguments - hence I would not expect that any of the operations here, except possibly writing to the output file (depending on its permissions).
You don't specify a path for the backup file - maybe it's writing it somewhere other than you expect?
(similarly, you should check if any of the executables are in a location which is not in the $PATH for the crontab execution environment).
but it doesn't work
....is not an error message.
The output of any command run via cron is mailed to the owner of the crontab - go read your mail.
I need to execute a mysql command on a remote server but seem to be hitting problem when it comes to executing the actual mysql bit
#!/usr/bin/expect -f
spawn /usr/bin/ssh -t root#10.0.0.2
expect "password: "
sleep 1
send "password\r"
sleep 2
/usr/bin/mysql databasename -e "update device_log set status = 'Y' where device_id in ('1','2');"
basically I want to change the flag to Y on device id's 1&2
but the script outputs
invalid command name "/usr/bin/mysql"
Just append the mysql command to the ssh command to run it in one go, like this:
#!/usr/bin/expect -f
spawn /usr/bin/ssh -t root#10.0.0.2 /usr/bin/mysql databasename -e "the query"
expect "password: "
sleep 1
send "password\r"
I'm not very much into expect, but I'm expecting that your attempt in the mysql line isn't actually valid syntax for expect to run a command.
Additionally:
You should use SSH keys for passwordless login instead of having a root password hardcoded in a script.
Consider running MySQL remotely e.g. mysql -h 10.0.0.2 -e "the query", or
Use port forwarding in SSH to connect to MySQL securely, e.g. run ssh -L 3307:localhost:3306 root#10.0.0.2 in the background and then connect to TCP port 3307 on localhost mysql -h 127.0.0.1 -P 3307.
It sounds like /usr/bin/mysql is not the the path to the mysql binary on that remote server. You could use just mysql instead, assuming that the binary is somewhere in that remote server's PATH. Otherwise you will have to go and find out where the binary is actually located and alter the absolute path accordingly.