Bash script with multiple queries and redirection in the same mysql session - mysql

My bash script queries a mysql database 3 times and redirects the standard out of each query to a file (3 different files in total with different columns structure ).
I want it to ask for the mysql password as it's important for me not to have the password in the script or on disk.
How can I include all queries and stdout redirection in the same mysql session in order to avoid asking for the password 3 times?
This is what I have now:
#!/bin/bash
mysql -h database.com -u user -p -e "USE database; mysql query1" > file1
mysql -h database.com -u user -p -e "USE database; mysql query2" > file2
mysql -h database.com -u user -p -e "USE database; mysql query3" > file3

You could use tee and notee commands and write a single query file, say queries.sql and invoke it in a single shot:
use database
tee file1
query1
notee
tee file2
query2
notee
tee file3
query3
notee
Then invoke it:
mysql -h database.com -u user -p -e "source queries.sql" > /dev/null
Related:
What is the equivalent of the spool command in mysql
How can I run an SQL script in MySQL?

You could use bash to prompt for the password, and then supply it to each of the mysql commands:
#!/bin/bash
echo "enter the password for MySQL:"
read -s PASSWD
mysql -h database.com -u user -p$PASSWD -e "USE database; mysql query1" > file1
mysql -h database.com -u user -p$PASSWD -e "USE database; mysql query2" > file2
mysql -h database.com -u user -p$PASSWD -e "USE database; mysql query3" > file3
Here is a POSIX-compliant version of silent prompting for non-bash shells:
stty -echo
printf "enter the password for MySQL:"
read PASSWD
stty echo
printf "\n"

Related

Redirecting stdout to mysql in bash

I can execute mysql passing in a file as follows.
mysql -u username -p < some_file
In a bash script I have a function which echoes output which I want to pass into the same command in a bash script.
some_function() {
echo "Some SQL"
}
How can I pass the output into mysql using pipes/redirection?
I have tried the following, but it fails with no such file or directory. How can I use the output from the function here instead.
mysql -u username -p < some_function
No need to use a pipe or a redirection in this case, you can use directly -e options to execute some SQL commands:
mysql -u username -p -e "SQL/MySQL commands"
Exemple on a specific database:
mysql -u username -p -e "use database_name; SHOW tables"
mysql -u username -p -e "SHOW tables" database_name
And you can also catch the output of a command or function and passing it as argument like this:
sql_command="$(your_function)"
mysql -u username -p -e "${sql_command}" database_name;
If you really want to use a pipe or a redirection (but I think it make no sense in this case):
$ mysql -u root -p database_name < <(echo "SHOW TABLES") # redirection
$ mysql -u root -p database_name <<< "$(echo "SHOW TABLES")" # another way to use redirection
$ echo "SHOW TABLES"|mysql -u root -p database_name # pipe

How to run the query from a file in mysql

How can I run a mysql query (just he query part) which is written to a file, eg. accounts.txt
/test/mysql -h testdb.com -P 8995 -p -u testaccount -e
"select distinct
amp.account_id
from
account_marketplace_groups amp;" -D company > /tmp/output.csv
How can I put the actual query in a file and execute the above and still get the output in a csv
Would this be the right way to do it ?
/test/mysql -h testdb.com -P 8995 -p -u testaccount -e /account.txt -D company > /tmp/output.csv
The -e option can't be used with a filename, it expects the parameter to be the query. Just use input redirection, since mysql reads from standard input.
/test/mysql -h testdb.com -P 8995 -p -u testaccount -D company < /account.txt > /tmp/output.csv
+1 to Barmar's answer but here is another solution, which works equally well:
/test/mysql -h testdb.com -P 8995 -p -u testaccount \
-e "source /account.txt" -D company > /tmp/output.csv
That is, if you don't want to use < for input redirection, you can use the -e "source ..." command to read the file from within the command-line.

Copy mysql database from remote server to local computer

I'm under VPN and I don't have SSH access to remote server.
I can connect to remote database by console
mysql -u username -p -h remote.site.com
Now I'm trying to clone the remote database to local computer
mysqldump -u username -p -h remote.site.com mysqldump | mysql -u root -ppassword webstuff
And I've got error
mysqldump: Got error: 1045: Access denied for user 'webstaff'#'10.75.1.2'
(using password: YES) when trying to connect
How to copy mysql database from remote server to local computer?
Assuming the following command works successfully:
mysql -u username -p -h remote.site.com
The syntax for mysqldump is identical, and outputs the database dump to stdout. Redirect the output to a local file on the computer:
mysqldump -u username -p -h remote.site.com DBNAME > backup.sql
Replace DBNAME with the name of the database you'd like to download to your computer.
Check syntax and execute one command at a time, then verify output.
mysqldump -u remoteusername -p remotepassword -h your.site.com databasename > dump.sql
mysql -u localusername -p localpassword databasename < dump.sql
Once you've matched all passwords, you can use pipe.
Often our databases are really big and the take time to take dump directly from remote machine to other machine as our friends other have suggested above.
In such cases what you can do is to take the dump on remote machine using MYSQLDUMP Command
MYSQLDUMP -uuser -p --all-databases > file_name.sql
and than transfer that file from remote server to your machine using Linux SCP Command
scp user#remote_ip:~/mysql_dump_file_name.sql ./
This can have different reasons like:
You are using an incorrect password
The MySQL server got an error when trying to resolve the IP address of the client host to a name
No privileges are granted to the user
You can try one of the following steps:
To reset the password for the remote user by:
SET PASSWORD FOR some_user#ip_addr_of_remote_client=PASSWORD('some_password');
To grant access to the user by:
GRANT SELECT, INSERT, UPDATE, DELETE, LOCK TABLES ON YourDB.* TO user#Host IDENTIFIED by 'password';
Hope this helps you, if not then you will have to go through the documentation
Please check this gist.
https://gist.github.com/ecdundar/789660d830d6d40b6c90
#!/bin/bash
# copymysql.sh
# GENERATED WITH USING ARTUR BODERA S SCRIPT
# Source script at: https://gist.github.com/2215200
MYSQLDUMP="/usr/bin/mysqldump"
MYSQL="/usr/bin/mysql"
REMOTESERVERIP=""
REMOTESERVERUSER=""
REMOTESERVERPASSWORD=""
REMOTECONNECTIONSTR="-h ${REMOTESERVERIP} -u ${REMOTESERVERUSER} --password=${REMOTESERVERPASSWORD} "
LOCALSERVERIP=""
LOCALSERVERUSER=""
LOCALSERVERPASSWORD=""
LOCALCONNECTION="-h ${LOCALSERVERIP} -u ${LOCALSERVERUSER} --password=${LOCALSERVERPASSWORD} "
IGNOREVIEWS=""
MYVIEWS=""
IGNOREDATABASES="select schema_name from information_schema.SCHEMATA where schema_name != 'information_schema' and schema_name != 'mysql' and schema_name != 'performance_schema' ;"
# GET A LIST OF DATABASES
databases=`$MYSQL $REMOTECONNECTIONSTR -e "${IGNOREDATABASES}" | tr -d "| " | grep -v schema_name`
# COPY ALL TABLES
for db in $databases; do
# GET LIST OF ITEMS
views=`$MYSQL $REMOTECONNECTIONSTR --batch -N -e "select table_name from information_schema.tables where table_type='VIEW' and table_schema='$db';"
IGNOREVIEWS=""
for view in $views; do
IGNOREVIEWS=${IGNOREVIEWS}" --ignore-table=$db.$view "
done
echo "TABLES "$db
$MYSQL $LOCALCONNECTION --batch -N -e "create database $db; "
$MYSQLDUMP $REMOTECONNECTIONSTR $IGNOREVIEWS --compress --quick --extended-insert --skip-add-locks --skip-comments --skip-disable-keys --default-character-set=latin1 --skip-triggers --single-transaction $db | mysql $LOCALCONNECTION $db
done
# COPY ALL PROCEDURES
for db in $databases; do
echo "PROCEDURES "$db
#PROCEDURES
$MYSQLDUMP $REMOTECONNECTIONSTR --compress --quick --routines --no-create-info --no-data --no-create-db --skip-opt --skip-triggers $db | \
sed -r 's/DEFINER=`[^`]+`#`[^`]+`/DEFINER=CURRENT_USER/g' | mysql $LOCALCONNECTION $db
done
# COPY ALL TRIGGERS
for db in $databases; do
echo "TRIGGERS "$db
#TRIGGERS
$MYSQLDUMP $REMOTECONNECTIONSTR --compress --quick --no-create-info --no-data --no-create-db --skip-opt --triggers $db | \
sed -r 's/DEFINER=`[^`]+`#`[^`]+`/DEFINER=CURRENT_USER/g' | mysql $LOCALCONNECTION $db
done
# COPY ALL VIEWS
for db in $databases; do
# GET LIST OF ITEMS
views=`$MYSQL $REMOTECONNECTIONSTR --batch -N -e "select table_name from information_schema.tables where table_type='VIEW' and table_schema='$db';"`
MYVIEWS=""
for view in $views; do
MYVIEWS=${MYVIEWS}" "$view" "
done
echo "VIEWS "$db
if [ -n "$MYVIEWS" ]; then
#VIEWS
$MYSQLDUMP $REMOTECONNECTIONSTR --compress --quick -Q -f --no-data --skip-comments --skip-triggers --skip-opt --no-create-db --complete-insert --add-drop-table $db $MYVIEWS | \
sed -r 's/DEFINER=`[^`]+`#`[^`]+`/DEFINER=CURRENT_USER/g' | mysql $LOCALCONNECTION $db
fi
done
echo "OK!"
Copy mysql database from remote server to local computer
I ran into the same problem. And I could not get it done with the other answers. So here is how I finally did it (yes, a beginner tutorial):
Step 1: Create a new database in your local phpmyadmin.
Step 2: Dump the database on the remote server into a sql file (here I used Putty/SSH):
mysqldump --host="mysql5.domain.com" --user="db231231" --password="DBPASSWORD" databasename > dbdump.sql
Step 3: Download the dbdump.sql file via FTP client (should be located in the root folder)
Step 4: Move the sql file to the folder of your localhost installation, where mysql.exe is located. I am using uniform-server, this would be at C:\uniserver\core\mysql\bin\, with XAMPP it would be C:\xampp\mysql\bin
Step 5: Execute the mysql.exe as follows:
mysql.exe -u root -pYOURPASSWORD YOURLOCALDBNAME < dbdump.sql
Step 6: Wait... depending on the file size. You can check the progress in phpmyadmin, seeing newly created tables.
Step 7: Done. Go to your local phpmyadmin to check if the database has been filled with the entire data.
Hope that helps. Good luck!
Note 1: When starting the uniformer-server you can specify a password for mysql. This is the one you have to use above for YOURPASSWORD.
Note 2: If the login does not work and you run into password problems, check your password if it contains special characters like !. If so, then you probably need to escape them \!.
Note 3: In case not all mysql data can be found in the local db after the import, it could be that there is a problem with the mysql directives of your dbdump.sql
Better yet use a oneliner:
Dump remoteDB to localDB:
mysqldump -uroot -pMypsw -h remoteHost remoteDB | mysql -u root -pMypsw localDB
Dump localDB to remoteDB:
mysqldump -uroot -pmyPsw localDB | mysql -uroot -pMypsw -h remoteHost remoteDB
C:\Users\>mysqldump -u root -p -h ip address --databases database_name -r sql_file.sql
Enter password: your_password
This answer is not remote server but local server. The logic should be the same. To copy and backup my local machine MAMP database to my local desktop machine folder, go to console then
mysqldump -h YourHostName -u YourUserNameHere -p YourDataBaseNameHere > DestinationPath/xxxwhatever.sql
In my case YourHostName was localhost. DestinationPath is the path to the download; you can drag and drop your desired destination folder and it will paste the path in.
Then password may be asked:
Enter password: xxxxxxxx

Can mysql take -e option and an input SQL script from command line at the same time?

$ mysql -h foobar.com -u user -p salesdb -e "SET #arg='date';" < "myscript.sql"
did not work. I wonder if using -e and < "myscript.sql" at the same time is allowed at all?
$ echo "SET #arg='date';" > input_script.sql
$ cat myscript.sql >> input_script.sql
$ mysql -h foobar.com -u user -p salesdb < input_script.sql
probably won't work at the same time, but you don't need that...
(echo "SET #arg='date';"; cat "myscript.sql") |
mysql -h foobar.com -u user -p salesdb
Found on http://www.cyberciti.biz/faq/using-mysql-in-shell-scripts/
#!/bin/bash
mysql dbnane<<EOFMYSQL
SELECT * from table;
EOFMYSQL

Mysqldump more than one table?

How can I use mysqldump to dump certain tables that start with common prefix?
Hehe, this is kind of a hack, but it works (using bash):
mysqldump -u USER -p DATABASE $(mysql -u USER -p -D DATABASE -Bse "show tables like 'PREFIX%'") > /tmp/DATABASE.out
Change the ALLCAPS words as needed.
You can create backup.sh script:
BACKUP_DIR="/path/to/backups"
DB_HOST="domain.com"
DB_USER="user"
DB_PASS="pass"
PREFIX="phpbb"
TMP_LIST = mysql --host=$DB_HOST --user=$DB_USER --password=$DB_PASS -e "show databases;" | grep $PREFIX
# just get to database name column (may be $1, don't remember)
TMP_LIST = cat $TMP_LIST | awk '{print $2}'
# Getting dbs
mkdir $BACKUP_DIR/tmp
for db in $TMP_LIST ; do
mysqldump --host=$DB_HOST --user=$DB_USER --password=$DB_PASS --opt $db > $BACKUP_DIR/tmp/$db.sql
zip -mj $BACKUP_DIR/tmp/$db.sql.zip $BACKUP_DIR/tmp/$db.sql 2>&1
done
Hope it help.