A better way to execute multiple MySQL commands using shell script - mysql

I would like to write a *.sh script to execute multiple MySQL commands.
Currently, what I can do is something like the following
mysql -h$host -u$user -p$password -e "drop database $dbname;"
mysql -h$host -u$user -p$password -e "create database $dbname;"
mysql -h$host -u$user -p$password -e "another MySQL command"
...
Is there a way to avoid typing mysql -h$host -u$user -p$password -e every time I want to execute a MySQL command?

I think you can execute MySQL statements from a text file, for example
here is the cmds.txt file which contains MySQL commands:
select colA from TableA;
select colB from TableB;
select colC from TableC;
To execute them using shell script, type
mysql -h$host -u$user -p$password db_dbname < cmds.txt
This way, you separate your MySQL commands from your shell script.
You may want your script to display progress information to you. For this you can invoke mysql with "--verbose" option.
For more information, see https://dev.mysql.com/doc/refman/5.6/en/mysql-batch-commands.html

You can use a single multiquery:
mysql -h$host -u$user -p$password -e "drop database $dbname;create database $dbname;another MySQL command;"
Simply write all your queries seperated by ;. They will be run one after the other.

Note that you can also use a HERE doc to have the queries within the same script:
mysql -h$host -u$user -p$password db_dbname <<'EOF'
select colA from TableA;
select colB from TableB;
select colC from TableC;
EOF
Note that I've used 'EOF' rather than EOF in the first line in order to prevent the contents of the script to disable parameter substitution (especially the ` can be problematic)
Also note that there should not be any whitespace before the final EOF (except if you use <<- rather than << -- in that case leading tab characters are stripped):
mysql -h$host -u$user -p$password db_dbname <<- 'EOF'
↠select colA from TableA;
↠select colB from TableB;
↠select colC from TableC;
↠EOF
(Replace the ↠ with a tab character).
For more info on the HERE doc syntax, see the bash documentation.

There are several ways, in linux you have:
From the mysql cli:
mysql> source mycmds.sql
Using Pipes:
echo "SELECT ..; INSERT ..;" | mysql ...
Executing commands from a file using pipes or redirection:
cat file.sql | mysql ... OR mysql .. < file.sql

Different from other answers that reduce repetition, but
there are ways to reduce options(user, host ... -u, -p, -h ...) from each line command
2 ways i know.
1. use my.cnf file
you can store your user information in option files (e.g ~/.my.cnf or etc)
[client]
user=your_username
password=your_password
# database=database_name
then you can just run mysql command with one option -e and query
mysql -e "drop database $dbname;"
mysql -e "create database $dbname;"
mysql -e "another MySQL command"
2. use mysql_config_editor
you can save login informations with mysql_config_editor
mysql_config_editor set --login-path=mypath1 --host=localhost --user=root --password
then run command just with login-path option
mysql --login-path=mypath1 -e "drop database $dbname;"
mysql --login-path=mypath1 -e "create database $dbname;"
mysql --login-path=mypath1 -e "another MySQL command"

Related

Receiving 'ERROR 1046 (3D000) at line 1: No database selected' when executing linux bash script

I'm trying to automate the creation of a mysql user with a database and table using Linux bash, but receiving ERROR 1046.
I've looked at various tutorials online, including this website, and from what I can see, my script looks correct!
If i log in to mysql via the command line and run these commands manually, it works!
But if i run the exact same commands in a bash script, it fails. (name of user/db replaced with $1 variable)
Posting this as a last resort as all the posts i have found referring to this error say to use 'use databasename;' but i already do that in the script.
#Code to create mysql user with db and table in linux bash.
#to run: ./createmysqldb Testuser1 TestPass1
$1 = Testuser1
$2 = TestPass1
...
mysql -u$user -p$password -e "create user '$1'#'%' identified by '$2';"
mysql -u$user -p$password -e "create database $1_DB;"
mysql -u$user -p$password -e "use $1_DB;"
mysql -u$user -p$password -e "CREATE TABLE persons (id int, name varchar(20), surname varchar(20));"
mysql -u$user -p$password -e "GRANT ALL PRIVILEGES ON $1_DB.* TO '$1'#'%';"
mysql -u$user -p$password -e "flush privileges;"
...
Manually tested this code successfully, so i know it works, not sure if the variables and the use $1_DB; is being executed correctly, I expect a new user to be created, with a database containing a persons table.

Exporting views from Mysql using Docker commmand

The code below extracts views separately from the database. However, I'm trying to get this to run in a single docker run or exec command.
Right now when I try, the pipe command and in combination with trying to escape quotes gives me errors.
mysql -u username INFORMATION_SCHEMA
--skip-column-names --batch
-e "select table_name from tables where table_type = 'VIEW'
and table_schema = 'database'"
| xargs mysqldump -u username database
> views.sql
Anyone know how to achieve this within one docker command?
For example:
docker exec -i $(docker-compose ps -q mysqldb) mysql ...
Much love.
You can run both the mysql client command and the mysqldump tool from somewhere that's not "on the database server". In your case, you can run them from the host that has the MySQL server, assuming you launched the database with options like docker run -p 3306:3306. It would look something like
mysql -h 127.0.0.1 -u username INFORMATION_SCHEMA \
--skip-column-names --batch \
-e "select table_name from tables where table_type = 'VIEW' and table_schema = 'database'" \
| xargs mysqldump -h 127.0.0.1 -u username database \
> views.sql
This avoids all of the shell quoting problems trying to feed this into docker exec, and also avoids the requirement to need root-level access on the host to do an administrative task (if you can run any Docker command at all then you can use docker run to add yourself to the host's /etc/sudoers, among other things).
I also agree with #MichaelBoesl's answer, though: this is long enough that trying to make it into a one-liner isn't really worth the trouble that the various quoting and escaping will bring. I'd probably write this into a script and put the SQL query into a file.
#!/bin/sh
: ${MYSQL_HOST:=127.0.0.1}
: ${MYSQL_USER:=username}
: ${MYSQL_DATABASE:=INFORMATION_SCHEMA}
cat >/tmp/dump_views.sql <<SQL
SELECT table_name
FROM tables
WHERE table_type='VIEW' AND table_schema='database';
SQL
mysql -h "$MYSQL_HOST" -u "$MYSQL_USER" --skip-column-names --batch \
"$MYSQL_DATABASE" </tmp/dump_views.sql \
| xargs mysqldump -h "$MYSQL_HOST" -u "$MYSQL_USER" "$MYSQL_DATABASE"
You can put all your commands into a bash script on the container and just execute the script!

connect to mysql db and execute query and export result to variable - bash script

I want to connect to mysql databse and execute some queries and export its result to a varibale, and do all of these need to be done entirely by bash script
I have a snippet code but does not work.
#!/bin/bash
BASEDIR=$(dirname $0)
cd $BASEDIR
mysqlUser=n_userdb
mysqlPass=d2FVR0NA3
mysqlDb=n_datadb
result=$(mysql -u $mysqlUser -p$mysqlPass -D $mysqlDb -e "select * from confs limit 1")
echo "${result}" >> a.txt
whats the problem ?
The issue was resolved in the chat by using the correct password.
If you further want to get only the data, use mysql with -NB (or --skip-column-names and --batch).
Also, the script needs to quote the variable expansions, or there will be issues with usernames/passwords containing characters that are special to the shell. Additionally, uppercase variable names are usually reserved for system variables.
#!/bin/sh
basedir=$(dirname "$0")
mysqlUser='n_userdb'
mysqlPass='d2FVR0NA3'
mysqlDb='n_datadb'
cd "$basedir" &&
mysql -NB -u "$mysqlUser" -p"$mysqlPass" -D "$mysqlDb" \
-e 'select * from confs limit 1' >a.txt 2>a-err.txt
Ideally though, you'd use a my.cnf file to configure the username and password.
See e.g.
MySQL Utilities - ~/.my.cnf option file
mysql .my.cnf not reading credentials properly?
Do this:
result=$(mysql -u $mysqlUser -p$mysqlPass -D $mysqlDb -e "select * from confs limit 1" | grep '^\|' | tail -1)
The $() statement of Bash has trouble handling variables which contain multiple lines so the above hack greps only the interesting part: the data

Search for specific string/pattern in MySQL Database from Terminal/SSH/Commandline

I have a Magento database in which I want to search for a particular string/pattern.
But the database's size is too large so I cannot export the database to .sql file and then search into that file(editor even Geany crashes opening such large files).
So how can I do a search the database for a perfect match of [string/pattern] and display fulltext information as result, through only using command-line and MySQL Database credentials ?
I tried below command, but it requires username to be given as -u[USERNAME], also it doesn't display full query or result in terminal window.
mysqldump -p[PASSWORD] [DATABASE] --extended=FALSE | grep [pattern] | less -S
Anyone have any solutions for this ?
You can first log into MySQL CLI as especified in http://dev.mysql.com/doc/refman/5.7/en/connecting.html
mysql --host=localhost --user=myname --password=mypass mydb
So, you can use a query command to find your pattern. If you know the table you want to search such as the column it make the thinks easy. The SELECT statement is like this:
SELECT column FROM table WHERE column LIKE '%pattern%';
http://dev.mysql.com/doc/en/select.html
If you don't know the table's name, you can list all and try to find by the meaning.
SHOW TABLES;
Edited with better code
You didn't say if this was a one off or not but this will check all tables in a schema for a value.
First in your home directory set up a file named .my.cnf with the following contents and change its permissions to 700 (Replace [USERNAME] and [PASSWORD] with your username and password.
[client]
user=[USERNAME]
password="[PASSWORD]"
Then execute the following (Replacing [DATABASE] and [CHECKSTRING] with your database and the check string)
mysql [DATABASE] --silent -N -e "show tables;"|while read table; do mysql [DATABASE] --silent -N -e "select * from ${table};"|while read line;do if [[ "${line}" == *"[CHECKSTRING]"* ]]; then echo "${table}***${line}";fi;done;done
If checking for 51584 the result would be something like
test_table***551584,'column 2 value','column 3 value'
test_table5***'column 1 value',251584,'column 3 value'
If you want to know which column had the value then select from INFORMATION_SCHEMA.COLUMNS and add another nest.
mysql [DATABASE] --silent -N -e "show tables;"|while read table; do mysql [DATABASE] --silent -N -e "select column_name from information_schema.columns where table_schema='[DATABASE]' and table_name = '${table}';"|while read column; do mysql [DATABASE] --silent -N -e "select ${column} from ${table};"|while read line;do if [[ "${line}" == *"[CHECKSTRING]"* ]]; then echo "${table}***${column}***${line}";fi;done;done;done
If checking for 51584 the result would be something like
test_table***column1***551584
test_table5***column2***251584
First of all you need to login into database with correct username and password by below command.
sudo mysql -u root -p
then check the database in which you want to operate operation.
eg.
SHOW DATABASES;
USE Test;
now your database is ready for operation through terminal. Here I assume my database name is "Test".
Now for String/pattern matching use command as below or follow the link http://www.mysqltutorial.org/mysql-regular-expression-regexp.aspx.
SELECT
column_list
FROM
table_name
WHERE
string_column REGEXP pattern;

Drop multiple databases with names matching a pattern

I want to drop all the databases starting with a word.
abc
xyz
cms_db1
cms_db2
cms_xyz
pqr
In the example given above, I will like to drop all the Databases starting with the word "cms".
I guess maatkit or shell script can do it. What is the best approach?
Here's a pure mySQL solution in two queries:
SELECT CONCAT('DROP DATABASE `', SCHEMA_NAME, '`;')
FROM `information_schema`.`SCHEMATA`
WHERE SCHEMA_NAME LIKE 'cms_%';
Then copy and paste the resulting recordset and run
I had to improve neurinos script because of special chars in my password, missing 'drop DATABASE ...' and not working comparision for DB_STARTS_WITH expression. The following script did work on Ubuntu Server:
#!/bin/bash
DB_STARTS_WITH="grp"
MUSER="root"
MPWD="YOUR_PASSWORD"
MYSQL="mysql"
DBS="$($MYSQL -u $MUSER -p"$MPWD" -Bse 'show databases')"
for db in $DBS; do
if [[ "$db" == $DB_STARTS_WITH* ]]; then
echo "Deleting $db"
$MYSQL -u $MUSER -p"$MPWD" -Bse "drop database $db"
fi
done
I would use something like:
echo "SHOW DATABASES LIKE 'cms_%'" \
| mysql \
| tail -n +2 \
| xargs -n1 mysqladmin -f drop
If you don't have your default username and password configured inside ~/my.cnf, you may need to supply the username and password via the -u and -p switches to the mysql/mysqladmin commands above.
(Edit - added -n arg to tail.)
Linux way:
#!/bin/bash
DB_STARTS_WITH="cms"
MUSER="root"
MPWD="yourpass"
MYSQL="mysql"
DBS="$($MYSQL -u$MUSER -p$MPWD -Bse 'show databases')"
for db in $DBS; do
if [[ "$db" =~ "^${DB_STARTS_WITH}" ]]; then
echo "Deleting $db"
$MYSQL -u$MUSER -p$MPWD -Bse "drop database $db"
fi
done
Of course use the drop part at your own risk ;)
If you wish to stay completely within MySQL/MariaDB (i.e. without using bash scripts and such) you can do the following:
DELIMITER //
CREATE PROCEDURE clean()
BEGIN
SET #query := (SELECT CONCAT('DROP DATABASE ', SCHEMA_NAME, ';') FROM `information_schema`.`SCHEMATA` WHERE SCHEMA_NAME LIKE 'dbtVDB%' LIMIT 1);
WHILE #query != '' DO
PREPARE stmt FROM #query;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #query := (SELECT CONCAT('DROP DATABASE ', SCHEMA_NAME, ';') FROM `information_schema`.`SCHEMATA` WHERE SCHEMA_NAME LIKE 'cms%' LIMIT 1);
END WHILE;
DELETE FROM mysql.db WHERE mysql.db.Db LIKE 'cms%';
END;
//
DELIMITER ;
CALL clean();
DROP PROCEDURE clean;
A Linux way:
for db_name in $(mysql -u USER -pPASS -e "show databases like 'cms_%'" -ss)
do
mysql -u USER -pPASS -e "drop database ${db_name}";
done
I liked the answer suggesting a "for" loop from the shell. In my case, I had subdirectory names matching my database names so I made arrays, then used them in the command.
(I could have done this using the mysql data directory come to think of it, even if I hadn't had the setup I had. On my bitnami VM this is
/opt/bitnami/mysql/data.)
created array from subset of files: tbtdirs=(tbt*2015*)
Tested a potentially spooky command first w/ "echo": for d in ${tbtdirs[#]}; do echo mysql -pPASS -e "drop database $d"; done
dropped all databases in the array: for d in ${tbtdirs[#]}; do mysql -pPASS -e "drop database $d"; done
Worked like a charm! Also modified the loop to remove subdirectories. I used Linux command line for quite some time before learning how useful the bash commands could be.
Using #léo-alves-de-araujo I have modified it to ask the user/password (More secure way) from command line (with linux)
#!/bin/bash
echo -n "Enter Mysql User:"
read user
echo -n "Enter Mysql Password:"
read -s password
for db_name in $(mysql -u $user --password=$password -e "SHOW DATABASES LIKE 'cms_%'" -ss 2>/dev/null)
do
mysql -u $user --password=$password -e "DROP DATABASE ${db_name}" 2>/dev/null;
done
Improved #neurino solution to avoid storing of MySQL credentials in the script and passing them through a command line (it might be visible in the list of processes then)
#!/bin/bash
DB_STARTS_WITH="cms"
MYSQL="mysql"
read -p "Enter MySQL user name: " MYSQL_USER
read -s -p "Enter password: " MYSQL_PASSWORD
CREDENTIALS_FILE="$(mktemp)"
chmod 600 $CREDENTIALS_FILE
cat > $CREDENTIALS_FILE <<- EOM
[client]
user=$MYSQL_USER
password=$MYSQL_PASSWORD
EOM
trap "{ rm -f $CREDENTIALS_FILE; }" EXIT
DATABASES="$(echo "show databases;" | $MYSQL --defaults-file=$CREDENTIALS_FILE)"
for DATABASE in $DATABASES; do
if [[ $DATABASE =~ ^${DB_STARTS_WITH} ]]; then
echo Removing $DATABASE...
echo "drop database $DATABASE" | $MYSQL --defaults-file=$CREDENTIALS_FILE
fi
done
Improvising on the excellent answer by #cloakedninjas, for easier retrieval of all the queries to execute in a single string.
Firstly, you can set the maximum value for group_concat_max_len to the maximum possible value, for this particular session:
SET SESSION group_concat_max_len = ##max_allowed_packet;
Now, you can prepare a query string (to execute later) using SQL. Using information_schema, we can get name of all the databases matching the pattern. Now, use Concat() to prepare a single DROP DATABASE .. query, and then utilize Group_Concat() to merge them all into a single string, for easier retrieval.
SELECT GROUP_CONCAT(CONCAT('DROP DATABASE `', SCHEMA_NAME, '`;')
SEPARATOR ' ') AS query_to_execute
FROM information_schema.SCHEMATA
WHERE SCHEMA_NAME LIKE 'cms_%'
Now copy the string in query_to_execute and run it separately.