mysql command line to vertical output - mysql

I'd like to get the result of a mysql query with a vertical output.
My problem when using --vertical (or G) is the starred lines.
$mysql -N -e 'select filed1, field2 from db.tlb\G'
*************************** 1. row ***************************
value1_field1
value1_field2
*************************** 2. row ***************************
value2_field1
value2_field2
...
Is there an option I didn't find to get rid of the lines ***[...] x. row [...]*** ?
At the moment, I'm using a egrep -v '^\*.*\*$', but I'm sure a better solution exists.

I'm not sure this is a good solution, but if explicitly using egrep is annoying, you could define a shell function to launch mysql with the desired pager. Assuming you're using a bash (or compatible):
# define a shell function to launch mysql with the required _pager_
sh$ mysql() { `which mysql` $* --pager="egrep -v '^\*.*\*$'" ; }
[ ... ]
# launch mysql "as usual" (in reality, this is the shell function that is invoked)
sh$ mysql -u user -p -h mysql-host mydb
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 67
Server version: 5.1.49-3 (Debian)
-- Use mysql normally -- but the "egrep pager" will remove the "star lines"
mysql> select * from T\G
col1: w
col2: x
col3: 0.4
1 row in set (0.00 sec)
As I said before, this is not a perfect solution since egrep will blindly remove from output any "star line" -- not only the one from an ego (\G) command.

try this (but I can not imagine why you need such an output)
mysql -N -B -e 'select * from db' mysql | tr "\t" "\n"

First Option
would be to change the MySQL pager like so:
test.sh
#!/usr/bin/env bash
# Some Basic Configuration
db="homestead"
dbuser="homestead"
dbhost="127.0.0.1"
# Executes the MySQL command using the basic configuration
# This also sets the MySql Pager to less -Sin to enable
# it also removes all lines starting with "*"
mysql -h $dbhost -u $dbuser -p --pager="grep -Ev '(^\*.*$)|(^$)' | less -Sin" $db
Usage
Firstly Edit the Config variables:
$ nano ./test.sh
Secondly Run the script
$ bash ./test.sh
Second Option
Change the pager after you're already in the MySQL CLI
$ mysql -u root -p -h 127.0.0.1 somedatabase
enter password:
mysql> pager grep -Ev '(^\*.*$)|(^$)' | less -Sin

Related

connect to mysql db and execute query and export result to variable - bash script

I want to connect to mysql databse and execute some queries and export its result to a varibale, and do all of these need to be done entirely by bash script
I have a snippet code but does not work.
#!/bin/bash
BASEDIR=$(dirname $0)
cd $BASEDIR
mysqlUser=n_userdb
mysqlPass=d2FVR0NA3
mysqlDb=n_datadb
result=$(mysql -u $mysqlUser -p$mysqlPass -D $mysqlDb -e "select * from confs limit 1")
echo "${result}" >> a.txt
whats the problem ?
The issue was resolved in the chat by using the correct password.
If you further want to get only the data, use mysql with -NB (or --skip-column-names and --batch).
Also, the script needs to quote the variable expansions, or there will be issues with usernames/passwords containing characters that are special to the shell. Additionally, uppercase variable names are usually reserved for system variables.
#!/bin/sh
basedir=$(dirname "$0")
mysqlUser='n_userdb'
mysqlPass='d2FVR0NA3'
mysqlDb='n_datadb'
cd "$basedir" &&
mysql -NB -u "$mysqlUser" -p"$mysqlPass" -D "$mysqlDb" \
-e 'select * from confs limit 1' >a.txt 2>a-err.txt
Ideally though, you'd use a my.cnf file to configure the username and password.
See e.g.
MySQL Utilities - ~/.my.cnf option file
mysql .my.cnf not reading credentials properly?
Do this:
result=$(mysql -u $mysqlUser -p$mysqlPass -D $mysqlDb -e "select * from confs limit 1" | grep '^\|' | tail -1)
The $() statement of Bash has trouble handling variables which contain multiple lines so the above hack greps only the interesting part: the data

mysql hangs at login

I am using InfiniDB (not by choice). When I use the linux shell, I am able to connect to a database just fine:
<user>#<host>:~$ /usr/local/Calpont/mysql/bin/mysql -u <user> -p -h <host>
Enter password: <password>
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1725
Server version: 5.1.39 MySQL Community / Calpont InfiniDB Community 2.2.11-1 Fin al (COSS LA)
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use <database_name>
Database changed
mysql>
But when I use a bash script, it hangs:
<user>#<host>:~$ bash -x /home/aliceell/db_export.sh
+ DB=al05_09
+ export_dir=/tmp/
++ /usr/local/Calpont/mysql/bin/mysql -u <user> -p -h <host> <db_name>
Enter password:
It stops responding after I enter my password and press 'enter'. I have to use ctrl-C to get out of it.
Here is the code that I have:
DB=<db_name>
export_dir="/tmp/"
err_exit()
{
echo -e 1>&2
exit 1
}
mysql_data_dir=$(/usr/local/Calpont/mysql/bin/mysql -u <user> -p -h <host> <db_name>) #it hangs at this stage
if ! (mysqlshow $DB 1>/dev/null); then
echo ERROR: unable to access database
exit 1
fi
echo "connected"
if ! [ -w $export_dir ]; then
echo ERROR: export dir is not writable
exit 1
fi
ExportTable()
{
/usr/local/Calpont/mysql/bin/mysql -u <user> -p -h <host> <db_name>"
SELECT * INTO OUTFILE '/tmp/$T.csv'
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n'
FROM $T
"
}
/usr/local/Calpont/mysql/bin/mysql -u <user> -p -h <host> <db_name> "show tables" |
while read T; do
echo exporting $T
ExportTable || err_exit
sudo mv $mysql_data_dir/$DB/$T.psv $export_dir/ || err_exit
done
I'm new to bash and trying to use it to loop through all the tables in my database and export them (similar to this script), but it's no use if I can't even connect to the database...
Thanks in advance for your help.
The command substitution that you are assigning to mysql_data_dir is not run interactively so the mysql command must run, generate its output and terminate before the output will be returned to the shell variable. Look closely at your linked example (excerpted below) and you will see that his mysql commmand is just this kind of command line; it includes a mysql command that gets executed and piped into an awk command for post-processing.
mysql_data_dir=$(mysql $myopts "show variables like 'datadir'" | awk '{sub(/\/$/,"");print$NF}')
What you need to do is identify what your "one-liner" mysql command is and put that into your $().
You can usually include the password on the command line.
/usr/local/Calpont/mysql/bin/mysql -u [user] -p[pwd] -h [host] [db_name]
NOTE: there must be NO SPACE between the -p and the password.
If your password has certain punctuation characters you may have to wrap it in double quotes.
Obviously this is the least secure way to do it, but it is the simplest.

How to use the --verbose flag in the MySQL 5.6 command line client?

Going by the list of flags here, with the line at the top about usage: Usage: mysql [OPTIONS] [database].
I am running Windows 8; my table is "contact"; I want to create a tee file for source C:/myfile.sql with the verbose option on.
I have tried mysql -v contact, -v contact, --verbose contact, --verbose source C:/myfile.sql, and various others.
EDIT: adding screenshot of where I'm trying to run this, in case it helps.
The correct syntax for a verbose interactive session is:
c:\> mysql -u yourUser -p -v yourDatabase
This will launch an interactive session (prompting you for password), and set yourDatabase as the default database for the session.
Once in this interactive session, issue a tee command:
mysql> tee c:/temp/my.out
And now you can source your script:
mysql> source c:/myfile.sql
Of course, you can avoid all this pain by simply putting this in your command prompt:
c:\> mysql -u yourUser -pYourPassword -v yourDatabase < myfile.sql > my.out
This will:
Push the contents of myfile.sql to an "interactive" mysql session... that's essentially a batch process
Redirect all contents of the batch process to my.out
Reference:
MySQL Reference Manual: mysql CLI: mysql options
That should work, be aware of the db name.
mysql -v contact
If you db requires login:
mysql -v -udbuser -p contact

Automate mysql_secure_installation with echo command via a shell script

I am trying to automate mysql_secure_installation script with automated response. My code is as follows :
echo "& y y abc abc y y y y" | ./usr/bin/mysql_secure_installation
The actual questions which i am automating are as follows:
Enter current password for root (enter for none): <enter>
Set root password? [Y/n] y
New password: abc
Re-enter new password: abc
Remove anonymous users? [Y/n] y
Disallow root login remotely? [Y/n] y
Remove test database and access to it? [Y/n] y
Reload privilege tables now? [Y/n] y
But it gives me an error "Sorry you cannot use an empty password here" but in the screen i used to press the return key for the first question.
I stumbled upon this question but decided to run the queries manually through a Bash script:
#!/bin/bash
# Make sure that NOBODY can access the server without a password
mysql -e "UPDATE mysql.user SET Password = PASSWORD('CHANGEME') WHERE User = 'root'"
# Kill the anonymous users
mysql -e "DROP USER ''#'localhost'"
# Because our hostname varies we'll use some Bash magic here.
mysql -e "DROP USER ''#'$(hostname)'"
# Kill off the demo database
mysql -e "DROP DATABASE test"
# Make our changes take effect
mysql -e "FLUSH PRIVILEGES"
# Any subsequent tries to run queries this way will get access denied because lack of usr/pwd param
Since mysql_secure_installation is just a Bash script, just check out the raw source code as shown here. Look for the lines that read, do_query (note that extra space I placed after do_query; need to find queries versus the function) and then you can find these commands.
UPDATE mysql.user SET Password=PASSWORD('root') WHERE User='root';
DELETE FROM mysql.user WHERE User='';
DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1');
DROP DATABASE IF EXISTS test;
DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%';
FLUSH PRIVILEGES;
Note that for this example, I have the password being set to root but feel free to change that to match your setup needs. Anyway, take that simply pile of MySQL commands and save it in a file named mysql_secure_installation.sql.
With that done, just run the following command via script to secure the MySQL install:
mysql -sfu root < "mysql_secure_installation.sql"
The s silences errors and the f forces the commands to continue even if one chokes. The u relates to the username that immediately follows it which—in this case—is clearly root.
Run that in a deployment script where MySQL is installed initially without a password and you are all set to lock it down without any keyboard interaction.
PS: This script was put together to secure a MySQL installation on Ubuntu 14.04 which was installed with the export DEBIAN_FRONTEND=noninteractive set and the actual install command being set to sudo -E aptitude install -y --assume-yes -q mysql-server mysql-client. Doing that will cleanly install MySQL on Ubuntu without a password; which is nice for deployment scripts. This mysql -sfu root < "mysql_secure_installation.sql" just locks it all down in seconds after that install happens.
I just did this on CentOS 6.7 with the following:
mysql_secure_installation <<EOF
y
secret
secret
y
y
y
y
EOF
You can try to use expect, that automates interactive applications.
Look at this automating mysql_secure_installation or at my modification.
Here is an automated script for a fresh MySQL 5.7 installation based on #Giacomo1968's answer. Works fine on CentOS 7.5.1804.
yum localinstall -y https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm
yum install -y mysql-community-server
# start mysql service
service mysqld start
# get Temporary root Password
root_temp_pass=$(grep 'A temporary password' /var/log/mysqld.log |tail -1 |awk '{split($0,a,": "); print a[2]}')
echo "root_temp_pass:"$root_temp_pass
# mysql_secure_installation.sql
cat > mysql_secure_installation.sql << EOF
# Make sure that NOBODY can access the server without a password
UPDATE mysql.user SET Password=PASSWORD('yourrootpass') WHERE User='root';
# Kill the anonymous users
DELETE FROM mysql.user WHERE User='';
# disallow remote login for root
DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1');
# Kill off the demo database
DROP DATABASE IF EXISTS test;
DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%';
# Make our changes take effect
FLUSH PRIVILEGES;
EOF
mysql -uroot -p"$root_temp_pass" --connect-expired-password <mysql_secure_installation.sql
sudo mysql -e "SET PASSWORD FOR root#localhost = PASSWORD('123');FLUSH PRIVILEGES;"
printf "123\n n\n n\n n\n y\n y\n y\n" | sudo mysql_secure_installation
Enter current password for root (enter for none)? (I have 123 set for root)
Switch to unix_socket authentication? n
Change the root password? n
Remove anonymous users? n
Disallow root login remotely? y
Remove test database and access to it? y
Reload privilege tables now? y
Version:
mysql Ver 15.1 Distrib 10.4.6-MariaDB, for osx10.14 (x86_64) using readline 5.1
It's not necessary to use expect or to fish the SQL commands out of the source code (although if you want to, the C++ file you are looking for is here: https://github.com/mysql/mysql-server/blob/7ed30a748964c009d4909cb8b4b22036ebdef239/client/mysql_secure_installation.cc)
If you are happy with the defaults in mysql_secure_installation (the most secure option is always the default) then you can use the --use-default option to skip most of the interaction. mysql_secure_installation will still ask you for a root password interactively if one is not set, so you can just script that away by setting it before calling mysql_secure_option.
Here's an example:
mysql -u root <<EOF
SET PASSWORD FOR root#localhost = '${ROOT_PASSWORD}';
FLUSH PRIVILEGES;
EOF
mysql_secure_installation -u root --password="${ROOT_PASSWORD}" --use-default
I had the same problem.
Replacing the echo command to use -e and \n seems to have fixed it.
echo -e "\ny\ny\nabc\nabc\ny\ny\ny\ny\n" | ./usr/bin/mysql_secure_installation
Works for AWS. Amazon Linux 2 AMI.
Custom settings to start an instance (AWS User data):
#!/bin/bash
sudo yum -y update &> /dev/null
wget https://repo.mysql.com/mysql80-community-release-el7-1.noarch.rpm &> /dev/null
sudo yum -y localinstall mysql80-community-release-el7-1.noarch.rpm
sudo yum -y install mysql-community-server &> /dev/null
sudo service mysqld start
# get Temporary root Password
root_temp_pass=$(sudo grep 'A temporary password' /var/log/mysqld.log |tail -1 |awk '{split($0,a,": "); print a[2]}')
echo "root_temp_pass: " $root_temp_pass
# mysql_secure_installation.sql
sudo cat > mysql_secure_installation.sql << EOF
# Make sure that NOBODY can access the server without a password
ALTER USER 'root'#'localhost' IDENTIFIED BY 'yourrootpass';
# Kill the anonymous users
DELETE FROM mysql.user WHERE User='';
# disallow remote login for root
DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1');
# Kill off the demo database
DROP DATABASE IF EXISTS test;
DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%';
# Make our changes take effect
FLUSH PRIVILEGES;
EOF
sudo mysql -uroot -p"$root_temp_pass" --connect-expired-password <mysql_secure_installation.sql
sudo systemctl enable mysql
You could try this:
echo -e "\ny\ny\nabc\nabc\ny\ny\ny\ny" | ./usr/bin/mysql_secure_installation
Just tested this on Ubuntu Bionic 18.04LTS
Step #1
export MYPWD="D33Ps3CR3T";
export NEWPWD="D33P3Rs3CR3T";
Step #2
# First time **ever**
sudo mysql_secure_installation 2>/dev/null <<MSI
n
y
${MYPWD}
${MYPWD}
y
y
y
y
MSI
# Did it work?
mysql -u root -p${MYPWD} -e "SELECT 1+1";
# -------
Step #3
# Every subsequent time
sudo mysql_secure_installation 2>/dev/null <<MSI2
${MYPWD}
n
y
${NEWPWD}
${NEWPWD}
y
y
y
y
MSI2
# Just in case (optional) ....
sudo service mysql restart
# Did it work?
mysql -u root -p${NEWPWD} -e "SELECT 1+1";
You should be able to cut'n paste steps #2 & #3 directly into a terminal, after editing the before and after passwords from step #1.
Notes
If a root password has already been set step #2 will fail, so go to step #3
It's just a heredoc fed into the command
sudo is obligatory.
MSI has no particular meaning (it's collision avoidance; I use EOF elsewhere in the script)
MYPWD == NEWPWD is allowed
2>/dev/null hides the warning "stty: 'standard input': Inappropriate ioctl for device"
You can use &>/dev/null for fully silent mode.
I am using simple command to change root password after MySql installation ,But getting the Above error (signal 9 kill)
(FATAL: Chef::Exceptions::ChildConvergeError: Chef run process terminated by signal 9 (KILL)) Though the command works and password is changed the error is confusing.
script "change password" do
interpreter "bash"
user "root"
cwd "/tmp"
code <<-EOH
#MYSQL
root_temp_pass=$(grep 'A temporary password' /mysql/log/mysqld.log |tail -1 |awk '{split($0,a,": "); print a[2]}')
#Login as root change password
mysql -uroot -p"$root_temp_pass" -Be "ALTER USER 'root'#'localhost' IDENTIFIED BY 'Czt732ck#';" --connect-expired-password
EOH
end
I use following lines. Works fine for AWS Linux AMI 2018
db_root_password=Password4root
cat <<EOF | mysql_secure_installation
y
0
$db_root_password
$db_root_password
y
y
y
y
y
EOF
The top-voted solution is a bit of a hack, and is version, spin, and OS specific. Unfortunately the elegant solution (using --use-defaults) is not supported by the script shipped with RHEL9.
Here is a somewhat more compatible hack to extract the correct commands from the shipped script:
grep 'do_query ' /usr/bin/mariadb-secure-installation | \
sed -e 's/ *do_query \"//' -e 's/\"$//' \
-e "s/\$esc_pass/$db_admin_pass/" \
-e 's/([^;])$/\\1;/' | \
grep -v 'show create' | \
mysql --user=$db_admin_user
The first 2 sed commands strip off the do_query call from each SQL command.
The second line sets your new root password.
The third line adds a trailing semicolon to any SQL commands not so terminated; this is broken in the script shipped with RHEL9
The command assumes you provide the variables $db_admin_user and $db_admin_pass
The code was pulled from my puppet manifest so command quoting may differ depending on how you use it.
The mysql command needs root privileges (which it should inherit from your automation engine)
echo -e "${MARIADB_ROOT_PASSWORD}\nY\nn\nY\nn\nY\nY\n" | ./usr/bin/mysql_secure_installation
...did work for me, in my mariadb docker container. Without this call I could not get simple things like /etc/init.d/mariadb status running.
In Windows OS just search for 'mysql_secure_installation' application usually found in Drive:\MySQL_Install-DIR\bin\
By pressing WindowKey + R just run the command 'Drive:\MySQL_Install-DIR\bin\mysql_secure_installation'
When you run this command a window will pop up that will walk you through the process of securing your MySQL installation. That's it!

Conditional commands in ssh & shell & mysql

I have the following commands in a shell script where I do a mysql dump, then I load that SQL file over ssh into a remote database, and then I update the timestamp.
1. mysqldump -u root files path | gzip -9 > $SQL_FILE
2. cat $SQL_FILE | ssh -i ~/metadata.pem ubuntu#1.2.3.4
"zcat | mysql -u 'root' -h 1.2.3.4 metadata"
3. TIMESTAMP=`date "+%Y-%m-%d-%T"`
4. mysql -u 'root' -h 1.2.3.4 metadata -e "UPDATE path_last_updated SET timestamp=DEFAULT"
Is there any way to improve the above commands. For example, what happens if line 2 fails (for example, due to a connectivity issue), but line 4 succeeds?
How would I make line 4 running conditional on the success of line 2?
You could chain all in one block:
mysqldump -u root files path |
gzip -9 |
ssh -i ~/metadata.pem ubuntu#1.2.3.4 "zcat |\
mysql -u 'root' -h 1.2.3.4 metadata" &&
mysql -u 'root' -h 1.2.3.4 metadata -e "
UPDATE path_last_updated SET timestamp=DEFAULT"
So last mysql command won't be executed if something fail before.
You can use $? for get return code of last command, if it's not 0, it failed.
Or you can use && for example : cmd1 && cmd2.
Or you can use set -e to stop script if occur an error.