I have a Magento database in which I want to search for a particular string/pattern.
But the database's size is too large so I cannot export the database to .sql file and then search into that file(editor even Geany crashes opening such large files).
So how can I do a search the database for a perfect match of [string/pattern] and display fulltext information as result, through only using command-line and MySQL Database credentials ?
I tried below command, but it requires username to be given as -u[USERNAME], also it doesn't display full query or result in terminal window.
mysqldump -p[PASSWORD] [DATABASE] --extended=FALSE | grep [pattern] | less -S
Anyone have any solutions for this ?
You can first log into MySQL CLI as especified in http://dev.mysql.com/doc/refman/5.7/en/connecting.html
mysql --host=localhost --user=myname --password=mypass mydb
So, you can use a query command to find your pattern. If you know the table you want to search such as the column it make the thinks easy. The SELECT statement is like this:
SELECT column FROM table WHERE column LIKE '%pattern%';
http://dev.mysql.com/doc/en/select.html
If you don't know the table's name, you can list all and try to find by the meaning.
SHOW TABLES;
Edited with better code
You didn't say if this was a one off or not but this will check all tables in a schema for a value.
First in your home directory set up a file named .my.cnf with the following contents and change its permissions to 700 (Replace [USERNAME] and [PASSWORD] with your username and password.
[client]
user=[USERNAME]
password="[PASSWORD]"
Then execute the following (Replacing [DATABASE] and [CHECKSTRING] with your database and the check string)
mysql [DATABASE] --silent -N -e "show tables;"|while read table; do mysql [DATABASE] --silent -N -e "select * from ${table};"|while read line;do if [[ "${line}" == *"[CHECKSTRING]"* ]]; then echo "${table}***${line}";fi;done;done
If checking for 51584 the result would be something like
test_table***551584,'column 2 value','column 3 value'
test_table5***'column 1 value',251584,'column 3 value'
If you want to know which column had the value then select from INFORMATION_SCHEMA.COLUMNS and add another nest.
mysql [DATABASE] --silent -N -e "show tables;"|while read table; do mysql [DATABASE] --silent -N -e "select column_name from information_schema.columns where table_schema='[DATABASE]' and table_name = '${table}';"|while read column; do mysql [DATABASE] --silent -N -e "select ${column} from ${table};"|while read line;do if [[ "${line}" == *"[CHECKSTRING]"* ]]; then echo "${table}***${column}***${line}";fi;done;done;done
If checking for 51584 the result would be something like
test_table***column1***551584
test_table5***column2***251584
First of all you need to login into database with correct username and password by below command.
sudo mysql -u root -p
then check the database in which you want to operate operation.
eg.
SHOW DATABASES;
USE Test;
now your database is ready for operation through terminal. Here I assume my database name is "Test".
Now for String/pattern matching use command as below or follow the link http://www.mysqltutorial.org/mysql-regular-expression-regexp.aspx.
SELECT
column_list
FROM
table_name
WHERE
string_column REGEXP pattern;
I need to run a monthly bash script via cron that is related to our company's billing system. This is done with two stored procedures. When I run them via the MySQL console and workbench, they work fine.
I've looked at this article and this is basically the way I do it.
I call via cron, a shell script that looks like this:
mysql -h 192.168.1.1 -u<username> -p<password> mydatabase < /path/to/billing_periods.sql
My text file that has the commands in it looks like this:
call sp_start_billing_period();
call sp_bill_clients();
What happens is that the first query runs, but the second one on the second line, doesn't.
I can make a stored procedure that wraps these two - but I just was hoping to learn why this was happening... Perhaps a mistake I made or a limit in the way you do this..
I also considered doing this (two calls to the MySQL shell):
mysql -h 192.168.1.1 -u<username> -p<password> mydatabase -e "call sp_start_billing_period();"
mysql -h 192.168.1.1 -u<username> -p<password> mydatabase -e "call sp_bill_clients();"
You could try separating each statement with a semicolon.
mysql -h 192.168.1.1 -u<username> -p<password> mydatabase -e "call sp_start_billing_period();call sp_bill_clients();"
If you have your statements in a file you can do:
while read LINE; do mysql -u<username> -p<password> mydatabase -e"$LINE";echo "-----------";done < statements.sql
I think you are only allowed to execute a single statement in your input .sql file, see the mysql documentation (manpage) for -e statement.
· --execute=statement, -e statement
Execute the statement and quit. The default output format is like that produced with --batch.
The -e is implicit. At least when I do different mysql queries I put them in their own script like you already suggested.
Am facing problem to connect the MySQL DB from shell script. Please find the below snippet i have written for connecting the MySQL data base. please suggest on this.
My shell Script:
#!bin/bash
Query="select * from Main"
MySQL -u root -p '!!root!!' -e kpi << EOF
$Query;
EOF
Please check the above code and suggest me how to connect the DB.
I think it should be
-pThePassword
So you should delete the space between -p and the pass. Also you should not use an apostrophe (except it is part of the pass itself. Use a backslash to escape special characters.
Second: *nix systems are case sensitive, please try mysql instead of MySQL
Update
You could also try to type your password into a file and read it with your script
mysql -u root -p`cat /tmp/pass` -e "SHOW DATABASES"
The file /tmp/pass should contain your password without any newline char at the end.
Update 2
Your Script is wrong.
You can either use mysql ... -e SELECT * FROM TABLE or mysql ... << EOF (without -e). You should not mix them.
Don't forget to pass the databasename as a parameter (or with use databasename;) in the sql
Don't forget to add a ; after every sql command, if you have multiple statements
Method One:
mysql -u root -ppassword databasename -e "SELECT * FROM main"
Method Two:
mysql -u root -ppassword databasename << EOF
SELECT * FROM main
EOF
Method Three:
mysql -u root -ppassword << EOF
USE databasename;
SELECT * FROM main;
EOF
mysql --user=root --password=xxxxxx -e "source dbscript.sql"
This should work for Windows and Linux.
If the password content contains a ! (Exclamation mark) you should add a \ (backslash) in front of it.
MySQL has an OPTIMIZE TABLE command which can be used to reclaim unused space in a MySQL install. Is there a way (built-in command or common stored procedure) to run this optimization for every table in the database and/or server install, or is this something you'd have to script up yourself?
You can use mysqlcheck to do this at the command line.
One database:
mysqlcheck -o <db_schema_name>
All databases:
mysqlcheck -o --all-databases
I made this 'simple' script:
set #tables_like = null;
set #optimize = null;
set #show_tables = concat("show tables where", ifnull(concat(" `Tables_in_", database(), "` like '", #tables_like, "' and"), ''), " (#optimize:=concat_ws(',',#optimize,`Tables_in_", database() ,"`))");
Prepare `bd` from #show_tables;
EXECUTE `bd`;
DEALLOCATE PREPARE `bd`;
set #optimize := concat('optimize table ', #optimize);
PREPARE `sql` FROM #optimize;
EXECUTE `sql`;
DEALLOCATE PREPARE `sql`;
set #show_tables = null, #optimize = null, #tables_like = null;
To run it, simply paste it in any SQL IDE connected to your database.
Notice: this code WON'T work on phpmyadmin.
How it works
It runs a show tables statement and stores it in a prepared statement. Then it runs a optimize table in the selected set.
You can control which tables to optimize by setting a different value in the var #tables_like (e.g.: set #tables_like = '%test%';).
Following example php script can help you to optimize all tables in your database
<?php
dbConnect();
$alltables = mysql_query("SHOW TABLES");
while ($table = mysql_fetch_assoc($alltables))
{
foreach ($table as $db => $tablename)
{
mysql_query("OPTIMIZE TABLE '".$tablename."'")
or die(mysql_error());
}
}
?>
If you want to analyze, repair and optimize all tables in all databases in your MySQL server, you can do this in one go from the command line. You will need root to do that though.
mysqlcheck -u root -p --auto-repair --optimize --all-databases
Once you run that, you will be prompted to enter your MySQL root password. After that, it will start and you will see results as it's happening.
Example output:
yourdbname1.yourdbtable1 OK
yourdbname2.yourdbtable2 Table is already up to date
yourdbname3.yourdbtable3
note : Table does not support optimize, doing recreate + analyze instead
status : OK
etc..
etc...
Repairing tables
yourdbname10.yourdbtable10
warning : Number of rows changed from 121378 to 81562
status : OK
If you don't know the root password and are using WHM, you can change it from within WHM by going to:
Home > SQL Services > MySQL Root Password
Do all the necessary procedures for fixing all tables in all the databases with a simple shell script:
#!/bin/bash
mysqlcheck --all-databases
mysqlcheck --all-databases -o
mysqlcheck --all-databases --auto-repair
mysqlcheck --all-databases --analyze
From phpMyAdmin and other sources/editors you can use:
SET SESSION group_concat_max_len = 99999999;
SELECT GROUP_CONCAT(concat('OPTIMIZE TABLE `', table_name, '`;') SEPARATOR '') AS O
FROM INFORMATION_SCHEMA.TABLES WHERE
TABLE_TYPE = 'BASE TABLE'
AND table_name!='dual'
AND TABLE_SCHEMA = '<your databasename>'
Then you can copy & paste the result to a new query or execute it from your own source.
If you don't see the whole statement in phpMyAdmin:
for all databases:
mysqlcheck -Aos -uuser -p
For one Database optimization:
mysqlcheck -os -uroot -p dbtest3
From command line:
mysqlcheck -o <db_name> -u<username> -p
then type password
You can optimize/check and repair all the tables of database, using mysql client.
First, you should get all the tables list, separated with ',':
mysql -u[USERNAME] -p[PASSWORD] -Bse 'show tables' [DB_NAME]|xargs|perl -pe 's/ /,/g'
Now, when you have all the tables list for optimization:
mysql -u[USERNAME] -p[PASSWORD] -Bse 'optimize tables [tables list]' [DB_NAME]
my 1 cent, added and TABLE_TYPE='BASE TABLE' so we can skip the 'VIEW' type.
for table in `mysql -sss -e "select concat(table_schema,'.',table_name) from information_schema.tables where table_schema not in ('mysql','information_schema','performance_schema') and TABLE_TYPE='BASE TABLE' order by data_free desc;"`
do
mysql -e "OPTIMIZE TABLE $table;"
done
The MySQL Administrator (part of the MySQL GUI Tools) can do that for you on a database level.
Just select your schema and press the Maintenance button in the bottom right corner.
Since the GUI Tools have reached End-of-life status they are hard to find on the mysql page. Found them via Google: http://dev.mysql.com/downloads/gui-tools/5.0.html
I don't know if the new MySQL Workbench can do that, too.
And you can use the mysqlcheck command line tool which should be able to do that, too.
A starter bash script to list and run a tool against the DBs...
#!/bin/bash
declare -a dbs
unset opt
for each in $(echo "show databases;" | mysql -u root) ;do
dbs+=($each)
done
echo " The system found [ ${#dbs[#]} ] databases." ;sleep 2
echo
echo "press 1 to run a check"
echo "press 2 to run an optimization"
echo "press 3 to run a repair"
echo "press 4 to run check,repair, and optimization"
echo "press q to quit"
read input
case $input in
1) opt="-c"
;;
2) opt="-o"
;;
3) opt="-r"
;;
4) opt="--auto-repair -c -o"
;;
*) echo "Quitting Application .."; exit 7
;;
esac
[[ -z $opt ]] && exit 7;
echo " running option: mysqlcheck $opt in 5 seconds on all Dbs... "; sleep 5
for ((i=0; i<${#dbs[#]}; i++)) ;do
echo "${dbs[$i]} : "
mysqlcheck $opt ${dbs[$i]} -u root
done
If you are accessing database directly then you can write following query:
OPTIMIZE TABLE table1,table2,table3,table4......;
This bash script will accept the root password as option and optimize it one by one, with status output:
#!/bin/bash
if [ -z "$1" ] ; then
echo
echo "ERROR: root password Parameter missing."
exit
fi
MYSQL_USER=root
MYSQL_PASS=$1
MYSQL_CONN="-u${MYSQL_USER} -p${MYSQL_PASS}"
TBLLIST=""
COMMA=""
SQL="SELECT CONCAT(table_schema,'.',table_name) FROM information_schema.tables WHERE"
SQL="${SQL} table_schema NOT IN ('information_schema','mysql','performance_schema')"
for DBTB in `mysql ${MYSQL_CONN} -ANe"${SQL}"`
do
echo OPTIMIZE TABLE "${DBTB};"
SQL="OPTIMIZE TABLE ${DBTB};"
mysql ${MYSQL_CONN} -ANe"${SQL}"
done
my 2cents: start with table with highest fragmentation
for table in `mysql -sss -e "select concat(table_schema,".",table_name) from information_schema.tables where table_schema not in ('mysql','information_schema','performance_schema') order by data_free desc;"
do
mysql -e "OPTIMIZE TABLE $table;"
done
If local moderators allow this, I would like to promote a PHP library I wrote quite some time ago - https://github.com/Simbiat/optimize-tables
The point of the library is to allow "smart" execution of the OPTIMIZE, ANALYZE, CHECK and REPAIR commands depending on table's parameters and statistics. I've been running it in CRON for https://simbiat.ru for over 2 years now and it has been smooth sailing (aside from some adjustments and minor fixes, of course).
Why would you want to use something like this? Well, the README provides some more details, but in short, it can help with running relevant operations only when you can actually benefit from them. At the least, it can save your resources.
I have many databases with different names.
I want to drop multiple databases, Is there any command since all names of db are different.
Eg: mysql db, Test db, live db.
As of I know, there is no specific command/query to delete multiple databases without having a specific pattern in their names. Even I was asked to do the favor several times. So I researched and found no specific solution. Then I tried the below hack. It worked without giving much trouble. May be it could help for you too.
Take all the databases using the below command.
SHOW DATABASES ;
Paste all of them in an excel/some other text file (I prefer NPP). Keep the only names which you want to delete from the list. Dont forget to remove your working db's from the list.
Add DROP DATABASE in front of those names.
That's it simple. Copy & Paste all of those in your workbench. You can execute all of them in one shot.
If you create a shell script this should remove all the databases. You will need to edit it to suit your needs.
DBUSER='user'
DBPASS='password'
SQLFILE='/path/to/file/databases.sql'
echo '* Dropping ALL databases'
DBS="$(mysql -u$DBUSER -p$DBPASS -Bse 'show databases' | grep -v Database | grep -v database | grep -v mysql | grep -v information_schema)"
for db in $DBS; do
echo "Deleting $db"
mysql -u$DBUSER -p$DBPASS -Bse "drop database $db; select sleep(0.1);"
done
First run this query to produce a list of drop commands:
select CONCAT('drop database `', schema_name,'`;') as database_name from information_schema.schemata where schema_name like '%DATABASES_TO_REMOVE%' order by schema_name;
Then copy the output rows of this query and paste into a query window
In my case I then needed to remove the single-quotes (') surrounding the resulting command queries which I did using a simple find + replace (often Ctrl + H, replace ' with < empty >)
And execute (highlighting all of the drop statements in my case)!
Unfortunetly, there is nothing like that, unless you create your own function.
simple bash script can be done this work
#!/bin/bash
cat /home/mshafee/file | while read line
do
mysql -u username -p****** -h 0.0.0.0 -e "drop database $line;"
done
here provide username, password and IP address.