I'm having trouble using MySQL; it pull extra data.
I have a data base with IPs and Host in it:
ips host
127.0.0.1 host1.exampl.com
127.0.0.2 host2.exampl.com
127.0.0.3 host3.exampl.com
127.0.0.4 host4.exampl.com
127.0.0.5 host5.exampl.com
Next, I have a file ips.txt:
127.0.0.2
127.0.0.4
And I have a bash script:
for i in `cat ips.txt` ; do echo "ip, host from ip_group_list where ip like '%$i%' GROUP BY ip" | mysql -uroot -p$PSSWD database -N >> ipsandhost.txt; done
I want it to return
127.0.0.2 host2.exampl.com
127.0.0.4 host4.exampl.com
However it returns
127.0.0.1 host1.exampl.com
127.0.0.2 host2.exampl.com
127.0.0.3 host3.exampl.com
127.0.0.4 host4.exampl.com
127.0.0.5 host5.exampl.com
Any Suggestions?
You don't really need neither %% nor GROUP BY
for i in `cat ips.txt` ; do
echo "SELECT ip, host from ip_group_list where ip = '$i'" | mysql -uroot -p$PSSWD database -N >> ipsandhost.txt;
done
Related
I am using MySQL on centos7. I have 50 databases Like database1, database2...., database50.
How can I set a cronjob for take a dump every day of all database same as database name [ Like database1.sql, database2.sql .... database50.sql ] using single command or script.
Please provide some adequate solution that will be appreciated.
Thanks.
Convert the current date to an integer number of days since some starting date.
Take that modulo 50. This gives you 0 .. 49.
Add 1 and concatenate. Now you have database1 .. database50. Put that in the shell variable db
mysqldump ... $db >$db.sql
I am using this script
#! /bin/bash
# MySQL database backup (databases in separate files) with daily, weekly and monthly rotation
# Sebastian Flippence (http://seb.flippence.net) originally based on code from: Ameir Abdeldayem (http://www.ameir.net)
# You are free to modify and distribute this code,
# so long as you keep the authors name and URL in it.
# Modified by IVO GELOV
# How many backups do you want to keep?
MAX_DAYS=5
# Date format that is appended to filename
DATE=`date +'%Y-%m-%d'`
DATSTR=`date '+%Y%m%d' -d "-$MAX_DAYS days"`
# MySQL server's name
SERVER=""
# Directory to backup to
BACKDIR="/var/db_arhiv/mysql"
#----------------------MySQL Settings--------------------#
# MySQL server's hostname or IP address
HOST="localhost"
# MySQL username
USER="user"
# MySQL password
PASS="password"
# List all of the MySQL databases that you want to backup,
# each separated by a space. Or set the option below to backup all database
DBS="db1 db2"
# Set to 'y' if you want to backup all your databases. This will override
# the database selection above.
DUMPALL="y"
# Custom path to system commands (enable these if you want use a different
# location for PHP and MySQL or if you are having problems running this script)
MYSQL="/usr/local/mysql/bin/mysql"
MYSQLDUMP="/usr/local/mysql/bin/mysqldump"
function checkMysqlUp() {
$MYSQL -N -h $HOST --user=$USER --password=$PASS -e status > /dev/null
}
trap checkMysqlUp 0
function error() {
local PARENT_LINENO="$1"
local MESSAGE="$2"
local CODE="${3:-1}"
if [[ -n "$MESSAGE" ]] ; then
echo "Error on or near line ${PARENT_LINENO}: ${MESSAGE}; exiting with status ${CODE}"
else
echo "Error on or near line ${PARENT_LINENO}; exiting with status ${CODE}"
fi
exit "${CODE}"
}
trap 'error ${LINENO}' ERR
# Check backup directory exists
# if not, create it
if [ ! -e "$BACKDIR/$DATE" ]; then
mkdir -p "$BACKDIR/$DATE"
echo "Created backup directory (${BACKDIR}/${DATE})"
fi
if [ $DUMPALL = "y" ]; then
echo "Creating list of databases on: ${HOST}..."
$MYSQL -N -h $HOST --user=$USER --password=$PASS -e "show databases;" > ${BACKDIR}/dbs_on_${SERVER}.txt
# redefine list of databases to be backed up
DBS=`sed -e ':a;N;$!ba;s/\n/ /g' -e 's/Database //g' ${BACKDIR}/dbs_on_${SERVER}.txt`
fi
echo "Backing up MySQL databases..."
#cd ${LATEST}
for database in $DBS; do
if [ ${database} = "information_schema" ] || [ ${database} = "performance_schema" ] || [ ${database} = "pinba" ]
then
continue
fi
echo "${database}..."
$MYSQLDUMP --host=$HOST --user=$USER --password=$PASS --default-character-set=utf8 --routines --triggers --lock-tables --disable-keys --force --single-transaction --allow-keywords --dump-date $database > ${BACKDIR}/${DATE}/${SERVER}$database.sql
done
if [ $DUMPALL = "y" ]; then
rm -f ${BACKDIR}/dbs_on_${SERVER}.txt
fi
# dump privileges
$MYSQL -N -h $HOST --user=$USER --password=$PASS --skip-column-names -A -e "SELECT CONCAT('SHOW GRANTS FOR ''',user,'''#''',host,''';') FROM mysql.user" | $MYSQL -N -h $HOST --user=$USER --password=$PASS --skip-column-names -A > ${BACKDIR}/${DATE}/${SERVER}_grants.sql
# delete older files
for x in `find ${BACKDIR}/20* -type d`
do
xd=`basename "${x//-/}"`
if [[ $xd < $DATSTR ]]
then
rm -rf "$x"
fi
done
echo "MySQL backup is complete"
I want to backup my database automatically using cron job and I want to use dynamic filename indicating the date of backup.
* * * * * mysqldump -udbuser -pdbpassword mydb > /backups/mydb.`date +"%Y-%m-%d"`.sql
But it seems doesn't work. I check the cron log it shows like this.
Oct 31 11:18:00 dbuser /usr/sbin/cron[94330]: (dbuser) CMD (mysqldump -udbuser -pdbpassword mydb > /backups/mydb.`date +")
It looks like the command is not executed completely. How can I fix it?
Cronjobs does not understand those variables.
Below is a php script backup that works for me for many years. It backup as sql and zip it directly.
$dbFile = '/data/Backup/backup-'.date('Ymd').'.sql.gz';
$dbHost = 'localhost'; // Database Host
$dbUser = 'username'; // Database Username
$dbPass = 'password'; // Database Password
exec( 'mysqldump --host="'.$dbHost.'" --user="'.$dbUser.'" --password="'.$dbPass.'" --add-drop-table "databasename" | gzip > "'.$dbFile.'"' );
and below is the cronjob
* * * * * /usr/bin/php /path_to_backup_script/backup.php
Sorry for not give the extract answer for your question but you can try this:
- Paste this command to bash file (for example: /tmp/backup.sh):
#!/bin/bash
export TIME=$(date +'%Y%m%d')
export DATABASEHOST=192.168.100.10
export SCHEMA="db01 db02"
export DBPORT="3306"
export DBPS=$(echo 'password')
export BACKUPDIR=$(echo '/backup')
ERR=0
backup()
{
for i in $SCHEMA;
do
echo $TIME >> /tmp/db_tar_backup_err.log
cd $BACKUPDIR
mysqldump -ubackup -p$DBPS -h $DATABASEHOST -P $DBPORT --single-transaction --routines --triggers $i > db_${i}_$TIME.sql
tar -czPf db_${i}_$TIME.tar.gz db_${i}_$TIME.sql 2>> /tmp/db_tar_backup_err.log || ERR=1
find db_${i}_$TIME.sql -delete
done
}
backup()
- Create crontab entry:
0 20 * * * /bin/sh /tmp/backup.sh
Explain:
Your script will backup database db01, db02 in order of SCHEMA varibles and using tar to compress it at 20:00
I am trying to break up a returned value from a mysql call in a shell script. Essentially what I have done so far is query the database for IP addresses that I have stored in a specific table. From there I store that returned value into a bash variable. The code is below:
#!/bin/bash
# This file will be used to obtain the system details for given ip address
retrieve_details()
{
# get all the ip addresses in the hosts table to pass to snmp call
host_ips=$(mysql -u user -ppassword -D honours_project -e "SELECT host_ip FROM hosts" -s)
echo "$host_ips"
# break up the returned host ip values
# loop through array of ip addresses
# while [ ]
# do
# pass ip values to snmp command call
# store details into mysql table host_descriptions
# done
}
retrieve_details
So this returns the following:
192.168.1.1
192.168.1.100
192.168.1.101
These are essentially the values I have in my hosts table. So what I am trying to do is break up each value such that I can get an array that looks like the following:
arr[0]=192.168.1.1
arr[1]=192.168.1.100
arr[2]=192.168.1.101
...
I have reviewed this link here: bash script - select from database into variable but I don't believe this applies to my situation. Any help would be appreciated
host_ips=($(mysql -u user -ppassword -D honours_project -e "SELECT host_ip FROM hosts" -s));
outer () will convert that in array. But you need to change your IFS (Internal Field Separator) to a newline first.
IFS=$'\n';
host_ips=($(mysql -u user -ppassword -D honours_project -e "SELECT host_ip FROM hosts" -s));
unset IFS;
for i in ${host_ips[#]} ; do echo $i ; done;
to print with key
for i in "${!host_ips[#]}"
do
echo "key :" $i "value:" ${host_ips[$i]}
done
wspace#lw:~$ echo $host_ips
192.168.1.1 192.168.1.100 192.168.1.101
wspace#lw:~$ arr=($(echo $host_ips))
wspace#lw:~$ echo ${arr[0]}
192.168.1.1
wspace#lw:~$ echo ${arr[1]}
192.168.1.100
wspace#lw:~$ echo ${arr[2]}
192.168.1.101
wspace#lw:~$ echo ${arr[#]}
192.168.1.1 192.168.1.100 192.168.1.101
wspace#lw:~$
maybe this is what you want
My bash script won't crate the Database. What am I doing wrong here?
Please have a look:
#!/bin/bash -x
set -x
function deebee() {
EXPECTED_ARGS=2
E_BADARGS=65
MYSQL=`which mysql`
Q1="CREATE DATABASE IF NOT EXISTS $1;"
Q2="GRANT USAGE ON *.* TO $2#localhost IDENTIFIED BY '$3';"
Q3="GRANT ALL PRIVILEGES ON $1.* TO $2#localhost;"
Q4="FLUSH PRIVILEGES;"
SQL="${Q1}${Q2}${Q3}${Q4}"
if [ $# -ne $EXPECTED_ARGS ]
then
echo "Usage: $0 dbname dbuser dbpass"
exit $E_BADARGS
fi
$MYSQL -uroot -p -e "$SQL"
}
deebee $1 $2 $3
I'm calling the script as I've put it in a function, but it just spits out the expected arguments telling me the syntax, i.e that I should type in the bashscrip name, dbname, dbuser dbpass, but obviously there' something wrong with the script of my login permissions or user so that I can't automate this... What's going on, I'd love to know!
Thanks!
What I would do :
creating a mysql config file with credentials (easier to automate now) :
cat ~/.my.cnf
[client]
host = localhost
user = root
password = xxx
And the script :
#!/bin/bash
set -x
deebee() {
EXPECTED_ARGS=3
E_BADARGS=65
MYSQL=$(type -p mysql)
if (($# != $EXPECTED_ARGS))
then
echo "Usage: $0 dbname dbuser dbpass"
exit $E_BADARGS
fi
$MYSQL <<EOF
CREATE DATABASE IF NOT EXISTS '$1';
GRANT USAGE ON *.* TO '$2'#'localhost' IDENTIFIED BY '$3';
GRANT ALL PRIVILEGES ON '$1'.* TO '$2'#'localhost';
FLUSH PRIVILEGES;
EOF
}
# TODO tests on input args
deebee "$1" "$2" "$3"
And like Denis said in comments, take care of what users can put as arguments to prevent sql injection.
You should add some tests on input args.
My database had lots of views and it was impossible to drop them one by one.
I would like to just drop them all because the database doesn't refresh structure changes of the tables in the view that select from them.
If you want to do this in the MySQL client, you can dynamically generate the DDL statements using information_schema, dump them to a SQL script, and then execute that script.
Example:
select concat('drop view ',table_schema,'.',table_name,';') as ddl
into outfile '/tmp/drop_all_views.sql'
from information_schema.views
where table_schema = 'your_schema';
\. /tmp/drop_all_views.sql
After searching on the web, I found a shell script to drop all tables here: http://www.cyberciti.biz/faq/how-do-i-empty-mysql-database/
Then, I changed that script to drop all views of the database.
This is the final result:
#!/bin/bash
PREFIX=""
SUFFIX=""
HOST="localhost"
PORT="3306"
while getopts p:s:h:P: OPCAO; do
case "${OPCAO}" in
p) PREFIX="${OPTARG}" ;;
s) SUFFIX="${OPTARG}" ;;
h) HOST="${OPTARG}" ;;
P) PORT="${OPTARG}" ;;
esac
done
shift $((OPTIND-1))
MUSER="$1"
MPASS="$2"
MDB="$3"
# Detect paths
MYSQL=$(which mysql)
AWK=$(which awk)
GREP=$(which grep)
if [ $# -eq 0 ]
then
echo "Usage: $0 [-h Host] [-P Port] [-p Prefix-View-Name] [-s Suffix-View-Name] {MySQL-User-Name} {MySQL-User-Password} {MySQL-Database-Name}"
echo "Drops all views from a MySQL"
exit 1
fi
TABLES=$($MYSQL -h $HOST -P $PORT -u $MUSER -p$MPASS $MDB -e "SELECT table_name FROM information_schema.views WHERE table_schema = '$MDB' AND table_name LIKE '$PREFIX%$SUFFIX';" | $AWK '{ print $1}')
for t in $TABLES
do
echo "Deleting $t view from $MDB database..."
$MYSQL -h $HOST -P $PORT -u $MUSER -p$MPASS $MDB -e "drop view $t"
done
This solution will work just for who uses Unix-like systems.
To call the script, I used:
./script.sh [-h host] [-P port] [-p prefixViewName] [-s suffixViewName] username password databaseName
EDIT: I improved the script to accept option parameters, like the host, port and also a prefix and a suffix to filter what views will be dropped.