How to make a wget and mysql bash script - mysql

I want to make a bash script that will get a bunch of urls from a mysql table and pass all the urls one by one to the wget --spider , and update the mysql with the response code from wget: "HTTP request sent, awaiting response... 200 OK"
I really don't know where to start from ..
Thank you

#!/bin/bash
DBNAME=DB_NAME
USER=USER
PASS=PASS
HOST=localhost
STATEMENT="select name from environment;"
COUNTER=$(mysql $DBNAME -h $HOST -u$USER -p$PASS -e "$STATEMENT")
for entry in $COUNTER; do
echo wget $entry
done
Here is the output
./mql.sh
wget name
wget Prod
wget uat

Related

Switch user on Google Compute Engine Startup Script

I pass the following as my GCE startup script but it always logs in as the root user and never as the demo-user. How do I fix it?
let startupScript = `#!/bin/bash
su demo-user
WHO_AM_I=$(whoami)
echo WHO_AM_I: $WHO_AM_I &>> debug.txt
cd..`
I think it should work like that:
#! /bin/bash
sudo -u demo-user bash -c 'WHO_AM_I=$(whoami);
echo WHO_AM_I; $WHO_AM_I &>> debug.txt;'
use "sudo-u" to specify the user, then bash -c 'with all the commands between these particular quotes '' and separated by ;
For example: bash -c 'command1; command2;'
You can try an easier test (it worked for me), for example:
#! /bin/bash
sudo -u demo-user bash -c 'touch test.txt'
And then check with ls -l /home/demo-test/text.txt that demo-test is the owner of the new file.

Automate MYSQL CREATE DATABASE on terminal

I'm running a script I have created that downloads and installs WordPress. To save more time I wanted to open MAMP MYSQL and create a new database like so:
/Applications/MAMP/Library/bin/mysql --host=localhost -uroot -proot
CREATE DATABASE $1; // passed into the function as argument
exit; // to get out of MYSQL
The rest of my function:
wordpress() {
cd /Volumes/example/example/Web/dev; mkdir $1;
cd $1;
curl -O https://wordpress.org/latest.tar.gz;
tar -xvzf latest.tar.gz;
mv wordpress/* .;
rmdir wordpress/;
rm latest.tar.gz;
cp wp-config-sample.php wp-config.php;
vim -s /Volumes/example/example/Web/dev/db_overwrite.txt wp-config.php;
curl -O https://downloads.wordpress.org/plugin/wp-super-cache.latest-stable.zip;
curl -O https://downloads.wordpress.org/plugin/wp-migrate-db.latest-stable.zip;
curl -O https://downloads.wordpress.org/plugin/contact-form-7.latest-stable.zip;
curl -O https://downloads.wordpress.org/plugin/wordpress-seo.latest-stable.zip
mv wp-super-cache.latest-stable.zip wp-migrate-db.latest-stable.zip contact-form-7.latest-stable.zip wordpress-seo.latest-stable.zip wp-content/plugins;
cd wp-content/plugins;
cp /Volumes/example/example/Web/Plugins/advanced-custom-fields-pro.zip .
unzip advanced-custom-fields-pro.zip;
unzip wp-super-cache.latest-stable.zip;
unzip wp-migrate-db.latest-stable.zip;
unzip contact-form-7.latest-stable.zip;
unzip wordpress-seo.latest-stable.zip;
rm wp-super-cache.latest-stable.zip wp-migrate-db.latest-stable.zip contact-form-7.latest-stable.zip wordpress-seo.latest-stable.zip;
cd ../themes;
git clone https://github.com/example/my-template;
}
How would I add the MYSQL lines to this function and make it work as intended?
Thanks
Why not use:
/Applications/MAMP/Library/bin/mysqladmin -uroot -proot create mydatabase
mysqladmin is a command-line tool that is part of the standard MySQL client software. It should be present everywhere that the mysql tool is present.
You might like to read https://dev.mysql.com/doc/refman/5.7/en/mysqladmin.html
Another alternative is to pass the CREATE DATABASE as an argument to mysql:
/Applications/MAMP/Library/bin/mysql ... -e "CREATE DATABASE mydatabase"

How do I fix my shell script for this curl command

Hey guys I'm running a shell script through a few loops to acquire records but the curl command is giving me an invalid json error, what am I doing wrong?
resp=$(curl -g -u "${usr}":"${pwd}" -X GET "${env}"/"${name}"/res/"${type}"?where={%22timestamp%22:{%22$gt%22:{%22$date%22:%22"${date1}"%22},%22$lt%22:{%22$date%22:%22"${date2}"%22}}}&paging=limit:100,page:${i})
Quoting hell. printf can help:
url=$(printf '%s/%s/res/%s?where={%%22timestamp%%22:{%%22$gt%%22:{%%22$date%%22:%%22%s%%22},%%22$lt%%22:{%%22$date%%22:%%22%s%%22}}}&paging=limit:100,page:%s' "$env" "$name" "$type" "$date1" "$date2" "$i")
resp=$(curl -g -u "$usr:$pwd" -X GET "$url")

How to deserialize a Riak backup into a JSON?

I have just dumped a riak db (back-up). But the backup file is a binary file.
Is there a lib that it deserialize it into a human readable file (JSON w/e) ?
I haven't found anything on google, neither on Stack Overflow.
Found a solution for my current problem:
Connect to the env and then run following command:
wget https://s3-us-west-2.amazonaws.com/ps-tools/riak-data-migrator-0.2.9-bin.tar.gz
tar -xvzf riak-data-migrator-0.2.9-bin.tar.gz
cd riak-data-migrator-0.2.9
java -jar riak-data-migrator-0.2.9.jar -d -r /var/riak_export -a -h 127.0.0.1 -p 8087 -H 8098
(source: https://github.com/basho-labs/riak-data-migrator)
EDIT
Another way to export riak db https://www.npmjs.com/package/riak-bucket-exporter
#!/bin/bash
for bucket in $(curl http://localhost:8098/riak?buckets=true | sed -e 's/[{}:"]//gi' -e 's/buckets\[//' -e 's/\]//' -e 's/,/ /g')
do
echo "Exporting bucket $bucket"
rm -f $bucket.json
riak-bucket-exporter -H localhost -p 8098 $bucket
done
echo "Export done"
As all the suggestions listed so far appear to be broken in one way or another (at least for me and riak-kv#2.x), I ultimately resorted to homegrow a bash shell script that leverages riak-kv's HTTP API with no other prerequisites than curl and jq to accomplish an export of sorts.
It can be found in this gist here: https://gist.github.com/cueedee/0b26ec746c4ef578cd98e93c93d2b6e8 hoping that someone will find it useful.

How to execute MySQL command from the host to container running MySQL server?

I have followed the instruction in https://registry.hub.docker.com/_/mysql/ to pull an image and running a container in which it runs a MySQL server.
The container is running in the background and I would like to run some commands.
Which is the best way to connect to the container and execute this command from command line?
Thanks.
You can connect to your mysql container and run your commands using:
docker exec -it mysql bash -l
(Where mysql is the name you gave the container)
Keep in mind that anything you do will not persist to the next time your run a container from the same image.
docker exec -i some_mysql_container mysql -uroot -ppassword <<< "select database();"
To connect to the MySQL database using MySQL command line client.
I connect to the bash into the running MySQL container:
$ docker exec -t -i container_mysql_name /bin/bash
-i is the shortcut for --interactive option. This options is used for keep STDIN open even if not attached
-t is the shortcut for --tty option, used to allocate a pseudo-TTY
I run MySQL client from bash MySQL container:
$ mysql -uroot -proot
-u is shortcut for --user=name option, used to define user for login if not current user.
-p is shortcut for -password[=name] option, used to define password to use when connecting to server. If password is not given it's asked from the tty.
Disco!
In my case the <<< solution did not work.
Instead I used -e.
Example:
docker exec ${CONTAINER_NAME} mysql -u ${USER_NAME} -p${PASSWORD} -e "drop schema test; create schema test;"
For #Abdullah Jibaly solution, after tested in MySQL 5.7, it would only entered into bash terminal prompt, whereby you still need to enter mysql command second time.
In order to directly enter into MySQL command line client after run MySQL container with one line of command, just run the following:
docker exec -it container_mysql_name mysql -u username -p
Its possible with docker run, start a new container just to execute your mysql statement.
This approach helped me to workaround the access denied problem when you try to run a statement with docker exec using localhost to connect to mysql
$ docker run -it --rm mysql mysql -h172.17.0.2 -uroot -pmy-secret-pw -e "show databases;"
I use the following to create a command that will sort out at least a couple of cases with databases outside or inside the container (with -h and -P) and supporting -e:
cat > ~/bin/mysql <<'EOF'
#/bin/bash
MARGS=()
MPORT="3306"
while test $# != 0; do
if [[ $1 == -h ]]; then MHOST=$2; shift;
elif [[ $1 == -h* ]]; then MHOST=${1#"-h"};
elif [[ $1 == -e ]]; then MEXEC=$2; shift;
elif [[ $1 == -e* ]]; then MEXEC=${1#"-e"};
elif [[ $1 == --execute=* ]]; then MEXEC=${1#"--execute="};
elif [[ $1 == -P ]]; then MPORT=$2; shift;
elif [[ $1 == -P* ]]; then MPORT=${1#"-P"};
else MARGS="$MARGS $1"
fi
shift;
done
if [ -z "${MHOST+x}" ]; then
MHOST=localhost
fi
if [ $(docker inspect --format '{{ .State.Status }}' mysql) == "running" ]; then
if [ ! -z "${MHOST+x}" ]; then
if [ "$MHOST" == "localhost" -o "$MHOST" == "127.0.0.1" ]; then
CPORT=$(docker port mysql 3306/tcp)
if [ ${CPORT#"0.0.0.0:"} == $MPORT ]; then
#echo "aiming for container port ($MPORT -> $CPORT)";
MHOST=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' mysql);
else
MHOST=$(ifconfig | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p' | head -1);
fi
fi
fi
fi
if [ -z "$MEXEC" ]; then
docker run --link mysql:mysql -i --rm mysql mysql "-h" $MHOST "-P" $MPORT $MARGS
else
docker run --link mysql:mysql -i --rm mysql mysql "-h" $MHOST "-P" $MPORT $MARGS <<< $MEXEC
fi
EOF
chmod +x ~/bin/mysql
i didn't find any of these solutions to be effective for my use case: needing to store the returned data from the SQL to a bash variable.
i ended up with the following syntax when making the call from inside a bash script running on the host computer (outside the docker mysql server), basically use 'echo' to forward the SQL statement to stdin on the docker exec command.
modify the following to specify the mysql container name and proper mysql user and password for your use case:
#!/bin/bash
mysqlCMD="docker exec -i _mysql-container-name_ mysql -uroot -proot "
sqlCMD="select count(*) from DBnames where name = 'sampleDB'"
count=`echo $sqlCMD | $mysqlCMD | grep -v count`
# count variable now contains the result of the SQL statement
for whatever reason, when i used the -e option, and then provided that string within the back-quotes, the interpreter modified the quotation marks resulting in SQL syntax failure.
richard