How to launch process with Fabric - mysql

How do you launch a process using Fabric?
I'm trying to write a task to reset MySQL's password, which requires I launch the mysqld_safe daemon using:
sudo bash -c "mkdir -p /var/run/mysqld; chown mysql /var/run/mysqld; mysqld_safe --skip-grant-tables &"
When I run this manually on the server, ps aux|grep -i mysql shows it's running just fine, but when I run this using Fabric's sudo(), it appears to terminate almost immediately. What am I doing wrong?
I've also tried using nohup like:
sudo bash -c "mkdir -p /var/run/mysqld; chown mysql /var/run/mysqld; nohup mysqld_safe --skip-grant-tables &"
but that had no effect.

http://www.fabfile.org/faq.html#why-can-t-i-run-programs-in-the-background-with-it-makes-fabric-hang has a very clear explanation of why that is. Also proposes the solution.
In your case, you should install dtach and run:
sudo(dtach -n `mktemp -u /tmp/{0}.XXXX` {1}'.format(some_name, cmd_to_execute))
You can drop the "sudo bash" in the beginning btw.

Related

How to Install mysql-server via dockerfile

Does anybody knows how to install mysql-server via dockerfile? I have written a Dockerfile, but the build ends with an error: /bin/sh: 1: /usr/bin/mysqld: not found
USER root
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.7
# Remove pre-installed database
RUN rm -rf /var/lib/mysql/*
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/"/etc/mysql/my.cnf
ENV DB_USER example
ENV DB_PASSWORD example
ENV DB_NAME example
ENV VOLUME_HOME "/var/lib/mysql"
EXPOSE 3306
RUN cp /etc/mysql/my.cnf /usr/share/mysql/my-default.cnf
RUN /usr/bin/mysqld && sleep 5 && \
mysql -uroot -e "CREATE USER '${DB_USER}'#'%' IDENTIFIED BY '${DB_PASSWORD}'" && \
mysql -uroot -e "GRANT ALL PRIVILEGES ON *.* TO '${DB_USER}'#'%' WITH GRANT OPTION" &&\
mysql -uroot -e "CREATE DATABASE ${DB_NAME}" && \
mysqladmin -uroot shutdown
For an ubuntu:16.04 base image, mysqld is found in /usr/sbin, not /usr/bin
If you can add a step RUN which mysqld before your final RUN command that will show you where the mysqld executable is found. It may vary depending on which base image/distro you're using.
You can also use RUN mysqld ... without a full path, if the file is in your $PATH
You may also need to update your RUN sed line as below, adding spaces around the quoted string:
RUN sed -i -e "s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
Otherwise, you may see the following error:
The command '/bin/sh -c sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/"/etc/mysql/my.cnf' returned a non-zero code: 1

Docker: MySQL refuses host IP unless I restart it manually

Here is my Dockerfile:
FROM debian:jessie-backports
RUN apt-get update --yes && apt-get upgrade --yes
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
git \
curl \
build-essential\
libssl-dev\
ca-certificates\
mysql-server\
redis-server\
elasticsearch
USER root
ENV HOME /root
# MYSQL SETUP
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/#bind-address = 0.0.0.0/" /etc/mysql/my.cnf
RUN sed -i -e"s/^#max_connections\s*=\s*100/max_connections = 200/" /etc/mysql/my.cnf
RUN echo "\n[mysqld]\nskip-grant-tables\n" >> /etc/mysql/my.cnf
VOLUME ["/var/lib/mysql", "/var/log/mysql"]
EXPOSE 3306
# REDIS SETUP
RUN echo "daemonize yes\nbind 0.0.0.0" >> /etc/redis/redis-serve.conf
RUN sed 's/^daemonize no/daemonize yes/' -i /etc/redis/redis.conf \
&& sed 's/^bind 127.0.0.1/bind 0.0.0.0/' -i /etc/redis/redis.conf \
&& sed 's/^# unixsocket /unixsocket /' -i /etc/redis/redis.conf \
&& sed 's/^# unixsocketperm 755/unixsocketperm 777/' -i /etc/redis/redis.conf \
&& sed '/^logfile/d' -i /etc/redis/redis.conf
VOLUME ["/var/lib/redis", "/var/log/redis"]
EXPOSE 6379 6380
# ELASTICSEARCH SETUP
RUN sed 's/^#START_DAEMON=true/START_DAEMON=true/' -i /etc/default/elasticsearch
VOLUME ["/opt/elasticsearch/data", "/opt/elasticsearch/logs"]
EXPOSE 9200 9300
ADD docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod 777 /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
The entrypoint:
#!/bin/bash
/etc/init.d/mysql start
/etc/init.d/redis-server start
/etc/init.d/elasticsearch start
/bin/bash
And the command I'm running to start the container:
docker run -i -t -p 3306:3306 -p 6379:6379 -p 9200:9200 -p 9300:9300 assethost
I want to connect to the MySQL server running in the container from the host, but this is what happens when I try to connect:
ERROR 1130 (HY000): Host '192.168.99.1' is not allowed to connect to this MySQL server
Both Redis and Elasticsearch expose themselves just fine. MySQL will only begin accepting external hosts if I restart it manually from the Bash shell that comes up from the entrypoint when I run the container. I have to run /etc/init.d/mysql restart. Only then, can I successfully connect from the host using the MySQL client.
Note that I am using docker-machine, so I have to give the MySQL client the host IP in order to connect.
How can I be able to connect from the host to the MySQL instance running in the container without having to manually restart MySQL?
Docker version 1.12.1, build 23cf638
I figured out a way to get the skip-grant-tables option to be applied on start.
Workaround
In docker-entrypoint.sh, I changed
/etc/init.d/mysql start to
HOME=/etc/mysql /usr/bin/mysqld_safe > /dev/null 2>&1 &
Other changes
Since there was already a [mysqld] section in my.cnf, I changed the following line in Dockerfile
RUN echo "\n[mysqld]\nskip-grant-tables\n" >> /etc/mysql/my.cnf
to
RUN sed -i '/\[mysqld\]/ a skip-grant-tables\nskip-name-resolve' /etc/mysql/my.cnf
I added skip-name-resolve so mysql doesn't attempt to resolve names for local ip addresses.
Caveats
The workaround no longer works when /etc/init.d/mysql restart is executed, but /etc/init.d/mysql stop; /etc/init.d/mysql start preserved the 'skip-grant-tables' option.
I'm guessing MySQL has already started after your installation and running sed to change configuration won't take effect. Entry point you have has mysql start but it is probably seeing that MySQL is already running and not doing anything. Try changing that command to mysql restart, so your entry point bash script becomes:
#!/bin/bash
/etc/init.d/mysql restart
/etc/init.d/redis-server start
/etc/init.d/elasticsearch start
/bin/bash
You're using skip-grant-tables flag to login without password. I think due to some reasons of /etc/init.d/mysql script, the flag is ignored on the first start, making the connection from the root#192.168.99.1 rejected. I believe Docker has nothing to do with this issue.
To answer your question, you don't have to restart by hand, you can change your entrypoint to either:
#!/bin/bash
/etc/init.d/mysql start
# Wait for mysql to be fully up, then restart (e.g., 5 seconds)
sleep 5
/etc/init.d/mysql restart
/etc/init.d/redis-server start
/etc/init.d/elasticsearch start
/bin/bash
Or, don't use /etc/init.d/mysql start, instead invoke mysqld directly
#!/bin/bash
mysqld &
/etc/init.d/redis-server start
/etc/init.d/elasticsearch start
/bin/bash
In addition, although creating one more [mysqld] group in my.cnf is not an issue, you should append new configuration to the existing group:
RUN sed -i '/\[mysqld\]/a skip-grant-tables' /etc/mysql/my.cnf
Refer the doc for several ways to start/stop Mysql server

Is it possible to upgrade the MySQL in MAMP to MySQL 5.7?

Is it possible to upgrade the MAMP MySQL library to 5.7?
I am currently running 5.6 (which I upgraded to using MAMP’s upgrade script
)
Or would I need to install MySQL natively on my system? (macOS 10.11 El Capitan)
If someone could point me in the right direction… Thanks!
Upgrade MAMP to Mysql 5.7
#!/bin/sh
wget http://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.10-osx10.10-x86_64.tar.gz
tar xfvz mysql-5.7*
echo "stopping mamp"
sudo /Applications/MAMP/bin/stop.sh
sudo killall httpd mysqld
echo "creating backup"
sudo rsync -arv --progress /Applications/MAMP ~/Desktop/MAMP-Backup
echo "copy bin"
sudo rsync -arv --progress mysql-5.7.*/bin/* /Applications/MAMP/Library/bin/ --exclude=mysqld_multi --exclude=mysqld_safe
echo "copy share"
sudo rsync -arv --progress mysql-5.7.*/share/* /Applications/MAMP/Library/share/
echo "fixing access (workaround)"
sudo chmod -R o+rw /Applications/MAMP/db/mysql/
sudo chmod -R o+rw /Applications/MAMP/tmp/mysql/
sudo chmod -R o+rw "/Library/Application Support/appsolute/MAMP PRO/db/mysql/"
echo "starting mamp"
sudo /Applications/MAMP/bin/start.sh
echo "migrate to new version"
/Applications/MAMP/Library/bin/mysql_upgrade -u root --password=root -h 127.0.0.1
UPDATE: Version 5.0 of MAMP now includes MySQL 5.7 already in the installer! Just upgrade your core MAMP setup and you are set to go with MySQL instead of having to jump through technical hoops like this.
Leaving answer below as a reference for anyone who needs it.
While I have read this answers and comments here—as well as some similar linked tutorials on GitHub and such—there were a few things that confused me in some of the tutorials. Such as instructions to set chmod -O o+rw and even a comment about creating a symbolic link to /tmp/mysql.sock; why do that when MAMP out of the box should be self contained and not require such changes? So here are the instructions I have put together based on my experience getting MySQL upgraded for MAMP 4.4.1 on mac OS 10.3.4 (High Sierra).
First, get a copy of the macOS binaries for MySQL 5.7; note that as of me posting this answer MySQL 5.7.22 is the current version so adjust this URL to whatever new version you might want to use:
curl -OL https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.22-macos10.13-x86_64.tar.gz
Decompress it like this:
tar xfvz mysql-5.7*
Copy the bin/ and share/ stuff into MAMP via Rsync like this:
sudo rsync -arv --progress mysql-5.7.*/bin/* /Applications/MAMP/Library/bin/ --exclude=mysqld_multi --exclude=mysqld_safe
sudo rsync -arv --progress mysql-5.7.*/share/* /Applications/MAMP/Library/share/
Copy your existing MySQL 5.6 database directory like this; just note that the mysql56 directory is temporarily needed during the upgrade but can be discarded after the rest of the MySQL 5.7 upgrade is done:
sudo cp -r /Applications/MAMP/db/mysql56 /Applications/MAMP/db/mysql57
Once that is done, get rid of MySQL database specific binaries like this for upgrade:
sudo rm -rf /Applications/MAMP/db/mysql57/mysql/innodb_*
sudo rm -rf /Applications/MAMP/db/mysql57/mysql/slave_*
And instead of changing permissions to o+rw just change the owner of the DB directory to your current user; this matches how MAMP installs this stuff:
sudo chown -R ${USER}:admin /Applications/MAMP/db/mysql57
Now run this Sed command to adjust the mysqld_safe script to point to the new MySQL 5.7 path; you could probably just open this file up in a text editor and change all instances of mysql56 to mysql57as well:
sed -i.bak 's/mysql56/mysql57/g' /Applications/MAMP/Library/bin/mysqld_safe
Finally, if you use MAMP and set a my.cnf file, that should be set in /Applications/MAMP/conf/my.cnf… But by doing this upgrade, the default search path of the my.cnf in MAMP will be /usr/local/mysql/etc/ instead of the expected /Applications/MAMP/conf/ since that is where the new binary expects it to be set. Clearly we’re not going to recompile MySQL at this point so the cleanest/simplest thing to do to make your MAMP setup truly portable again is to change this line in the startMysql.sh from this:
/Applications/MAMP/Library/bin/mysqld_safe --port=8889 --socket=/Applications/MAMP/tmp/mysql/mysql.sock --pid-file=/Applications/MAMP/tmp/mysql/mysql.pid --log-error=/Applications/MAMP/logs/mysql_error_log &
To this; note we are adding the --defaults-extra-file= option before all the otgers:
/Applications/MAMP/Library/bin/mysqld_safe --defaults-extra-file=/Applications/MAMP/conf/my.cnf --port=8889 --socket=/Applications/MAMP/tmp/mysql/mysql.sock --pid-file=/Applications/MAMP/tmp/mysql/mysql.pid --log-error=/Applications/MAMP/logs/mysql_error_log &
With all of that command line work done, launch MAMP via the application, start the MySQL and Apache servers and then drop back into the command like to run this command to upgrade the databases:
/Applications/MAMP/Library/bin/mysql_upgrade --user=root --password=root --port=3306 --socket=/Applications/MAMP/tmp/mysql/mysql.sock --force
And finally run this command to get the mysql.sock properly set for MAMP path instead of that /tmp/mysql.sock path:
/Applications/MAMP/Library/bin/mysql_config_editor --verbose set --socket=/Applications/MAMP/tmp/mysql/mysql.sock
When this is all done, and you have confirmed MySQL is running as expected, just toss the old MySQL 5.6 directory like this:
sudo rm -rf /Applications/MAMP/db/mysql56
With all that done you should all be set to cleanly use MySQL 5.7 under MAMP 4.4.1.
I encountered problems upgrading to MySQL 5.7.22 described in Giacomo1968’s answer.
The updated procedure worked well on El Capitan with MySQL 5.7.18.
I have written an updated bash script for this procedure:
#!/bin/sh
curl -OL https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.18-macos10.12-x86_64.tar.gz
tar xfvz mysql-5.7*
echo "Stopping MAMP"
sudo /Applications/MAMP/bin/stop.sh
sudo killall httpd mysqld
echo "Copy Bin"
sudo rsync -arv --progress mysql-5.7.*/bin/* /Applications/MAMP/Library/bin/ --exclude=mysqld_multi --exclude=mysqld_safe
echo "Copy Share"
sudo rsync -arv --progress mysql-5.7.*/share/* /Applications/MAMP/Library/share/
echo "Building Mysql 5.7 Folder"
sudo cp -r /Applications/MAMP/db/mysql56 /Applications/MAMP/db/mysql57
sudo rm -rf /Applications/MAMP/db/mysql57/mysql/innodb_*
sudo rm -rf /Applications/MAMP/db/mysql57/mysql/slave_*
sudo chown -R ${USER}:admin /Applications/MAMP/db/mysql57
sed -i.bak 's/mysql56/mysql57/g' /Applications/MAMP/Library/bin/mysqld_safe
echo "Finally, if you use MAMP and set a my.cnf file, that should be set in /Applications/MAMP/conf/my.cnf… But by doing this upgrade, the default search path of the my.cnf in MAMP will be /usr/local/mysql/etc/ instead of the expected /Applications/MAMP/conf/ since that is where the new binary expects it to be set. Clearly we’re not going to recompile MySQL at this point so the cleanest/simplest thing to do to make your MAMP setup truly portable again is to change this line in the startMysql.sh from this:
/Applications/MAMP/Library/bin/mysqld_safe --port=8889 --socket=/Applications/MAMP/tmp/mysql/mysql.sock --pid-file=/Applications/MAMP/tmp/mysql/mysql.pid --log-error=/Applications/MAMP/logs/mysql_error_log &
To this; note we are adding the --defaults-extra-file= option before all the otgers:
/Applications/MAMP/Library/bin/mysqld_safe --defaults-extra-file=/Applications/MAMP/conf/my.cnf --port=8889 --socket=/Applications/MAMP/tmp/mysql/mysql.sock --pid-file=/Applications/MAMP/tmp/mysql/mysql.pid --log-error=/Applications/MAMP/logs/mysql_error_log &"
read -p "With all of that command line work done, launch MAMP via the application, start the MySQL and Apache servers."
read -p "Press [Enter] key to start migration..."
echo "Starting MySQL"
/Applications/MAMP/Library/bin/mysql_upgrade --user=root --password=root --port=3306 --socket=/Applications/MAMP/tmp/mysql/mysql.sock --force
echo "Migrate, finaly, to new version"
/Applications/MAMP/Library/bin/mysql_config_editor --verbose set --socket=/Applications/MAMP/tmp/mysql/mysql.sock

Can't log into MariaDB/MySQL after install (RedHat) using mysql -u root -p

I have run a binary install of MariaDB, i will provide the following commands that i used.
(Forgive me I have a fairly basic level of mysql so I have annotated with my understanding of what the commands do).
Added group called mysql
shell> groupadd mysql
shell> useradd -r -g mysql mysql
shell> cd /usr/local
Untars the mariaDB binaries into which directory you choose.
shell> tar zxvf /usr/local/mysql/mysql-VERSION-OS.tar.gz
Created a symbolic link.
shell> ln -s /usr/local/mysql/mysql-VERSION-OS mysql
shell> cd mysql
Recursively changes ownership to the user/group.
shell> chown -R mysql .
shell> chgrp -R mysql .
Runs the mysql install db.
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data
Makes a copy of the my.cnf file to put into the etc folder
shell> cp support-files/my-medium.cnf /etc/my.cnf
shell> bin/mysqld_safe --user=mysql &
Makes a copy of the server file to the init.d file which allows it to start automatically
shell> cp support-files/mysql.server /etc/init.d/mysql.server
Running my my_secure_installation script
./mysql_secure_installation --basedir=/usr/local/mysql/mariadb-5.5.34-linux-x86_64
I then closed the terminal and reopened, did:
ps -ef | grep mysql
to check the mysqld server was running (it was).
So I have done the above steps, I try and enter:
mysql -u root -p
and I receive the error
bash: mysql: command not found.
Any ideas why I cannot access it? Thanks in advance.

Can't delete mysql database, table or even alter table inside docker

I want to have a mysql database with some basic dataset.
I create mysql docker image using this https://index.docker.io/u/brice/mysql/ Dockerfile, but delete VOLUME ["/var/lib/mysql", "/var/log/mysql"] line, so the Dockerfile looks like:
FROM ubuntu:12.10
MAINTAINER Brandon Rice <brice84#gmail.com>
RUN dpkg-divert --local --rename --add /sbin/initctl
RUN ln -s /bin/true /sbin/initctl
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get -y install mysql-server
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
RUN /usr/bin/mysqld_safe & \
sleep 10s && \
mysql < create_my_db.sql && \
mysql -e "GRANT ALL ON *.* to 'root'#'%'; FLUSH PRIVILEGES"
EXPOSE 3306
CMD ["mysqld_safe"]
After that I build image such as:
docker build -t my_db_mysql .
Everything is ok while i append data, but when i want to delete db, for example:
FROM my_db_mysql
RUN /usr/bin/mysqld_safe & \
sleep 10s && \
mysql -e "DROP DATABASE my_db;"
EXPOSE 3306
CMD ["mysqld_safe"]
I obtain next error:
ERROR 6 (HY000) at line 1: Error on delete of './my_db//db.opt' (Errcode: 1)
It is appears not only when I want to build image, but even when I exec: mysql -u user -p -e "DROP DATABASE my_db;"
How to solve this?
Thanks
Update: Also I tried to run docker with different filesystem, e.g -s vfs or -s devicemapper, but nothing changed.
When I build image with VOLUME ["/var/lib/mysql", "/var/log/mysql"] - everything works properly, but i can't commit this changes.
Update: Seems I resolve this issue. Problem was in host machine with ubuntu 12.04. Issue disappeared when I update ubuntu to 13.10. Thank a lot!