MySQL Docker file : sed: couldn't open temporary file - mysql

I am trying to create new MySQL image and deploying in Kubernetes.
FROM oraclelinux:7-slim
USER root
ARG MYSQL_SERVER_PACKAGE=mysql-community-server-minimal-8.0.19
ARG MYSQL_SHELL_PACKAGE=mysql-shell-8.0.19
# Install server
RUN yum install -y https://repo.mysql.com/mysql-community-minimal-release-el7.rpm \
https://repo.mysql.com/mysql-community-release-el7.rpm \
&& yum-config-manager --enable mysql80-server-minimal \
&& yum install -y \
$MYSQL_SERVER_PACKAGE \
$MYSQL_SHELL_PACKAGE \
libpwquality \
&& yum clean all \
&& mkdir /docker-entrypoint-initdb.d
VOLUME /var/lib/mysql
COPY docker-entrypoint.sh /entrypoint.sh
COPY healthcheck.sh /healthcheck.sh
ENTRYPOINT ["/entrypoint.sh"]
HEALTHCHECK CMD /healthcheck.sh
EXPOSE 3306 33060
RUN chmod +rwx /entrypoint.sh
RUN chmod +rwx /healthcheck.sh
RUN groupadd -r mysql && useradd -r -g mysql mysql
EXPOSE 3306
CMD ["mysqld"]
It's working fine in the container. But throwing error when I deployed in Kubernetes like below:
How can I understand this issue?
ADDED
docker-entrypoint.sh:
if [ -n "$MYSQL_LOG_CONSOLE" ] || [ -n "console" ]; then
# Don't touch bind-mounted config files
if ! cat /proc/1/mounts | grep "/etc/my.cnf"; then
sed -i 's/^log-error=/#&/' /etc/my.cnf
fi
fi
P.S : I have added content of the file.

The problem is related with sed's in-place editing implementation. When you edit a file using the -i or --in-place option, the edition doesn't actually happen in-place. sed saves the changes into a temporary file and then uses it to replace the original one.
It happens that you don't have write permission to /etc directory, where sed is trying to create its temporary file.
As suggested in comments most probably the command is run by user mysql. For sure it is not run as root as it has enough privileges to be able to write to /etc:
bash-4.2# ls -ld /etc
drwxr-xr-x 1 root root 4096 Mar 27 15:04 /etc
As you can see others don't have write permission. Changing permissions or owner of /etc directory itself is a really bad idea and I won't advise you to run this command as root user either.
The simplest solution is to give up on using --in-place option, save the result in a directory such as /tmp, to which everyone has access:
bash-4.2# ls -ld /tmp
drwxrwxrwt 1 root root 4096 Mar 27 16:39 /tmp
and after that replace the content of the original file with the content of the temporary one.
Your command may look like this:
sed 's/^log-error=/#&/' /etc/my.cnf > /tmp/my.cnf && cat /tmp/my.cnf > /etc/my.cnf
One important caveat:
You need to make sure you have write permission on /etc/my.cnf file. As you can see below, by default you don't have such permission either, so the error will occur later, when the command will try to write to the original config file.
bash-4.2# ls -l /etc/my.cnf
-rw-r--r-- 1 root root 1239 Mar 27 15:04 /etc/my.cnf
You need to modify it in your Dockerfile either by making it availeble for edit by everyone:
RUN chmod 666 /etc/my.cnf
or better option:
RUN chown mysql /etc/my.cnf
to change its owner to mysql, if this is the user that executes the entrypoint.sh script.
Please let me know if it helps.

Related

Can I change the global port number without my.cnf file?

I made the following mysql container in the docker.
docker run -p 3307:3307 --name todolist -e MYSQL_ROOT_PASSWORD=1234 -e MYSQL_DATABASE=todolist -e MYSQL_USER=allssu -e MYSQL_PASSWORD=1234 -d mysql
However, I confirmed that the port of mysql on the docker is still 3306. So I tried to modify the port number in the my.cnf file in the /etc/mysql path. But, only the conf.d file exists in the /etc/mysql folder, and there is no my.cnf file.
To make matters worse, an error occurs in my bash that the apt-get command does not work.
bash-4.4# apt-get update
bash: apt-get: command not found
bash-4.4# apt-get install vim
bash: apt-get: command not found
First, I want to find the invisible my.cnf file in the etc/mysql folder. Also I want to set the port of this container(name:todolist) to 3307, is there any way?
And, I want to use the vi command in bash.
I would appreciate it if you could consider this matter together.
I tried the following and still couldn't find my.cnf.
bash-4.4# mysqld --verbose --help | grep -A 1 'Default options'
Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf /usr/etc/my.cnf ~/.my.cnf
result:
bash-4.4# pwd
/etc/mysql
bash-4.4# ls
conf.d

Mariadb multi stage container with an existing database

I need to create a container with an existing database and operate some actions on it before distribute it.
In production, I've got some really big databases:
[root]# ls /home/db-backup/test/prepare/26_06_2020/full/
ibdata1
ib_logfile0
ib_logfile1
ibtmp1
mysql
performance_schema
my_existing_database1
my_existing_database2
my_existing_database3
I just need to publish a container with my_existing_database1 for my team.
I tried many Dockerfile but I can't find a way do to it and I don't understand why.
Here a simplified Dockerfile :
FROM mariadb:latest as builder
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
# for easier debug, i will remove that in prod
RUN sed -i '/\[mysqld\]/a plugin-load-add = auth_socket.so' /etc/mysql/my.cnf
WORKDIR /initialized-db
COPY ibdata1 .
COPY mysql ./mysql
COPY performance_schema ./performance_schema
COPY my_existing_database1 ./my_existing_database1
COPY db-init.sh /docker-entrypoint-initdb.d/
RUN chown mysql:mysql . \
&& chmod 660 ibdata1 \
&& chmod +x /docker-entrypoint-initdb.d/db-init.sh
RUN ["/usr/local/bin/docker-entrypoint.sh", "mysqld", "--datadir", "/initialized-db", "--aria-log-dir-path", "/initialized-db"]
# No file named test
RUN ls /initialized-db/
FROM mariadb:latest
COPY --from=builder /initialized-db /var/lib/mysql
As you can see, I try to execute my script db-init.sh. Simplified version :
#!/bin/bash
set -e -x
mysql -u root -e "CREATE USER 'test'#'%' IDENTIFIED BY 'test';"
touch /initialized-db/test
Unfortunetly, my script is not executed as the file test is not created.
I try to bypass the /usr/local/bin/docker-entrypoint.sh with my own script (copy/paste of the file with some edit) but it's not working either (user and file test not created).
Can you help me on this one please?
You should preferably mount a directory for the data and distribute it then extra.
If you must distibute it with the container itself, you should be able to edit the config of that database to use a custom data directory and initialize your databases in it with RUN and COPY.

How to redirect mysql log to /dev/stdout inside container?

I am trying to write my own mariadb-alpine docker image. Everything works file but while I am trying to collect the mariadb logs I am getting nothing. I tried to follow a lot of related issue like this and tried those but in vain.
FROM alpine:edge
COPY my.cnf /etc/mysql/my.cnf
RUN set -ex \
&& apk add mariadb mariadb-client shadow \
&& ln -snf /usr/lib/mariadb /usr/lib/mysql \
&& mysql_install_db --user=mysql --skip-name-resolve --auth-root-authentication-method=socket --auth-root-socket-user=root --force --rpm --skip-test-db \
&& usermod -a -G tty mysql \
&& ln -sf /dev/stdout /var/log/mysqld.err \
&& chown -h mysql:mysql /var/log/mysqld.err
CMD ["mysqld_safe"]
EXPOSE 3306
Is it required to mysqld take pid=1 to work stdout ? In my case it is some like below.
# ps aux
PID USER TIME COMMAND
1 root 0:00 {mysqld_safe} /bin/sh /usr/bin/mysqld_safe
134 mysql 0:00 /usr/bin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mariadb/plugin --user=mysql --log-error=/var/log/mysqld.err --pid-file=49ea99ae9348.p
166 root 0:00 sh
171 root 0:00 ps aux
You're likely running into these issues:
https://github.com/moby/moby/issues/31243
https://github.com/moby/moby/issues/31106
Something with alpine breaks the ability to access /dev/stdout when you change user accounts. The workaround I've used involves:
Running the container with a tty
Adding the user inside the container to the tty group
Starting the command with a gosu/exec to replace the pid 1 shell script with your app
I'm not sure if the last part was required, and you may not have access to do this with the mysql command. You're already doing the second item. That just leaves the first item that you can implement with:
docker run -t your_image
or in a compose file:
services:
mysql:
image: your_image
tty: true
....
The only other option is to run your application directly as mysql instead of starting it as root with user: mysql in the compose file, but that may not be supported by mysql itself.
If none of those work, the option used by the official image is to pick a debian base image instead of the alpine image. You can see their Dockerfile here:
https://github.com/docker-library/mysql/blob/696fc899126ae00771b5d87bdadae836e704ae7d/8.0/Dockerfile

How to create a initialised MySQL Docker image?

Using Docker image mysql:5.7.21 I want to create a new image based of it that will already have its database initialised. (I want to use this for my acceptation environment).
I know about the /docker-entrypoint-initdb.d and I could do the fallowing:
FROM mysql:5.7.21
COPY ./init /docker-entrypoint-initdb.d
But the problem is that it will only copy the tar/sql files, but it will not execute until the first time it's run. Another disadvantage is also the image size. If you a copying for example 200MB of data inside the container then those files will stay in the image (as a layer).
So I was wondering if there is a combination I haven't thought of with multi-stage or the new --squash flag (that would enable, adding the files, executing the files, removing the files).
In my search for this problem I also found about the --datadir flag. Not sure how this could help.
edit:
So far I have the fallowing:
FROM mysql:5.7.21 as builder
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
ENV MYSQL_DATABASE example
COPY ./init /docker-entrypoint-initdb.d
RUN head -n-2 < /usr/local/bin/docker-entrypoint.sh > /usr/local/bin/docker-entrypoint.sh \
&& docker-entrypoint.sh mysqld \
&& /etc/init.d/mysql start \
&& mysql -uroot -e 'FLUSH TABLES;' \
&& mysql -uroot -e 'show tables;' mysql \
&& mysql -uroot -e 'show tables;' example \
&& /etc/init.d/mysql stop \
# This shows the files!
&& ls -la /var/lib/mysql
# This shows no files?!!
RUN ls -la /var/lib/mysql
FROM mysql:5.7.21
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
ENV MYSQL_DATABASE example
COPY --from=builder /var/lib/mysql /var/lib/mysql
RUN ls -la
RUN ls -la /var/lib/mysql \
&& /etc/init.d/mysql start \
&& mysql -uroot -e 'show tables;' mysql \
&& mysql -uroot -e 'show tables;' example
See the above inline comments... something really strange is happening? So in the same RUN when I perform an ls -la on /var/lib/mysql I can see the files. But in a new layer (new RUN) it's empty :S
So I found the solution!!! My problem lied in the fact that /var/lib/mysql is an volume: https://github.com/docker-library/mysql/blob/ad625c64a06e16683e997e5a0147508d115f4989/5.7/Dockerfile#L71
A working multistage Dockerfile:
FROM mysql:5.7.21 as builder
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
ENV MYSQL_DATABASE example
COPY ./init /docker-entrypoint-initdb.d
RUN head -n-2 < /usr/local/bin/docker-entrypoint.sh > /usr/local/bin/docker-entrypoint.sh
RUN mkdir -p /var/lib/mysql_tmp
RUN docker-entrypoint.sh mysqld --datadir /var/lib/mysql_tmp
FROM mysql:5.7.21
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
ENV MYSQL_DATABASE example
COPY --from=builder /var/lib/mysql_tmp /var/lib/mysql_tmp
CMD ["mysqld", "--datadir", "/var/lib/mysql_tmp"]
So what we do here is make sure to remove the last 2 lines (could be changed to remove just the last line) because it contains exec "$#" which basically means exec msqld. That's the MySQL deamon and so it would get stuck there.
Further I changed the datadir from var/lib/mysql (which is a volume) to var/lib/mysql_tmp (for lack of a better name).
In the last stage I copy var/lib/mysql_tmp and make sure mysqld uses that dir.
Now I can bake (for me readonly) db's for my acceptation env.
If you check the Dockerfile for the mysql:5.7.21 image (https://hub.docker.com/r/library/mysql/), you will see that the entrypoint is set to ["docker-entrypoint.sh"], so this is the first command that is executed when the image is run.
If you want to execute it during image creation (thus, making image size huge, but ok) you just have to put this in your Dockerfile:
RUN docker-entrypoint.sh /bin/true
This entrypoint will execute the scripts found in /docker-entrypoint-initdb.d but, as you well noted, the scripts also take up space. In order to discard the files, you can use multistage build to just copy the raw database.
Check again the Dockerfile to see the only volume in the image is /var/lib/mysql. This will supposedly be the directory containing all the information on the database, and it should be what you copy. The result is something like this:
FROM mysql:5.7.21 as base
COPY ./init /docker-entrypoint-initdb.d
RUN docker-entrypoint.sh /bin/true
FROM mysql:5.7.21
COPY --from=base /var/lib/mysql /var/lib/mysql
All this said, I wouldn't recommend to create "initialized" images, as it is somewhat against the spirit of the use of containers, but that's another question ;)

Reinstall MySQL (mariadb) in Debian Stretch

I removed mysql completely including configuration files and when I try to reinstall, there is no mysql.socket (I searched in /var/lib/mysql/ and there's nothing.
Typically you will see mysql.sock file when you start the mysql daemon process. This is create the sock file.
But you can also search and see whether it has created somewhere.
netstat -ln | grep -o -m 1 -E '\S*mysqld?\.sock'
or
you can check with this command:
% mysqladmin variables
If you want to create the file, you can do this
mkdir /var/mysql
ln -s /tmp/mysql.sock /var/mysql/mysql.sock
This thread explains things in more details.