Docker container running supervisor shuts down too early - mysql

Question

I am running a docker container with multiple processes (to achieve a LAMP environment) by using supervisor as described here. Everything works fine but when I stop the container the MySQL process doesn't get properly terminated even though I've set up supervisor's pidproxy in the supervisor config
[program:mysql]
command=/usr/bin/pidproxy /run/mysqld/mysqld.pid /bin/sh -c "exec /usr/bin/mysqld_safe"
When I access the container through nsenter and restart the MySQL daemon with
supervisorctl restart mysql
the shutdown completes without throwing an error next time it comes up. So I think supervisor is configured correctly. To me it seems docker (I am running 1.2.0) terminates the container just a bit early while mysqld is still shutting down.
Edit
I was able to debug some more details by running supervisord with the -e debug switch.
Shutdown with supervisorctl restart mysql via nsenter:
DEBG fd 17 closed, stopped monitoring <POutputDispatcher at 39322256 for <Subprocess at 38373280 with name mysql in state RUNNING> (stderr)>
DEBG fd 14 closed, stopped monitoring <POutputDispatcher at 39324128 for <Subprocess at 38373280 with name mysql in state RUNNING> (stdout)>
DEBG killing mysql (pid 1128) with signal SIGTERM
INFO stopped: mysql (exit status 0)
DEBG received SIGCLD indicating a child quit
CRIT reaped unknown pid 1129)
DEBG received SIGCLD indicating a child quit
And externally via docker restart container_name:
DEBG fd 17 closed, stopped monitoring <POutputDispatcher at 39290136 for <Subprocess at 38373280 with name mysql in state RUNNING> (stderr)>
DEBG fd 14 closed, stopped monitoring <POutputDispatcher at 39290424 for <Subprocess at 38373280 with name mysql in state RUNNING> (stdout)>
DEBG killing mysql (pid 7871) with signal SIGTERM
INFO stopped: mysql (exit status 0)
DEBG received SIGCLD indicating a child quit
This is the process structure before the first attempt:
1128 S 0:00 /usr/bin/python /usr/bin/pidproxy /run/mysqld/mysqld.pid /usr/bin/mysqld_safe
1129 S 0:00 \_ /bin/sh /usr/bin/mysqld_safe
1463 Sl 0:00 \_ /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-erro
So while pid 1463 gets properly reaped as it's in /run/mysqld/mysqld.pid, pid 1129 causes the trouble as docker shuts down the container before it's gone. Is it a bug in supervisor's pidproxy or can it be fixed by a different configuration?

Answer 1

I had a similar problem and I fixed it by not using mysql_safe, but rather starting mysql directly. I did this by seeing what command mysql_safe generated in the ps aux overview.
Maybe it fails because mysql_safe doesn't create a proper parent -> child process relationship?
Anyway my config would look something like this if I use your mysql process as a template:
[program:mysql]
command=/usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-erro

Answer 2

I believe that Docker will only wait for ten seconds before killing a container it's trying to stop (or restart). This is so that the action requested is ultimately carried out, and it doesn't just hang. You can configure the time period using the "--time" parameter on "docker stop" or "docker restart".