MySQL Docker Image: initialized with env variable of K8S Secret - mysql

Question

When i tried to deploy with K8S using env variable for Root Password (MYSQL_ROOT_PASSWORD , a variable used in docker-entrypoint.sh) such as
kind: Deployment
#omitted...
spec:
containers:
#omitted...
env:
- name: MYSQL_ROOT_PASSWORD
value: my-secret-passw
it seems that the deployment with kubectl apply -k is working. Of course, this insecure deployment. MySql initialization works and everything is as expected.
So i tried to create a secret and retrieving the value from that secret, such that
kind: Deployment
#omitted...
spec:
containers:
#omitted...
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
key: MYSQL_ROOT_PASSWORD
name: mysql-root-password
mysql initialization is not working. And i have no idea what the heck is going wrong. The kubectl logs or stdout from the container is the following (bear in mind that using literal root pass didn't return result like this):
[MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock'
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
[Note] [Entrypoint]: /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/sql-script.sql
mysql: [ERROR] unknown option '-- "'.
When checking the docker image source code, it seems there is no string about "unknown option" (see here the github and i am using mysql 8.0. It seems it's coming out of mysql itself.
So why is it that when using literal value on k8S env variable (which is not safe!) working but not with reading from secret?

Answer 1

Seems like version 8.0 is broken or missing something because it cannot see the password.
I'm not an expert in this field so maybe someone else can add some more details.
2020-02-07T16:09:06.648827Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2020-02-07T16:09:06.649084Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.19) initializing of server in progress as process 44
2020-02-07T16:09:10.821145Z 5 [Warning] [MY-010453] [Server] root#localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2020-02-07 16:09:14+00:00 [Note] [Entrypoint]: Database files initialized
2020-02-07 16:09:14+00:00 [Note] [Entrypoint]: Starting temporary server
2020-02-07T16:09:14.823453Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2020-02-07T16:09:14.823582Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.19) starting as process 94
2020-02-07T16:09:15.631008Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2020-02-07T16:09:15.635649Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2020-02-07T16:09:15.655790Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.19' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
2020-02-07 16:09:15+00:00 [Note] [Entrypoint]: Temporary server started.
2020-02-07T16:09:15.740600Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock'
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2020-02-07 16:09:19+00:00 [Note] [Entrypoint]: Stopping temporary server
2020-02-07T16:09:19.476870Z 10 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.19).
I've tested the image 5.6 from official mysql repository and it's seems to work perfectly fine.
I've run whole MySQL + WordPress deployment using the Tutorial on kubernetes.io which I do recommend.
I've created the secret manually, using:
kubectl create secret generic mysql-pass --from-literal=password=test
Looks like there is an issue open for problem similar to this "Unable to start server" on Amazon Linux AMI (works with 5.7) #628.

Answer 2

After sometime, i realized that the problem is not with the Secret of K8S. I accidentally changed my password into string of chars without Special characters, and that solved it! From the post, you see that the character -- was read as an input of the next process (dang it, i am giving away my pass! nah, changed it already).
So anyone of you with the same problem (with mysql), please consider choosing a secret with no-special-characters password. Long password with combination of numbers/case-sensitive could be strong enough, especially if you paper and pencil!