I am working on an existing Dockerfile that I have been asked to modify as less as possible. The docker image is based on a CentOS Linux image and is supposed to contain a MySQL service.
I want to enable the verbose logging for all the queries (i.e. general_log and general_log_file variables on the /etc/my.cnf file).
The MySQL service needs to be run in the mysqld_safe mode and I've checked that the configuration lines I am adding (see below the printf) are after the [mysqld_safe] line in the /etc/my.cnf file, so I am assuming this setting should be fine.
What I've done so far is adding to the Dockerfile the following statements:
RUN groupadd -r mysql && useradd -r -g mysql mysql
# [...] Lots of Mysql stuff regarding importing DBs etc.
# Adding some more configuration details to the database service
RUN printf '\n%s\n%s\n%s\n\n' '# Set General Log to log all the queries' 'general_log=1' 'general_log_file=/var/log/mysql_general.log' >> /etc/my.cnf
# Getting the new log file prepared to get written by the MySQL service
RUN touch /var/log/mysql_general.log
RUN chown mysql.mysql /var/log/mysql_general.log
# MySQL Port
EXPOSE 3306
ENTRYPOINT ["mysqld_safe"]
After building the docker image and running the docker container I see this in the /var/log folder:
-rw-r-----. 1 mysql mysql 5108 Nov 20 17:07 mysqld.log
-rw-r--r--. 1 mysql mysql 3880 Nov 20 17:44 mysql_general.log
If I grep the mysqld.log for keywords like ERROR or general I can not find anything interesting. The mysql_general.log file is empty.
I see this also this:
mysql> show variables like '%general_log%';
+------------------+----------------------------+
| Variable_name | Value |
+------------------+----------------------------+
| general_log | OFF |
| general_log_file | /var/run/mysqld/mysqld.log |
+------------------+----------------------------+
2 rows in set (0.00 sec)
mysql>
I am not able to get the SQL queries written in the log file, why?
Related
I try to configure mysql as mentioned here: https://websiteforstudents.com/install-erpnext-erp-platform-on-ubuntu-16-04-18-04-lts-with-nginx-mariadb-support/ on a Linux Mint 18.2 system.
I added innodb-file-format=barracuda to the [mysqld] section in /etc/mysql/mariadb.conf.d/50-server.cnf and restarted the mysql service. But these settings are taking no effect.
If I run following commands
sudo mysql -u root -p
SHOW VARIABLES LIKE 'innodb_file_format';
then I get following output.
+--------------------+----------+
| Variable_name | Value |
+--------------------+----------+
| innodb_file_format | Antelope |
+--------------------+----------+
The settings file ~/.my.cnf is not existing.
Why are my settings taking no effect?
++++++++++++ UPDATE 2019-06-03 ++++++++++++
I think I found the issue now. /etc/mysql/mariadb.conf.d/50-server.cnf is the correct logfile and the settings were not overwritten by another *.cnf-file. After a restart of the PC, the changed settings were used. The problem is, that restarting the mysql service (sudo service mysql restart or sudo /etc/init.d/mysql restart) is not enough.
So the question should be:
How do I correctly restart mysql so that changes in the *.cnf-files get active?
First, you should try to find my.cnf in other locations like
How do I find the MySQL my.cnf location
If no luck, scan all *.cnf files in /etc/mysql/mariadb.conf.d/ because Mariadb may read all *.cnf as config in that folder
Therefore you may add this value on 50-server.cnf, but it is overwritten in another files
Hope this helps
Don't do this:
systemctl restart mysqld
Instead, do:
systemctl stop mysqld
systemctl start mysqld
Not sure exactly what the syntax is for Debian's 'service' command but I believe that's just a wrapper around systemctl. Syntax should be easily found. Change the service name if your system uses something different, of course. I have found that for some services, doing a restart doesn't seem to reread the config files, while a stop and then start forces a complete reload and config file read.
I would like to solve this problem:
ERROR 1290 (HY000): The MySQL server is running with the --secure-file-priv option so it cannot execute this statement
with the official MySQL 8 image on docker.
the command:
SHOW VARIABLES LIKE "secure_file_priv";
gives:
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| secure_file_priv | NULL |
+------------------+-------+
1 row in set (0.01 sec)
Is it possible to overwrite the default value NULL for the --secure-file-priv by using the official MySQL 8 Docker image?
the default value in this image is set to NULL
config/my.cnf
Perfectly I would like to set just an environmental variable or parameter when using docker run or create instead of bringing my own config file.
but if this is not possible then how to use custom config file?
is it possible that custom file just overwrites this one parameter and leaves others as they are in the official image config?
According to this documentation, you can configure secure-file-priv through command-line by passing --secure-file-priv=dir_name
secure-file-priv possible values are: empty string, dirname or NULL as explained in the privous url.
From mysql-docker page:
Configuration without a cnf file: Many configuration options can be passed as flags to mysqld. This will give you the flexibility to customize the container without needing a cnf file. For example, if you want to change the default encoding and collation for all tables to use UTF-8 (utf8mb4) just run the following:
$ docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
So in our case would be like this:
dir_name should be a directory inside your container otherwise you will get the following error: mysqld: Error on realpath() on 'dir_name' (Error 2 - No such file or directory)
$ docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag --secure-file-priv=dir_name
And now our change is committed in MySQL
mysql> SHOW VARIABLES LIKE "secure_file_priv";
+------------------+----------+
| Variable_name | Value |
+------------------+----------+
| secure_file_priv | dir_name |
+------------------+----------+
Alternatively, you can use a custom configuration file as explained in here:
Using a custom MySQL configuration file: The default configuration for MySQL can be found in /etc/mysql/my.cnf, which may !includedir additional directories such as /etc/mysql/conf.d or /etc/mysql/mysql.conf.d. Please inspect the relevant files and directories within the mysql image itself for more details.
If /my/custom/config-file.cnf is the path and name of your custom configuration file, you can start your mysql container like this (note that only the directory path of the custom config file is used in this command):
$ docker run --name some-mysql -v /my/custom:/etc/mysql/conf.d -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag
This will start a new container some-mysql where the MySQL instance uses the combined startup settings from /etc/mysql/my.cnf and /etc/mysql/conf.d/config-file.cnf, with settings from the latter taking precedence.
I met a problem, the value of max_connction in MySQL is 214 after I set it 1000 via edit the my.cnf, just like below:
hadoop#node1:~$ mysql -V
mysql Ver 14.14 Distrib 5.7.15, for Linux (x86_64) using EditLine wrapper
MySQL version: 5.7
OS version : ubuntu 16.04LTS
mysql> show variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
1 row in set (0.00 sec)
As we can see, the variable value of max_connections is 151. Then , I edit the configuration file of MySQL.
yang2#node1:~$ sudo vi /etc/mysql/my.cnf
[mysqld]
character-set-server=utf8
collation-server=utf8_general_ci
max_connections=1000
Restart MySQL service after save the configraion.
yang2#node1:~$ service mysql restart
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to restart 'mysql.service'.
Multiple identities can be used for authentication:
1. yangqiang,,, (yang2)
2. ,,, (hadoop)
Choose identity to authenticate as (1-2): 1
Password:
==== AUTHENTICATION COMPLETE ===
yang2#node1:~$
Now, we guess the max_connection is 1000, really?
mysql> show variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 214 |
+-----------------+-------+
1 row in set (0.00 sec)
It is 214. I do not really understand this result, who can help me? thx!
You may set the value manually, e.g.
set global max_connections=500;
however, after a restart of MySQL the value is reset to 214.
The solution depends on the (version of) OS and the MySQL version. With Ubuntu 16.04 and MySQL >= 5.7.7 following works:
systemctl edit mysql
Enter
[Service]
LimitNOFILE=8000
save, this will create a new file
/etc/systemd/system/mysql.service.d/override.conf
and restart the server:
systemctl daemon-reload
systemctl restart mysql
For other environments: Can not increase max_open_files for Mysql max-connections in Ubuntu 15
As MySQL documentation on max_connections setting says:
Increasing this value increases the number of file descriptors that
mysqld requires. If the required number of descriptors are not
available, the server reduces the value of max_connections.
This means that probably your MySQL server does not have enough resources to maintain the required number of descriptors.
MySQL documentation on How MySQL Opens and Closes Tables makes it clear that:
The table_open_cache and max_connections system variables affect the
maximum number of files the server keeps open. If you increase one or
both of these values, you may run up against a limit imposed by your
operating system on the per-process number of open file descriptors.
Many operating systems permit you to increase the open-files limit,
although the method varies widely from system to system. Consult your
operating system documentation to determine whether it is possible to
increase the limit and how to do so.
Follow the following steps:
cp /lib/systemd/system/mysql.service /etc/systemd/system/
echo -e "\r\nLimitNOFILE=infinity" >> /etc/systemd/system/mysql.service
echo "LimitMEMLOCK=infinity" >> /etc/systemd/system/mysql.service
sudo systemctl daemon-reload
sudo service mysql restart
And change or add he following line into file /etc/mysql/mysql.conf.d/mysqld.cnf :
[mysqld]
max_connections=110
Just this!
#Mahdi_Mohammadi
1.Edit mysql service file
#sudo cat /etc/systemd/system/multi-user.target.wants/mysql.service
# MySQL systemd service file
[Unit]
Description=MySQL Community Server
After=network.target
[Install]
WantedBy=multi-user.target
[Service]
Type=forking
User=mysql
Group=mysql
PIDFile=/run/mysqld/mysqld.pid
PermissionsStartOnly=true
ExecStartPre=/usr/share/mysql/mysql-systemd-start pre
ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/run/mysqld/mysqld.pid
TimeoutSec=600
Restart=on-failure
RuntimeDirectory=mysqld
RuntimeDirectoryMode=755
##this bellow for tuneup
LimitNOFILE=infinity
LimitMEMLOCK=infinity
2.edit /etc/mysql/mysql.conf.d/mysqld.cnf
max_connections = 99999
Add session required pam_limits.so in /etc/pam.d/common-session (usually is not present by default).
The in /etc/security/limits.conf you can add some limits:
* hard nofile 8192
* soft nofile 4096
Also check using ulimit -a the open files limit.
This you can increase with ulimit -n 4096
Make sure you reboot at the end.
Changes I make to my.cnf don't seem to have any effect on the mysql environment. Here's a summary of what's happened...
I installed mysql 5.7 on Ubuntu 16.04 but then realized I needed to downgrade to mysql 5.6 due to incompatibility issues.
I apt purged the related applications and then removed any remaining directories such at /etc/mysql and /var/lib/mysql
I then installed mysql-5.6 (server and client) and related packages.
I was able to load one database from a dump from a server also running mysql 5.6 but when I tried to load a second database from a second dump from that same server, I got this error:
ERROR 2006 (HY000) at line 1721: MySQL server has gone away
When I Googled that, I saw results saying to set various options via the my.cnf file.
When I run...
updatedb && locate my.cnf
...I only see four results which are all links back to the same file: /etc/mysql/my.cnf.fallback. E.g. /etc/mysql/my.cnf.fallback == /etc/mysql/my.cnf
There are no .my.cnf files in either the root home directory or my user's home directory. I put a typo into the my.cnf file and reloaded mysql just to see the expected error and know the file was being loaded. I then removed the erroneous code and added the following:
[mysqld]
max_allowed_packet=1073741824
I then reloaded mysql by running in various ways:
service mysql restart
or
service mysql stop
service mysql start
or
/etc/init.d/mysql stop
/etc/init.d/mysql start
I then kept getting this default value indicating that it was not getting set from my.cnf:
mysql> SHOW VARIABLES LIKE 'max_allowed_packet';
+--------------------+---------+
| Variable_name | Value |
+--------------------+---------+
| max_allowed_packet | 4194304 |
+--------------------+---------+
If I do this:
mysql> SET GLOBAL max_allowed_packet=1073741824;
and log out and back into the mysql client, I see the correct value:
mysql> SHOW VARIABLES LIKE 'max_allowed_packet';
+--------------------+------------+
| Variable_name | Value |
+--------------------+------------+
| max_allowed_packet | 1073741824 |
+--------------------+------------+
But of course, if I restart the mysql server, the value reverts.
I've exhausted my search ability. What can I possibly be doing wrong?
The config files are fine. The root cause is a bug in the MySQL 5.6 packaging for Ubuntu 16.04.
If you check your /var/log/syslog you'll probably see a line like this:
Sep 15 18:56:09 ip-172-31-18-162 kernel: [ 383.840275] audit: type=1400 audit(1505501769.234:50): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/etc/mysql/my.cnf.fallback" pid=25701 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
A security tool called AppArmor is denying access to a symlinked file (/etc/mysql/my.cnf.fallback).
Try this workaround, which will allow symlinks to be read by mysqld.
echo '/etc/mysql/** lr,' >> /etc/apparmor.d/local/usr.sbin.mysqld
systemctl reload apparmor
Now mysqld should see your custom config.
This bug appears to be fixed in the MySQL 5.7 Ubuntu package.
Obviously my.cnf.fallback is not the correct configuration file.
If you try this commands you can get output for possible my.cnf locations:
$ which mysqld
/usr/sbin/mysqld
$ /usr/sbin/mysqld --verbose --help | grep -A 1 "Default options"
Default options are read from the following files in the given order:
/etc/mysql/my.cnf ~/.my.cnf /usr/etc/my.cnf
It means mysql will check those locations for my.cnf file. Simply rename /etc/mysql/my.cnf.fallback as /etc/mysql/my.cnf:
mv /etc/mysql/my.cnf.fallback /etc/mysql/my.cnf
I am able to connect to mysql database and query it. But, I am NOT able to find the socket file.
$ps -ef|grep mysql
mysql 31408 30874 0 18:46 pts/1 00:00:00 /bin/sh /usr/bin/mysqld_safe --defaults-file=/mysql/admin/ofile/TEST1.cnf
mysql 31959 31408 0 18:46 pts/1 00:00:01 /usr/sbin/mysqld --defaults- file=/mysql/admin/ofile/TEST1.cnf --basedir=/usr -- datadir=/mysql01data/TEST1/data --plugin-dir=/usr/lib64/mysql/plugin --log- error=/mysql/admin/TEST1/errors/mysqld_safe.err --pid- file=/mysql/admin/TEST1/run/mysqld_safe.pid
Here is my socket file entry in TEST1.cnf:
$ cat /mysql/admin/ofile/TEST1.cnf|grep sock
socket = /mysql/admin/TEST1/run/TEST1.sock
The corresponding directory only contains pid file. There is no socket file.
-sh-4.1$ cd /mysql/admin/TEST1/run
-sh-4.1$ ls -lrt
total 4
-rw-rw---- 1 mysql mysql 6 Apr 29 18:46 mysqld_safe.pid
This is the MySQL 5.6 version I installed through RPM's on RHEL 6.5. I have my old custom scripts which uses socket file to connect to the database.
So, I am wondering how I can use the socket file to connect to the database? Why the socket file is not created by default?
The socket file for a running instance of MySQL Server should be something that can be found with this shell command:
sudo lsof -a -U -p $(pgrep -d, -f /path/to/your/running/mysqld)
One possible cause of being unable to find the socket file would be if it had been deleted after the server was started. In that case the above command should work, and show something like (deleted) after the path.
That was my original assumption on this question... but here the issue was a configuration oversight. The "defaults file," commonly called my.cnf contains multiple sections. The [client] section configures client utilities, like mysql and mysqldump, while the [mysqld] section configures the server daemon. If the socket directive isn't in the appropriate section, the server (and/or client utilities) will look in the location compiled in by default, with /tmp/mysql.sock or /var/lib/mysql/mysql.sock being a couple of examples of common default locations.