I searched about the topic subject and tested options, but I still cant increase the open-files-limit on my mariadb server that is used as remote database server for cpanel/whm server.
here is s good reference
http://duntuk.com/how-raise-ulimit-open-files-and-mysql-openfileslimit
I increased it in
/etc/my.cnf
open-files-limit=65550
here is some results
#ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 63471
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 65535
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
#ulimit -Hn -Sn
open files (-n) 65535
open files (-n) 65535
cat /etc/systemd/system/mariadb.service.d/limits.conf
[Service]
LimitNOFILE=65500
cat /usr/lib/systemd/system/mariadb.service
[Unit]
Description=MariaDB database server
After=syslog.target
After=network.target
[Service]
Type=simple
User=mysql
Group=mysql
LimitNOFILE=infinity
LimitMEMLOCK=infinity
and still in mysql I get
show global variables like 'open%';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| open_files_limit | 1024 |
+------------------+-------+
1 row in set (0.00 sec)
and im getting following error over and over after restart
60108 16:30:02 [ERROR] mysqld: Can't open file: './username_wp/wp_users.frm' (errno: 24)
160108 16:30:02 [ERROR] Error in accept: Too many open files
160108 16:30:04 [ERROR] Error in accept: Too many open files
160108 16:30:06 [ERROR] Error in accept: Too many open files
160108 16:30:11 [ERROR] mysqld: Can't open file: './username_db/strikes.frm' (errno: 24)
and here is the startup log of mariadb
[Warning] Could not increase number of max_open_files to more than 1024 (request: 132107)
RH/CentOS mariadb integration is quiet confusing, using in some places mariadb as its name, mysqld elsewhere...
You should edit(or create) a .conf file in
/etc/systemd/system/mariadb.service.d/
For eg: /etc/systemd/system/mariadb.service.d/centreon.conf
Then edit it as kujiy suggested
[Service]
LimitNOFILE=320000
Don't forget to reload systemd services files:
# systemctl daemon-reload
Cedric
This is an OS problem.
I think you have to increase the "hard limit" in /etc/security/limits.conf, something like this:
* hard nofile 65536
* soft nofile 16384
then use ulimit to increase the "soft limit".
finally i find solution but without any reasonable cause !
i was running MariaDB 5.5 and all settings where fine but the soft limit did not goes more than 1024 !
i was thinking my mariadb is 10.0 as i freshly installed it
after i find out that its 5.5 i tried to upgrade it to v10.0 ! and then BOOM ! problem solved without any extra action or setting ( kernel setting allows maximum files limit as described in reference link )
i hope it help other peoples have same problem ( but after try all other required settings )
You can see the official instruction in the mariadb.service file;
[root#db1 system]# cat /usr/lib/systemd/system/mariadb.service | grep exam -A 5
# For example, if you want to increase mariadb's open-files-limit to 10000,
# you need to increase systemd's LimitNOFILE setting, so create a file named
# "/etc/systemd/system/mariadb.service.d/limits.conf" containing:
# [Service]
# LimitNOFILE=10000
Though I think this should be written in the official manual...
I have CentOS 7.3 with MariaDB 10.0.29 installed from official repo.
For some reason, systemd unit is named mysql and I had to create /etc/systemd/system/mysql.service.d/oioki.conf:
[Service]
LimitNOFILE=500000
Don't forget to run systemctl daemon-reload after that.
open_files_limit is dynamic value that depends on next params:
#my.cnf file
# max connections
max_connections = 64
# table_open_cache = max_connections * tables used in one transaction + 5
table_open_cache = 800
# table_definition_cache = all tables(50) + max_connections + 5
table_definition_cache = 400
# open_files_limit = table_open_cache * 1.4
open_files_limit = 1120
To check the value of open_files_limit run this command from mysql:
mysql> select ##open_files_limit;
...it will output:
+--------------------+
| ##open_files_limit |
+--------------------+
| 12903 |
+--------------------+
1 row in set (0.000 sec)
I had this problem running RedHat Enterprise Linux 7.2 with Software Collections (SCL) and SystemD
rh-mariadb100-mariadb-server-10.0.20-1.el7.x86_64
In my case the hard limit was ok, but the soft limit was stuck on 1024
Summary of the old and new
old : RHEL6.x : MySQL : init.d : mysql_safe : 'root', mysqld : 'mysql'
new : RHEL7.x : MariaDB : SystemD : mysql_safe : 'mysql', mysqld 'mysql'
Note that the old initl.d wrapper script runs as root, but the new SystemD wrapper script runs as 'mysql'. This can cause mysql_safe to not make the call to ulimit. In my case setting LimitNOFILE in SystemD didn't help - all it did was increase the file limit for the wrapper, but NOT for the daemon itself.
I found a two step process to get it working
Step 1 - re-instate the mysqld_safe section to the conf file so that mysql_safe will read it
sudo vi /etc/opt/rh/rh-mariadb100/my.cnf.d/mariadb-server.cnf
[mysqld_safe]
open_files_limit=2048
Step 2 - create a SystemD drop-in to start the wrapper as root (the daemon will still run as 'mysql'). Note that LimitNOFILE is commented out.
/etc/systemd/system
mkdir rh-mariadb100-mariadb.service.d
cd rh-mariadb100-mariadb.service.d
vi limits.conf
[Service]
#LimitNOFILE=2048
User=
Group=
Reload SystemD service files
systemctl daemon-reload
now restart the service, check value of open_files_limit in mysql client
it should now say 2048 (new soft limit)
The proper solution would probably be to get rid of the wrapper and use native SystemD.
UPDATE
it looks like this is fixed in the more recent version, which no longer uses the mysql_safe wrapper
rh-mariadb101-mariadb-10.1.19-6.el7.x86_64
To increase open-files-limit in MySQL 5.6 on Centos 7 you must:
nano /usr/lib/systemd/system/mysqld.service
add at the end of the file:
LimitNOFILE=65535
LimitNPROC=65535
then:
systemctl daemon-reload
systemctl restart mysqld
Done! , I assume that for MariaDB is the same thing
Ensure that no other files exist in the:
/etc/systemd/system/mariadb.service.d
folder that might be limiting the number of files.
That was my case, there was another file that was reducing the limit I set :)
Related
I'm trying to change max_connections and open_files_limit for MariaDB version 5.5.60 on CentOS 7.
Since there could be multiple my.cnf files, I used mysql --help --verbose to get possible locations. Here's the output:
Default options are read from the following files in the given order:
/etc/mysql/my.cnf /etc/my.cnf ~/.my.cnf
The /etc/mysql/my.cnf doesn't exist and .my.cnf isn't in my user directory, /root/ or mysql user's directory (which doesn't exist to begin with).
So I'm left with /etc/my.cnf. This is what it looks like:
[mysqld]
datadir=/data/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
max_connections = 5000
open_files_limit = 10240
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid
#
# include all files from the config directory
#
!includedir /etc/my.cnf.d
The /etc/my.cnf.d from the last line contains server.cnf, which includes these two lines:
[server]
max_connections=5000
open_files_limit=10240
However, when I run systemctl restart mariadb, only the max_connections is changed, but I just can't get it to read the new value of open_files_limit.
It keeps doing this:
MariaDB [(none)]> SHOW VARIABLES LIKE 'open_files_limit';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| open_files_limit | 1024 |
+------------------+-------+
1 row in set (0.00 sec)
Do you have any idea why it's doing this and how I could get the server to read the new value of open_files_limit?
Thank you in advance
Sasha
The value of open_file_limits variable can't be higher than your system limits.
Check output of ulimit -n and change your system configuration. Preferable assign these limits not system wide, but for the user under which MariaDB server is running.
I created this issue to solve a problem you have with zabbix, version 3.2 running on centos 7. The above error appeared when trying to access the zabbix GUI in a few moments.
I edited the php.ini file in CentOS as I saw here on stack overflow and it did not solve, it was necessary to edit the file /etc/httpd/conf.d/zabbix.conf, and modify the attribute php_value memory_limit 128M to 256M or 512M.
Just in case:
You may reset filter settings by adding filter_rst=1 after ? symbol in php query.
For example:
https://your.zabbix.server.com/zabbix/latest.php?filter_rst=1
First of all I had to find the file which contains the 'memory_limit' string by running this command:
grep -rnw '/etc' -e 'memory_limit'
Based on the result I increased the limit from 128M to 1024M in this below file:
/etc/opt/rh/rh-php72/php-fpm.d/zabbix.conf
Finally, restart the Apache, Zabbix, and PHP services:
systemctl restart rh-php72-php-fpm.service httpd zabbix-server
I met a problem, the value of max_connction in MySQL is 214 after I set it 1000 via edit the my.cnf, just like below:
hadoop#node1:~$ mysql -V
mysql Ver 14.14 Distrib 5.7.15, for Linux (x86_64) using EditLine wrapper
MySQL version: 5.7
OS version : ubuntu 16.04LTS
mysql> show variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
1 row in set (0.00 sec)
As we can see, the variable value of max_connections is 151. Then , I edit the configuration file of MySQL.
yang2#node1:~$ sudo vi /etc/mysql/my.cnf
[mysqld]
character-set-server=utf8
collation-server=utf8_general_ci
max_connections=1000
Restart MySQL service after save the configraion.
yang2#node1:~$ service mysql restart
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to restart 'mysql.service'.
Multiple identities can be used for authentication:
1. yangqiang,,, (yang2)
2. ,,, (hadoop)
Choose identity to authenticate as (1-2): 1
Password:
==== AUTHENTICATION COMPLETE ===
yang2#node1:~$
Now, we guess the max_connection is 1000, really?
mysql> show variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 214 |
+-----------------+-------+
1 row in set (0.00 sec)
It is 214. I do not really understand this result, who can help me? thx!
You may set the value manually, e.g.
set global max_connections=500;
however, after a restart of MySQL the value is reset to 214.
The solution depends on the (version of) OS and the MySQL version. With Ubuntu 16.04 and MySQL >= 5.7.7 following works:
systemctl edit mysql
Enter
[Service]
LimitNOFILE=8000
save, this will create a new file
/etc/systemd/system/mysql.service.d/override.conf
and restart the server:
systemctl daemon-reload
systemctl restart mysql
For other environments: Can not increase max_open_files for Mysql max-connections in Ubuntu 15
As MySQL documentation on max_connections setting says:
Increasing this value increases the number of file descriptors that
mysqld requires. If the required number of descriptors are not
available, the server reduces the value of max_connections.
This means that probably your MySQL server does not have enough resources to maintain the required number of descriptors.
MySQL documentation on How MySQL Opens and Closes Tables makes it clear that:
The table_open_cache and max_connections system variables affect the
maximum number of files the server keeps open. If you increase one or
both of these values, you may run up against a limit imposed by your
operating system on the per-process number of open file descriptors.
Many operating systems permit you to increase the open-files limit,
although the method varies widely from system to system. Consult your
operating system documentation to determine whether it is possible to
increase the limit and how to do so.
Follow the following steps:
cp /lib/systemd/system/mysql.service /etc/systemd/system/
echo -e "\r\nLimitNOFILE=infinity" >> /etc/systemd/system/mysql.service
echo "LimitMEMLOCK=infinity" >> /etc/systemd/system/mysql.service
sudo systemctl daemon-reload
sudo service mysql restart
And change or add he following line into file /etc/mysql/mysql.conf.d/mysqld.cnf :
[mysqld]
max_connections=110
Just this!
#Mahdi_Mohammadi
1.Edit mysql service file
#sudo cat /etc/systemd/system/multi-user.target.wants/mysql.service
# MySQL systemd service file
[Unit]
Description=MySQL Community Server
After=network.target
[Install]
WantedBy=multi-user.target
[Service]
Type=forking
User=mysql
Group=mysql
PIDFile=/run/mysqld/mysqld.pid
PermissionsStartOnly=true
ExecStartPre=/usr/share/mysql/mysql-systemd-start pre
ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/run/mysqld/mysqld.pid
TimeoutSec=600
Restart=on-failure
RuntimeDirectory=mysqld
RuntimeDirectoryMode=755
##this bellow for tuneup
LimitNOFILE=infinity
LimitMEMLOCK=infinity
2.edit /etc/mysql/mysql.conf.d/mysqld.cnf
max_connections = 99999
Add session required pam_limits.so in /etc/pam.d/common-session (usually is not present by default).
The in /etc/security/limits.conf you can add some limits:
* hard nofile 8192
* soft nofile 4096
Also check using ulimit -a the open files limit.
This you can increase with ulimit -n 4096
Make sure you reboot at the end.
I have mysql ver. 5.1.49-3, I am working on linux debian. I am trying to set open-files-limit to 65535. so I edited te my.cnf in /etc/mysql/
[mysqld]
open_files_limit = 65535
[mysqld_safe]
open_files_limit = 65535
then in /etc/security/limit.conf
* soft nofile 100000
* hard nofile 200000
After restarting mysql service, when I run this command in linux
ps -ef|grep mysql
I got 65535. when I log into mysql as root and fetch the value of open-files-limit
show global variables like "%open_files_limit%";
I got 1024. Please help.
If mysql is started with systemd, this setting is important:
In the file /lib/systemd/system/mysql.service you have to add this 2 lines in the [Service] section at the end:
LimitNOFILE = infinity
LimitMEMLOCK = infinity
After this restart systemctl and mysql:
systemctl daemon-reload
/etc/init.d/mysql restart
To check if the configuration is effective, you can get the parameter from the running mysql process like this:
cat /proc/$(pgrep mysqld$)/limits | grep files
All I need is to add this line to /etc/pam.d/common-session:
session required pam_limits.so
then restart apache
An issue with older versions of MySQL require you to use use open-files-limit (dashes not underbars) in my.cnf. See http://bugs.mysql.com/bug.php?id=40368
do ulimit -a for show. ulimit -n NUMBER can change to YOUR_NUMEBR open files
Take a look at the official documentation:
"The value of this variable at runtime is the real value permitted by the system and might be different from the value you specify at server startup."
On Unbuntu 14.04 this worked
vim /etc/mysql/my.cnf
[mysqld]
open-files-limit=16000
After this, just restart mysql
/etc/init.d/mysql restart
I have recently started on a PoC project wherein we are developing a small web app. The initial setup is done on a micro instance from AWS. We are on rails+mysql stack.
After installing/running MySQL, I see that about 500+ MB RAM has been consumed already; leaving quite less for rest of the systems (micro instances have barely 620 MB RAM).
Our app is fairly simple at this stage. Can I do something to reduce the memory consumed by MySQL server?
Appreciate the help.
As of MySQL 8.0.30:
Edit your /etc/mysql/my.cnf file and add the following:
[mysqld]
performance_schema = 0
Restart your mysql server and happiness should ensue.
To verify that the configuration change has been loaded correctly, start a new mysql session (e.g. mysql -u root -p) and run the following:
SHOW VARIABLES LIKE '%perf%';
You should see the following line at the top:
| performance_schema | OFF |
It should read OFF. If it reads ON, your config was not properly loaded for some reason.
Change this setting in the MySQL configuration file (my.cnf)
key_buffer = 8M
max_connections = 30 # Limit connections
query_cache_size = 8M # try 4m if not enough
query_cache_limit = 512K
thread_stack = 128K
Just to add to the other answer. I recently had this problem myself with the Amazon micro instance (not Ubuntu). The my.cnf file is almost empty so what I did was this:
cp /etc/my.cnf /etc/my.cnf.orig
cp /usr/share/mysql/my-medium.cnf /etc/my.cnf
Edit my.cnf and enable the innodb lines if applicable. Restart mysqld.
Also the micro instance has no swap, that might be a problem..
SWAPFILE=/mnt/swapfile.swap
dd if=/dev/zero of=$SWAPFILE bs=1M count=512
mkswap $SWAPFILE
swapon $SWAPFILE
Then in /etc/rc.local add:
swapon /mnt/swapfile.swap
To save memory in ruby you might want to use ruby enterprise:
bash < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
gpasswd -a root rvm
source /etc/profile.d/rvm.sh
rvm get head
rvm reload
rvm install ree
rvm --default use ree
I have a server with only 500mb ram and found that mysql started using a lot of ram as my tables got larger. After playing with a bunch of the settings, what reduced memory usage for me was to convert all my tables to MyISAM.
If you dont need the features of innodb converting tables to MyISAM helps quite a bit.
You can convert tables like this :
ALTER TABLE test.mytable ENGINE=MyISAM;
After this change I found that memory usage decreased by 20%.
To get a further reduction in memory usage you can convert ALL of your tables to MyISAM and then turn off innodb support in mysql altogether.
This reduced my memory usage by 50%.
You can do this by adding :
[mysqld]
default_storage_engine=myisam
innodb=OFF
and then restarting mysql.
Configure Swapfile
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo cp /etc/fstab /etc/fstab.bak
sudo nano /etc/fstab
Add the following line at the end, save and close:
/swapfile swap swap defaults 0 0
sudo sysctl vm.swappiness=10
sudo vi /etc/sysctl.conf
Add the following line at the end, save and close:
vm.swappiness=10
Configure PHP
sudo nano /opt/bitnami/php/etc/memory.conf
; Bitnami memory configuration for PHP-FPM
;
; Note: This will be modified on server size changes
pm.max_children=10
pm.start_servers=2
pm.min_spare_servers=2
pm.max_spare_servers=10
pm.max_requests=5000
Configure MariaDB (MySQL)
sudo nano /opt/bitnami/mariadb/conf/bitnami/memory.conf
[mysqld]
#wait_timeout = 120
long_query_time = 1
query_cache_limit=2M
query_cache_type=1
query_cache_size=8M
innodb_buffer_pool_size=16M
#innodb_log_file_size=128M
#innodb_flush_method=O_DIRECT
tmp_table_size=16M
max_connections = 100
max_user_connections = 250
key_buffer_size=8M
sudo /opt/bitnami/ctlscript.sh restart