I have never really played around with mysql settings before, but on our new linux cloud server it appears that mysql is eating up all the memory till it crashes, then it cannot restart as there is no more memory to restart the service, I have to reboot the cloud server.
So I was looking at how I could tame the memory usage, and after reading about key_buffer_size (and another setting I cannot recall of the my head) I had a look at the my.cnf file, and there is nothing with this setting on it. My my.cnf is as follows...
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysql.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
datadir = /var/lib/mysql
[mysql]
!includedir /etc/mysql/conf.d
Without the key_buffer_size set... will it just keep eating up memory till it runs out? Shouldn't this setting be set?
Cheers
The default for key_buffer_size is 8M per http://dev.mysql.com/doc/refman/5.0/en/server-parameters.html
However, I would ask that you set these first (and restart MySQL) and see if the problem persists.
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
query_cache_limit = 1M
query_cache_size = 16M
If after those settings are used, the problem persists, then it's likely not a configuration issue, but something deeper.
Additionally, see these:
https://dba.stackexchange.com/questions/31182/mysql-slowly-uses-memory-until-it-starts-to-use-swap
https://dba.stackexchange.com/questions/7400/why-does-mysql-use-all-of-memory-and-goes-into-swap-when-doing-lots-of-delayed-i
Thanks kinds for the advice!
Turns out it was the innodb_buffer_pool_size. It was set to high for my machine. I have adjusted it as well as optimized how much memory Apache uses. Everything seems to be running better and there is a bit of memory headroom. Hope that fixes it.
Related
I am trying to get my web server to connect to my remote database (Mysql).
The connection is happening in my web application which is built on yii2.
I have created all user details, GRANTED all privileges and allowed connections from ALL hosts (%) (Progressively in troubleshooting this).
Now I can connect from home using HeidiSQL or using Mysql Workbench without issue.
I have even created a blank yii2 site at home and it works instantly, but uploading the site to the web server instantly gives the same issue.
Database Exception – yii\db\Exception
SQLSTATE[HY000] [2003] Can't connect to MySQL server on '46.105.111.111' (110)
↵
Caused by: PDOException
SQLSTATE[HY000] [2003] Can't connect to MySQL server on '46.105.111.111' (110)
in /home/xxxxxxx/xtest.xxxxxxxx.tv/vendor/yiisoft/yii2/db/Connection.php at line 602
I have run netstat on the db server:
I have been through the etc/mysql/my.cnf
<pre>[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 7999
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
#bind-address = *
key_buffer = 16M
max_allowed_packet = 64M
thread_stack = 192K
thread_cache_size = 8
myisam-recover = BACKUP
query_cache_limit = 1M
query_cache_size = 16M
expire_logs_days = 10
max_binlog_size = 100M
max_connections = 65535
open-files-limit = 65535
max_connect_errors = 65535
innodb_buffer_pool_size = 1G
table_open_cache = 1000
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT
tmp_table_size = 256M
max_heap_table_size = 256M
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
[mysql]
[isamchk]
key_buffer = 16M</pre>
I have looked at the iptables:
I am totally bedazzled as having read about 20 pieces of information about mysql security, i get the feeling its a permissions issue.
I have even tried using my own PDO Wrapper, so i a, 99% sure its not a Yii2 issue, would really appreciate any help.
Hopefully I have provided enough info as I got a downvote previously.
many thanks for the help.
It turns out that the web host was actually blocking the outgoing port for the connection.
The output error from yii being database error, nievly led me to think the issue was at the database end.
Simple logic when you read back the question.
I would say to anyone that finds this question, remember with a connection issue that communication is two way, check both ends of the connection.
I would like to access to my MySQL DB through local network which is installed by Wamp(2.2e) server.
I managed to access phpMyAdmin from other local computer so I can view the content of the DB, but I would like to use the data from it by other application.
I turned off my firewall just for testing on my "server" and my "client" machine as well.
I cannot telnet to the DB (telnet "local network IP of my server" 3306) from my client machine.
on my server netstat -an shows the MySQL DB:
Proto | Local Address | Foreign Address | State
TCP | 0.0.0.0:3306 | 0.0.0.0:0 | Listening
I have add users("client" local network ip, "server" local network ip) to the database with all of the permissions.
On the "server" machine I can telnet to its own MySQL DB by using its local network IP after I add it to the DB users table as well.
Here is my my.ini file without the comment lines:
[client]
port = 3306
socket = /tmp/mysql.sock
[wampmysqld]
port = 3306
socket = /tmp/mysql.sock
key_buffer = 16M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
basedir=c:/wamp/bin/mysql/mysql5.5.24
log-error=c:/wamp/logs/mysql.log
datadir=c:/wamp/bin/mysql/mysql5.5.24/data
skip-federated
log-bin=mysql-bin
binlog_format=mixed
server-id = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
[mysqld]
port=3306
I have tried a few things already which did not work:
skip-name-resolve at mysqld
bind-address=0.0.0.0 (it is the same as the default) at mysqld
bind-address="local network IP" at mysqld did not do any difference
I was rewriting other files as well, but I guess I need to do changes here(my.ini) to access to the database on local network.
I my wamp server is in offline mode just now, as I can access to phpMyAdmin I thought I should be able to access to the database with left it offline. I tried it in online mode as well, did not do any difference.
I spent quite a bit of a time solving the problem but unfortunately I have no more ideas. Every suggestions, ideas will be appreciated.
So I kind of Figured it out.
I am using Kaspersky endpoint security on my development machine. For some reason it is running weird on it (the restrictions managed my a central server, computers on the same group as me acting normal; responding for the rules as they should). It was not enough to disable my firewall I needed to disable whole kaspersky then I was able to telnet to the MySQL port from an other machine.
So if firewall rules added to "normal" machines (not my weird one), client and server as well; Wamps MySQL DB was accessible in local network without any other changes.
I know it is rather a special case but I hope it still could help for some one.
I want to increase max_allowed_packet variable on my server and I don't know why, when I add the sentence in /etc/my.cnf, it doesn't work!! This is the line I added:
max_allowed_packet=62M
But when I check the current value through mysqladmin variables, it shows 10M!! I've even restarted my server! And nothing happens!
What happens here?
The settings file will normally have several sections (what MySQL calls groups). You're possibly disregarding that fact:
# The following options will be passed to all MySQL clients
[client]
port = 3306
socket = /tmp/mysql.sock
# Here follows entries for some specific programs
# The MySQL server
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-external-locking
key_buffer_size = 384M
max_allowed_packet = 1M
I was wondering if these my.cnf values would be ok for a Xeon 2.33GHz VPS with 1GB of RAM.
There are ~5 sites on the server with low traffic and the server is running Apache/PHP/MySQL... Also I want to enable the MySQL query cache and would appreciate any advise re the amount of RAM to allocate to the cache...
MySQL Settings:
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0
key_buffer = 16K
max_allowed_packet = 1M
thread_stack = 64K
table_cache = 4
sort_buffer = 64K
net_buffer_length = 2K
bind-address = 127.0.0.1
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
Thanks for the help :)
Some memory chache was set too small compare to 1GB RAM, even if you may use a virtual machine actually.
Performance may be better if you change some options and that won't eat too much resources.
If you are not sue what to change, take a look at MySQL recommanded configuration.
These files was located at /usr/share/doc/mysql-server-x.x/examples
Your configuration may belongs to my-small.conf, my-medium.cnf is better suitable for you.
I recommend you use mytop to analyze real-time activity of your database.
Also used mysqltuner to give you suggestions on your my.cnf values.
Too enables logging of slow queries:
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
The configuration of my dedicated server for a high rate of transactions if anything helps:
max_allowed_packet=16M
key_buffer_size=8M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=512M
join_buffer_size=40M
table_open_cache=1024
query_cache_size=40M
table_definition_cache=256
innodb_additional_mem_pool_size=10M
key_buffer_size=16M
max_allowed_packet=32M
max_connections = 300
query_cache_limit = 10M
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
I hope it helps somewhat. If you want to refer details about these(and many other) server parameters, refer this Official MySQL Documentation
Regards.
I am running a drupal site. I got an error in my site user warning: Got a packet bigger than 'max_allowed_packet' bytes query. I have set the value as high as 128M . Even after that same error is reported.
What is the issue here?? Why is it not working ??
Is there a maximum limit for the value max_allowed_packet ?
This is the bleeding edge:
set global max_allowed_packet=1073741824;
Although, it is probably not a good idea to set it that high in your case.
As a side note, I experienced this error with mysqldump, and setting this maximum didn't help. This did the trick: $ mysqldump --max_allowed_packet=999M -u root -p table_name > table_name.sql
Often this can be caused by the variables not actually taking effect- you make the configuration change, but in the wrong my.cnf, or you forget to bounce the app, etc.
An easy way to check a running mysql instance is to do something like this in a shell:
mysqladmin variables -u root -p
and enter in your root password. This will dump all of the current variables (including max_allowed_packet), and will let you verify what it's set to. If it's set to 128M and you're still choking on it, then you'll need to increase it- but it's pretty unlikely.
first is you need to set your max_allowed_packet to 128M in your my.cnf file.
to find it, use "locate my.cnf" command in your command line.
the file should look like this:
#
!includedir /etc/my.cnf.d
#max_allowed_packet = 1024M
[mysqld]
port = 3306
key_buffer_size = 256M
# max_allowed_packet = 100M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
bind-address = 202.90.158.47
# skip-networking
log = /var/log/mysql.access.log
log-error = /var/log/mysql.error.log
wait_timeout = 1
[mysqldump]
#max_allowed_packet = 101M
be sure to uncomment(remove the # sign before the max_allowed_packet = 128M line)
and finally, restart your sql using command "/etc/init.d/mysqld restart"
that should do the trick. :D
Edit your /etc/my.cnf, adding the max_allowed_packet variable.
It should look like this:
[mysqld]
max_allowed_packet=1000000000
Then restart your server.
Try setting max_allowed_packet = 128M as the very last option under the [mysqld] category of my.cnf.
When I had it as the first option, it would not work, but when I had it as the last option, it worked! I think this is because some other variables were over-riding max_allowed_packet.
After changing my.cnf, restart MySQL using sudo service mysql restart, and check the setting using SHOW VARIABLES LIKE 'max_allowed_packet';
You need to set the setting in all sections that apply to the action you are doing, and always in the [MySQLd] section. The setting applies to the buffer of the elements you are using. So under [MySQLd] is for your MySQL server proces deamon on linux / service on windows. And if you want to make a dump with MySQLDump add it as parameter on the command line or make a section [MySQLDump] in your my.ini as well for this tool with the same parameter to make it permanent. If you want to import the dump again with MySQL again use the parameter on the command line or make a section [MySQL] with again the same parameter in your my.ini to make the choice permanent for this tools also.
I kept on talking about my.ini because i am on windows but on linux that is my.cnf of course.
I decided to explain it here because it took me ages to figure this out because it is not explained anywhere. In examples however i noticed some ppl having the setting under multiple sections so i started to google more and found correlation between the sections and the actions they where doing. Now i never have this problem anymore and settings as high as the mentioned 128M here are not needed in most cases. However because it's the maximum the server will use for this buffer if you have the memory just set it high enough to never get into trouble with your actions. The size you actually need is a little bit larger than the largest record in your database.
I experienced this error with mysqldump with LONGBLOB fields, and setting this maximum didn't help. This did the trick:
$ mysqldump --max_allowed_packet=999M -u root -p table_name > table_name.sql