The following is my default production MySQL configuration file (my.cnf) for a pure UTF-8 setup with InnoDB as the default storage engine.
[server]
bind-address=127.0.0.1
innodb_file_per_table
default-character-set=utf8
default-storage-engine=innodb
The setup does the following:
Binds to localhost:3306 (loopback) instead of the default *:3306 (all interfaces). Done to increase security.
Sets up one table space per table. Done to increase maintainability.
Sets the default character set to UTF-8. Done to allow for easy internationalization by default.
Sets the default storage engine to InnoDB. Done to allow for row-level-locking by default.
Assume that you could further improve the setup by adding a maximum of three (3) configuration parameters. Which would you add and why?
An improvement would in this context mean either a performance improvement, a reliability improvement or ease-of-use/ease-of-maintainability increase. You can assume that the machine running the MySQL instance will have 1000 MB of RAM.
To cache more data:
innodb_buffer_pool_size = 512M
If you write lots of data:
innodb_log_file_size = 128M
, to avoid too much log switching.
There is no third I'd add in any case, all other depend.
Allocating more memory than the default of 8M to InnoDB (using innodb_buffer_pool_size) is surely an enhancement. Regarding the value, on a dedicated database server as yours you can set it up to the 80% of your RAM and the higher you set this value, the fewer the interactions with the hard disk will be. Just to give my two cents, I'd like to mention that you can have some performance boost tweaking the value of innodb_flush_log_at_trx_commit, however sacrificing ACID compliance... According to the MySQL manual:
If the value of
innodb_flush_log_at_trx_commit is 0,
the log buffer is written out to the
log file once per second and the flush
to disk operation is performed on the
log file, but nothing is done at a
transaction commit.
So you might loose some data that were not written properly in the database due to a crash or any malfunction. Again according to the MySQL manual:
However, InnoDB's crash recovery is
not affected and thus crash recovery
does work regardless of the value.
So, I would suggest:
innodb_flush_log_at_trx_commit = 0
Finally if you have a high connection rate (i.e. if you need to configure MySQL to support a web application that accesses the database) then you should consider increasing the maximum number of connections to something like 500. But since this is something more or less trivial and well known, so I'd like to emphasize on the importance of back_log to ensure connectivity.
I hope these information will help you optimize your database server.
Increase the innodb buffer pool size, as big as you can practically make it:
innodb_buffer_pool_size=768M
You'll also want some key buffer space for temp tables:
key_buffer_size=32M
Others would depend on what you are doing with the database, but table_cache or query_cache_size would be a couple other potentials.
Related
we are trying to improve the efficiency of our database server.
One of the recommendations of MySQLTunner is rising the innodb_log_file_size to 12 GB. As we see this change could improve significantly the speed and the performance of our queries. The problem came when we increase this parameter more than 1 GB, the service wont start, we delete the logs, stop it cleanly, and still cant start with this parameter above 1 GB.
Someinfo:
mysql Ver 14.14 Distrib 5.5.62, for debian-linux-gnu
innodb_buffer_pool_size = 100G
innodb_file_per_table = ON
innodb_buffer_pool_instances = 64
innodb_stats_on_metadata = OFF
innodb_log_file_size = 1G
innodb_log_buffer_size = 8M
Theres algo enough space in partitions for keeping this log size
Thanks!
In MySQL 5.5, you can't increase the innodb log file size over 4GB total. The innodb_log_file_size can only be 4GB / innodb_log_files_in_group (which is 2 by default, and there's no benefit to change that). So you can set the log file to a max of 2GB.
See https://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_log_file_size
The max combined size of the log files increased to 512GB in 5.6.3. Again, innodb_log_file_size should be the size of one log file, so if you use multiple log files, the total cannot exceed 512GB.
I agree with Rick James' answer that increasing the log file size is not a magical solution to make queries run faster. It won't do that.
It's sometimes useful to increase the innodb log file size, if the bottleneck is that you run out of log space faster than dirty pages can be flushed to the tablespace, because you have very high write traffic. That's affected by the rate of writes, not the speed of individual writes.
For most apps, two 2GB log files is more than enough. If it isn't, it's probably time to run multiple MySQL instances, and distribute your write traffic over them as evenly as you can.
It's tricky to change the log_file_size in that old version. For the steps, see https://dba.stackexchange.com/questions/1261/how-to-safely-change-mysql-innodb-variable-innodb-log-file-size/4103#4103
However, I predict that changing the log_file_size will not help much. "Performance" usually implies a few slow queries. Do you have the slowlog turned on? with a low value of long_query_time? Find the few worst queries; we can probably improve performance by tackling them. Steps on that: http://mysql.rjweb.org/doc.php/mysql_analysis
If you change the innodb_log_file_size parameter, you need to remove the old log files. Otherwise, Innodb won't start successfully if the existing files do not match the specified size in the config file.
On the other hand, The innodb_buffer_pool_size should be set to about 70% of available RAM if you are running InnoDB only.
Recently I updated my VPS from 1GB to 4GB memory. I'd hoped that the queries (MYSQL/InnoDB) were running faster with more memory, but unfortunately that's not the case. Does mysql automatically takes more memory when a server has more memory or do I have to change some settings in my.cnf? And if so, what changes should I make?
MySQL will not automatically take the benefit of more memory installed.
In your case (given that you are using InnoDB) you can do at least these to improve the performance of mysql:
increase innodb_buffer_pool_size (default value for this option is 128MB). This defines how much memory is dedicated to mysql innodb to cache its data tables and idexes. Which means if you can allocate more memory mysql will cache more of its data resulting in faster queries (because mysql will look in memory instead of doing I/O operations for data lookup).
Of course you should allocate reasonable amount of memory (not the whole 4G :)) may be not more than 2G. You should try and test it on the server for more accurate result. (read this for more info, before you change this option https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool-resize.html)
increase innodb_buffer_pool_instances. For you case may be 1 or 2 instances are more than enough. (you can read more here: https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_buffer_pool_instances)
But before starting with editing of my.ini do your calculations for your case. Consider your mysql server load, slow queries etc. for more accurate setup of the options in my.ini
I have a 16GB dedicated Mysql server database.My innodb_buffer_pool_size is set to around 11GB ,i am implementing query cache in my system ,which has a size of 80mb. From where should i make this space ,innodb_buffer_pool_size or RAM ?
Back in Jun 2014 I answered https://dba.stackexchange.com/questions/66774/why-query-cache-type-is-disabled-by-default-start-from-mysql-5-6/66796#66796
In that post, I discussed how InnoDB micromanages changes between the InnoDB Buffer Pool and the Query Cache.
NOT USING THE QUERY CACHE
The simplest answer would be to just disable the query cache, but there is a catch: You must set both query_cache_size and query_cache_type to zero(0).
If you don't set query_cache_type to zero(0), the micromanagement of changes still occurs. This is verified by what the Paragraph 8 of the MySQL Documentation "Query Cache Configuration" says
If query_cache_size is 0, you should also set query_cache_type variable to 0. In this case, the server does not acquire the query cache mutex at all, which means that the query cache cannot be enabled at runtime and there is reduced overhead in query execution.
USING THE QUERY CACHE
If you really wish to use the Query Cache, then you need to study your data. Basically, you need to know the size of your results. You would then set query_cache_limit and query_cache_min_res_unit to accommodate the average size result set as well as the maximum size result set.
As an alternative, you can enable the Query Cache globally but disable its use from your DB Session as stated in Paragraph 9:
Setting the GLOBAL query_cache_type value determines query cache behavior for all clients that connect after the change is made. Individual clients can control cache behavior for their own connection by setting the SESSION query_cache_type value. For example, a client can disable use of the query cache for its own queries like this:
mysql> SET SESSION query_cache_type = OFF;
YOUR ACTUAL QUESTION
The Query Cache, InnoDB Buffer Pool, Per Connection Threads (See my post https://dba.stackexchange.com/questions/16969/how-costly-is-opening-and-closing-of-a-db-connection/16973#16973), and OS all compete for RAM.
When looking at just the Query Cache and the InnoDB Buffer Pool, you need to see how much space is free in the InnoDB Buffer Pool after MySQL has been running for days or weeks.
You can run this the query
SELECT variable_value / 64 free_mb
FROM information_schema.global_status
WHERE variable_name='Innodb_buffer_pool_pages_free';
This will tell you how much you can lower the InnoDB Buffer Pool. You could then increase the Query Cache by that size. If this query returns less that 1024, it is probably not worth the effort. You will either have get the additional space from RAM or just go with disabling the Query Cache.
NO. The Query cache runs out of steam after perhaps 80MB -- it spends more an more time pruning entries.
I would suggest that over 90% of production systems do (or should) turn off the QC completely.
Keep in mind that every write to a table purges all entries in the QC for that table.
11G on a 16GB dedicated server is a good number for innodb_buffer_pool_size.
I have a 16GB dedicated Mysql server database.My innodb_buffer_pool_size is set to around 11GB ,i am implementing query cache in my system ,which has a size of 80mb. From where should i make this space ,innodb_buffer_pool_size or RAM ?
Back in Jun 2014 I answered https://dba.stackexchange.com/questions/66774/why-query-cache-type-is-disabled-by-default-start-from-mysql-5-6/66796#66796
In that post, I discussed how InnoDB micromanages changes between the InnoDB Buffer Pool and the Query Cache.
NOT USING THE QUERY CACHE
The simplest answer would be to just disable the query cache, but there is a catch: You must set both query_cache_size and query_cache_type to zero(0).
If you don't set query_cache_type to zero(0), the micromanagement of changes still occurs. This is verified by what the Paragraph 8 of the MySQL Documentation "Query Cache Configuration" says
If query_cache_size is 0, you should also set query_cache_type variable to 0. In this case, the server does not acquire the query cache mutex at all, which means that the query cache cannot be enabled at runtime and there is reduced overhead in query execution.
USING THE QUERY CACHE
If you really wish to use the Query Cache, then you need to study your data. Basically, you need to know the size of your results. You would then set query_cache_limit and query_cache_min_res_unit to accommodate the average size result set as well as the maximum size result set.
As an alternative, you can enable the Query Cache globally but disable its use from your DB Session as stated in Paragraph 9:
Setting the GLOBAL query_cache_type value determines query cache behavior for all clients that connect after the change is made. Individual clients can control cache behavior for their own connection by setting the SESSION query_cache_type value. For example, a client can disable use of the query cache for its own queries like this:
mysql> SET SESSION query_cache_type = OFF;
YOUR ACTUAL QUESTION
The Query Cache, InnoDB Buffer Pool, Per Connection Threads (See my post https://dba.stackexchange.com/questions/16969/how-costly-is-opening-and-closing-of-a-db-connection/16973#16973), and OS all compete for RAM.
When looking at just the Query Cache and the InnoDB Buffer Pool, you need to see how much space is free in the InnoDB Buffer Pool after MySQL has been running for days or weeks.
You can run this the query
SELECT variable_value / 64 free_mb
FROM information_schema.global_status
WHERE variable_name='Innodb_buffer_pool_pages_free';
This will tell you how much you can lower the InnoDB Buffer Pool. You could then increase the Query Cache by that size. If this query returns less that 1024, it is probably not worth the effort. You will either have get the additional space from RAM or just go with disabling the Query Cache.
NO. The Query cache runs out of steam after perhaps 80MB -- it spends more an more time pruning entries.
I would suggest that over 90% of production systems do (or should) turn off the QC completely.
Keep in mind that every write to a table purges all entries in the QC for that table.
11G on a 16GB dedicated server is a good number for innodb_buffer_pool_size.
We're running a moderate size (350GB) database with some fairly large tables (a few hundred million rows, 50GB) on a reasonably large server (2 x quad-core Xeons, 24GB RAM, 2.5" 10k disks in RAID10), and are getting some pretty slow inserts (e.g. simple insert of a single row taking 90 seconds!).
Our innodb_buffer_pool_size is set to 400MB, which would normally be way too low for this kind of setup. However, our hosting provider advises that this is irrelevant when running on ZFS. Is he right?
(Apologies for the double post on https://dba.stackexchange.com/questions/1975/is-tuning-the-innodb-buffer-pool-size-important-on-solaris-zfs, but I'm not sure how big the audience is over there!)
Your hosting provider is incorrect. There are various things you should tune differently when running MySQL on ZFS, but reducing the innodb_buffer_pool_size is not one of them. I wrote an article on the subject of running MySQL on ZFS and gave a lecture on it a while back. Specifically regarding innodb_buffer_pool_size, what you should do is set it to whatever would be reasonable on any other file system, and because O_DIRECT doesn't mean "don't cache" on ZFS, you should set primarycache=metadata on your ZFS file system containing your datadir. There are other optimisations to be made, which you can find in the article and the lecture slides.
I would still set the innodb_buffer_pool_size much higher that 400M. The reason? InnoDB Buffer Pool will still cache the data and index pages you need for tables accessed frequently.
Run this query to get the recommended innodb_buffer_pool_size in MB:
SELECT CONCAT(ROUND(KBS/POWER(1024,IF(pw<0,0,IF(pw>3,0,pw)))+0.49999),SUBSTR(' KMG',IF(pw<0,0,IF(pw>3,0,pw))+1,1)) recommended_innodb_buffer_pool_size FROM (SELECT SUM(data_length+index_length) KBS FROM information_schema.tables WHERE engine='InnoDB') A,(SELECT 2 pw) B;
Simply use either the result of this query or 80% of installed RAM (in your case 19660M) whichever is smaller.
I would also set the innodb_log_file_size to 25% of the InnoDB Buffer Pool size. Unfortunately, the maximum value of innodb_log_file_size is 2047M. (1M short of 2G) Thus, set innodb_log_file_size to 2047M since 25% of innodb_buffer_pool_size of my recommendated setting is 4915M.
Yet another recommedation is to disable ACID compliance. Use either 0 or 2 for innodb_flush_log_at_trx_commit (default is 1 which support ACID compliance) This will produce faster InnoDB writes AT THE RISK of losing up to 1 second's worth of transactions in the event of a crash.
May be worth reading slow-mysql-inserts if you haven't already. Also this link to the mysql docs on the matter - especially with regards to considering a transaction if you are doing multiple inserts to a large table.
More relevant is this mysql article on performance of innodb and zfs which specifically considers the buffer pool size.
The headline conclusion is;
With InnoDB, the ZFS performance curve suggests a new strategy of "set the buffer pool size low, and let ZFS handle the data buffering."
You may wish to add some more detail such as the number / complexity of the indexes on the table - this can obviously make a big difference.
Apologies for this being rather generic advice rather than from personal experience, I haven't run zfs in anger but hope some of those links might be of use.