Recently I updated my VPS from 1GB to 4GB memory. I'd hoped that the queries (MYSQL/InnoDB) were running faster with more memory, but unfortunately that's not the case. Does mysql automatically takes more memory when a server has more memory or do I have to change some settings in my.cnf? And if so, what changes should I make?
MySQL will not automatically take the benefit of more memory installed.
In your case (given that you are using InnoDB) you can do at least these to improve the performance of mysql:
increase innodb_buffer_pool_size (default value for this option is 128MB). This defines how much memory is dedicated to mysql innodb to cache its data tables and idexes. Which means if you can allocate more memory mysql will cache more of its data resulting in faster queries (because mysql will look in memory instead of doing I/O operations for data lookup).
Of course you should allocate reasonable amount of memory (not the whole 4G :)) may be not more than 2G. You should try and test it on the server for more accurate result. (read this for more info, before you change this option https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool-resize.html)
increase innodb_buffer_pool_instances. For you case may be 1 or 2 instances are more than enough. (you can read more here: https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_buffer_pool_instances)
But before starting with editing of my.ini do your calculations for your case. Consider your mysql server load, slow queries etc. for more accurate setup of the options in my.ini
Related
in mysql there is feature to cache large chunk of database in Memory. Using the mysql configuration file. (my.ini)
Database in RAM InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and row data. The bigger you set this the less disk I/O is needed to access data in tables. On a dedicated database server you may set this parameter up to 80% of the machine physical memory size. Do not set it too large, though, because competition of the physical memory may cause paging in the operating system. Note that on 32bit systems you might be limited to 2-3.5G of user level memory per process, so do not set it too high.
innodb_buffer_pool_size=6000M
Query caching Size in my.ini
Query cache is used to cache SELECT results and later return them without actual executing the same query once again. Having the query cache enabled may result in significant speed improvements, if your have a lot of identical queries and rarely changing tables. See the "Qcache_lowmem_prunes" status variable to check if the current value is high enough for your load.
query_cache_type = 1
query_cache_size = 80M
They drastically boost db performance for a medium scale database.
Do we have similar features in SQL Server?
Imagine we have a MYSQL DB that's data size is 500 MB.
If I will set the innodb_buffer_pool_size at 500MB (or more), is it correct to think that all the data will be cached in RAM, and my queries won't touch disk?
Is effective_cache_size in POSTGRESS is the same as MYSQL's buffer_pool and it also can help avoid reading from disc?
I believe you are on the right track in regards to MySQL innoDB tables. But you must remember that when measuring the size of a database, there are two components: data length and index length.
MySQL database size.
You also have no control over which databases are loaded into memory. If you want to guarantee a particular DB is loaded, then you must make sure the buffer pool is large enough to hold all of them, with some room to spare just in case.
MySQL status variables can then be used to see how the buffer pool is functioning.
I also highly recommend you use the buffer pool load/save variables so that the buffer pool is saved on shutdown and reloaded on startup of the MySQL server. Those variables are available from version 5.6 and up, I believe.
Also, check this out in regards to sizing your buffer pool.
Is "effective_cache_size", a parameter to indicate the planner as to what OS is actually doing ?
http://www.cybertec.at/2013/11/effective_cache_size-better-set-it-right/
and for caching the tables, do we not need to configure "shared_buffers" ?
And with regards to MySQL, yes the "innodb_buffer_pool" size will cache the data for Innodb tables and preventing disc reads. Make sure its configured adequate to hold all the data in memory.
I have a server with 32G, a database of 20G. I run MySQL with innodb_buffer_pool_size of 10G, in trying to improve the performance increasing the value to 20G just slows down the select-queries. Can anybody explain me why that happens?
Try EXPLAIN to get the general idea why the queries are slow.
Try SHOW ENGINE INNODB STATUS to see how buffer pool is used and what your InnoDB engine is doing.
There may be a lot of other factors in play here. Increasing buffer_pool_size you may be getting out of physical RAM (in accord with other configuration options), and then it goes into virtual memory, which is on disk... and disk operations are always slow...
The following is my default production MySQL configuration file (my.cnf) for a pure UTF-8 setup with InnoDB as the default storage engine.
[server]
bind-address=127.0.0.1
innodb_file_per_table
default-character-set=utf8
default-storage-engine=innodb
The setup does the following:
Binds to localhost:3306 (loopback) instead of the default *:3306 (all interfaces). Done to increase security.
Sets up one table space per table. Done to increase maintainability.
Sets the default character set to UTF-8. Done to allow for easy internationalization by default.
Sets the default storage engine to InnoDB. Done to allow for row-level-locking by default.
Assume that you could further improve the setup by adding a maximum of three (3) configuration parameters. Which would you add and why?
An improvement would in this context mean either a performance improvement, a reliability improvement or ease-of-use/ease-of-maintainability increase. You can assume that the machine running the MySQL instance will have 1000 MB of RAM.
To cache more data:
innodb_buffer_pool_size = 512M
If you write lots of data:
innodb_log_file_size = 128M
, to avoid too much log switching.
There is no third I'd add in any case, all other depend.
Allocating more memory than the default of 8M to InnoDB (using innodb_buffer_pool_size) is surely an enhancement. Regarding the value, on a dedicated database server as yours you can set it up to the 80% of your RAM and the higher you set this value, the fewer the interactions with the hard disk will be. Just to give my two cents, I'd like to mention that you can have some performance boost tweaking the value of innodb_flush_log_at_trx_commit, however sacrificing ACID compliance... According to the MySQL manual:
If the value of
innodb_flush_log_at_trx_commit is 0,
the log buffer is written out to the
log file once per second and the flush
to disk operation is performed on the
log file, but nothing is done at a
transaction commit.
So you might loose some data that were not written properly in the database due to a crash or any malfunction. Again according to the MySQL manual:
However, InnoDB's crash recovery is
not affected and thus crash recovery
does work regardless of the value.
So, I would suggest:
innodb_flush_log_at_trx_commit = 0
Finally if you have a high connection rate (i.e. if you need to configure MySQL to support a web application that accesses the database) then you should consider increasing the maximum number of connections to something like 500. But since this is something more or less trivial and well known, so I'd like to emphasize on the importance of back_log to ensure connectivity.
I hope these information will help you optimize your database server.
Increase the innodb buffer pool size, as big as you can practically make it:
innodb_buffer_pool_size=768M
You'll also want some key buffer space for temp tables:
key_buffer_size=32M
Others would depend on what you are doing with the database, but table_cache or query_cache_size would be a couple other potentials.
I am running a mysql server. I would like to somehow make sure that the whole DB is loaded into the ram as I heard it would be alot faster.
Is this true? and how do I vertify it?
If you are using InnoDB, then you can set innodb_buffer_pool_size to be at least as large as your data and indexes. Then as you access data, it will be cached into memory and all subsequent accesses will occur via memory.
If you wanted to pre-cache the data, you could do some of your common queries, or full table/index scans to force it to load all of the data into memory.
There is an init-file option for mysqld which can have the server automatically execute some commands on startup, where you could include the pre-caching queries to execute.
Keep in mind you don't want to run out of memory, so be careful with setting innodb_buffer_pool_size too large.
Well, one option is to use a ramdrive.. though it's not permanent.
http://www.linuxforums.org/forum/misc/670-how-create-ram-drive-linux.html
(Just for completeness,) You could use HEAP as table engine. But keep in mind that whatever data you put in a HEAP table is gone when you reboot the server.
Tables in RAM are very fast.
You can change your existing tables by altering their storage engine. But make a backup if you want to use the data after a server restart.
ALTER TABLE test ENGINE = MEMORY;
Keep in mind, that MEMORY tables cannot contain BLOB or TEXT columns and data and index size is limited to available memory.