I Have a dedicated server - Intel Xeon L5320 with 8GB of RAM and 2 x 500GB 7200RMP HDD
I need to optimize mysql to cope with a large 5Gb MyISAM table + around 25 - 30 smaller databases currently it looks like this:
key_buffer = 3G
thread_cache_size = 16
table_cache = 8192
query_cache_size = 512M
As it is the server really struggles and I get continues tmp disk full warnings could you please help me out / suggest the best my.cnf configuration for my server and or any other settings changes that would improve performance.
Thanks in advance
I recommend you use mytop and mysqltuner to analyze using mysql resources (RAM and CPU).
Too enable the option to log slow queries:
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
And check out this post about ntpd service:
MySQL high CPU usage
Finally I leave you in a setting that I have a dedicated server for a high rate of transactions.
max_allowed_packet=16M
key_buffer_size=8M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=512M
join_buffer_size=40M
table_open_cache=1024
query_cache_size=40M
table_definition_cache=256
innodb_additional_mem_pool_size=10M
key_buffer_size=16M
max_allowed_packet=32M
max_connections = 300
query_cache_limit = 10M
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
Greetings.
If /tmp is filling up, you are running some large, inefficient queries somewhere which are falling back to FILESORT. Well-written, efficient queries should typically not need this -- turn on slow query logging (if it isn't already) and check the log to see what needs optimizing.
Related
DB server
16 cores
63Gb RAM
CentOS release 6.8
etc/my.cnf
[mysqld]
pid_file=/var/lib/mysql/fatty01.pid
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
innodb_buffer_pool_size = 50G
innodb_log_file_size = 2G
innodb_flush_log_at_trx_commit = 0
sync_binlog = 0
innodb_flush_method = O_DIRECT
innodb_buffer_pool_instances = 16
innodb_thread_concurrency = 16
skip_name_resolve = 1
innodb_io_capacity = 4000
innodb_io_capacity_max = 6000
innodb_buffer_pool_dump_at_shutdown = 1
innodb_buffer_pool_load_at_startup = 1
query_cache_size = 0
query_cache_type = OFF
innodb_checksum_algorithm = crc32
table_open_cache_instances = 16
innodb_read_io_threads = 20
innodb_write_io_threads = 10
max_connections = 700
when we have peaks of 3000 concurrent clients the mysqld does not seem
to pull all the resources posibles from the machine.
I see the load at 40 but the cpu does not seem to overpass the 60%
That reflects in the front end server
**
My question is clear, how can I improve the performance without compromising the server? Also how can decrease the MYSQL waiting time in the front end server, since clearly is a problem with the configurations on the DB server side.
**
**
UPDATE After research the problem seem to be in the slow queries, so I
guess this configuration is optimal for this hardware
**
No, it is not likely to be a simple tuning change. As I said, my.cnf looks good -- based on limited information.
Based on the charts, something happened suddenly. Or a flurry of activity.
Turn on the slowlog, set long_query_time=1, wait until the problem happens again, then use pt-query-digest to tell you the naughty query.
Your max_connections is only set to 700. How did you determine that you have 3000 concurrent clients? Site visits can be different from concurrent database connections. You might try increasing the connections available to your clients, as they may be experiencing slowdowns while waiting to connect.
Try checking SHOW PROCESSLIST; during peak usage to see how many connections your server is handling, and look for Too many connections in your mysql error log.
If you do increase your max_connections limit watch your CPU and RAM. MySQL will use more memory with more connections made available to clients.
I want to increase the performance of MySQL. So I have done the configuration level changes to MySQL. I used innodb_flush_method = O_DIRECT, but insert rate is not increasing much. Normally, insertion rate is 650 inserts/sec. How do I know weather O_DIRECT is working properly.
I am using Ubuntu 14.04.1 server and MySQL v5.6. CPU Memory and Disk I/O rates are normal (I use RAID, 16 GB RAM, 8 CPU cores) I use WSO2 CEP for insertion. I have implement that part and measured using MySQL workbench. But I couldn't get much more performance though I increase the insertion rate through wSO2 CEP.
I have used following my.cnf.
my.cnf
[mysqld]
innodb_buffer_pool_size = 9G
query_cache_size = 128M
innodb_log_file_size = 1768M
innodb_flush_log_at_trx_commit = 0
innodb_io_capacity = 1000
innodb_flush_method = O_DIRECT
max_heap_table_size = 536870912
innodb_lock_wait_timeout = 1
max_connections = 400
sort_buffer_size = 128M
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
skip-host-cache
skip-name-resolve
event_scheduler=on
In this case if you are using Event tables, older CEP/siddhi version does not perform batch insertions.. That could be the cause for above.. In latest SNAPSHOT source (of Siddhi) we have fixed this.. And you can gain considerably good numbers in next release..
I have VPS, CentOS 6 64bit with directadmin + custombuild.
Web server: Apache with reverse proxy by Nginx.
I am hosting a lot of websites on this VPS.
when there are a lot of hits on website "example.com", that website becoming extremely slow (can be 50 seconds and more), but the others work perfect and fast.
I checked CPU and Memory, but there are nothing weird.
I installed "mytop" for monitoring database, and it happened when there are 20+- running queries.
my.cnf content:
[mysqld]
max_allowed_packet=16M
innodb_buffer_pool_size = 5096M
innodb_buffer_pool_instances = 12
innodb_file_per_table = 1
innodb_log_file_size = 64M
innodb_log_files_in_group = 2
innodb_log_buffer_size = 10M
innodb_flush_log_at_trx_commit = 0
innodb_buffer_pool_load_at_startup = 1
innodb_log_buffer_size = 8
innodb_thread_concurrency = 12
innodb_flush_method = O_DIRECT
innodb_read_io_threads = 4
innodb_write_io_threads = 8
#max_connections = 800
#max_user_connections = 400
local-infile=0
max_connections=120 #
interactive_timeout=300
join_buffer_size=512K
key_buffer_size=64M
query_cache_limit=4G
tmp_table_size=1024M
max_heap_table_size=512M #
thread_cache_size=4
open_files_limit=50000
table_open_cache=3000
query_cache_type=1
query_cache_size=128M #
I dont know if its mysql problem or apache and nginx problem.
The information you give is pretty sparse. You should probably check the general cpu usage and see what applications use the cpu. However if the other applications on the server are still performing well then you probably have some slow database queries. 20+ sounds pretty much depending on what you call lots of hits. You should probably analyse which database queries are running and how long they take. Maybe you need to add indexes.
Another problem could be locks in the database.
I have a Drupal application which has been running on a single MySQL database server for 12 months, and has been performing relatively well (apart from peak load events). We needed to be able to support much higher spikes than the current DB server allowed, and at 32GB there was not much gain to be had from simply vertically scaling the single DB server.
We decided to set up a new MariaDB Galera cluster with 2x 32GB instances. We matched the configuration as far as possible with the soon-to-be-obselete DB server.
After migrating to the new database servers, we noticed that the CPU usage on those instances was constantly at 100%, and load was steadily increasing. Over the course of 1 hour, load average went from 0.1 to 150.
Initially we thought it might have something to do with the synchronisation between servers, but even with 1 server turned off and no sync occurring the it was still maxing out CPU as long as the web application was making requests to it.
After a lot of experimentation I found that reducing a few of the configuration options had a profound effect on the CPU usage and load. After making the below changes, the load average has stabilised between 4 and 6 on both instances.
The questions
What are some possible reasons for such a dramatic difference in CPU usage between the old and new servers, despite essentially migrating the configuration from the old server?
Load is currently hovering between 4 and 6 (and this is a low traffic period for our website). What should I be looking at to try and reduce this value, and ensure that when the site gets hit with some real traffic it wont fall over?
Config changes
innodb_buffer_pool_instances
Original value: 500 (there are 498 tables total in all databases)
New value: 92
table_cache
Original value: 8
New value: 4
max_connections
Original value: 1000
New value: 400
Current configuration
Here is the full configuration file from one of the servers /etc/mysql/my.cnf
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
query_cache_type=1
bind-address=0.0.0.0
max_connections = 400
wait_timeout = 600
key_buffer_size = 16M
max_allowed_packet = 16777216
max_heap_table_size = 512M
table_cache = 92
thread_stack = 196608
thread_cache_size = 8
myisam-recover = BACKUP
query_cache_limit = 1048576
query_cache_size = 128M
expire_logs_days = 10
general_log = 0
max_binlog_size = 10485760
server-id = 0
innodb_file_per_table
innodb_buffer_pool_size = 25G
innodb_buffer_pool_instances = 4
innodb_log_buffer_size = 8388608
innodb_additional_mem_pool_size = 8388608
innodb_thread_concurrency = 16
net_buffer_length = 16384
sort_buffer_size = 2097152
myisam_sort_buffer_size = 8388608
read_buffer_size = 131072
join_buffer_size = 131072
read_rnd_buffer_size = 262144
tmp_table_size = 512M
long_query_time = 1
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
# Galera Provider Configuration
wsrep_provider=/usr/lib/galera/libgalera_smm.so
#wsrep_provider_options="gcache.size=32G"
# Galera Cluster Configuration
wsrep_cluster_name="xxx"
wsrep_cluster_address="gcomm://xxx.xxx.xxx.107,xxx.xxx.xxx.108"
# Galera Synchronization Congifuration
wsrep_sst_method=rsync
#wsrep_sst_auth=user:pass
# Galera Node Configuration
wsrep_node_address="xxx.xxx.xxx.107"
wsrep_node_name="xxx01"
[mysqldump]
quick
quote-names
max_allowed_packet = 16777216
[isamchk]
key_buffer_size = 16777216
We ended up getting a Percona consultant to assist with this problem. The main issue they identified was a large number of EXPLAIN queries were being executed. Turns out this was some debugging code that was left enabled (devel.module query logging for drupal devs). Disabling this saw CPU usage fall off a cliff.
There were a number of additional fixes which they recommended we implement.
Add a third node to the cluster to act as an observer and maintain the integrity of the cluster.
Add primary keys to tables that do not have one.
Change MyISAM tables to InnoDB.
Change wsrep_sst_method from rsync to xtrabackup-v2.
Set innodb_log_file_size to 512M.
Set innodb_flush_log_at_trx_commit to 2 as the cluster maintains the integrity of the data.
I hope this information helps anyone who runs into similar issues.
innodb_buffer_pool_instances should not be a function of the number of tables. The manual advocates that each instance be no smaller than 1GB. So, I suggest that even 92 is much too high. But my.cnf says only innodb_buffer_pool_instances = 4??
table_cache = 92
Maybe your comments are messed up? 500 would be more reasonable for table_open_cache. (table_cache is the old name.)
This may be the problem:
query_cache_size = 128M
Whenever a write occurs, all entries in the QC for the table(s) involved are purged from the QC. Recommend no more than 50M. Or, better yet, turn the QC off completely.
You have the slowlog turned on. What does pt-query-digest say are the top couple of queries? (This may be your best way to get a handle on the problem.)
I have what seems to be a slowing MySQL restore, and am looking for some tuning advice (I am a PostgreSQL and SQL Server guy).
The dev server has 48GB of RAM, 8 cores, running Centos 6.2 64-bit and MySQL 5.1.61 (same as production MySQL), and 4 x 7200 RPM SAS drives in software managed RAID-10 / XFS. The only MySQL client process is the restore. The dump was taken with a plain mysqldump of all databases on the production server.
I have applied some of the options from http://derwiki.tumblr.com/post/24490758395/loading-half-a-billion-rows-into-mysql, including setting FOREIGN_KEY_CHECKS and UNIQUE_CHECKS to zero. I have included my.cnf below.
Monitoring the restore with mytop and pv (pv backup.sql | mysql -u root -p), it appears that the INSERT INTO statements begin to progressively get slower. qps shown by mytop starts at 3, and drops to 0 at 60% through the dump file. Not sure how accurate mytop is in this case, as 3 inserts (with values) still seems slow. htop shows < 10% CPU utilization on the CPU used by MySQL, and less than 8GB of the 48GB of RAM is being utilized.
Different databases, but similar restore techniques, run about 5-10x faster on the same server using PostgreSQL.
Ideas?
[mysqld]
# my.cnf
socket=/var/lib/mysql/mysql.sock
user=mysql
symbolic-links=0
slow-query-log
long_query_time = 60
log-slow-admin-statements
slow_query_log_file = /var/log/mysql_slow.log
innodb_buffer_pool_size = 2G
max_allowed_packet = 1G
key_buffer_size = 1G
concurrent_insert = 1
innodb_flush_log_at_trx_commit = 2
bulk_insert_buffer_size = 1G
innodb_flush_method = O_DIRECT
Sounds like your innodb indexes are slowing you down. If can change the way you dump the database you can remove all non-primary key indexes load the data then re-add them. Better still order the data to be loaded by the primary key. This is probably too much to ask.
Sounds like you are already aware of these tips: http://dev.mysql.com/doc/refman/5.5/en/optimizing-innodb-bulk-data-loading.html
The flush to disk operation (innodb_flush_log_at_trx_commit = 2) may be happening many times a second. Check your innodb_log_file_size * innodb_log_files_in_group is sufficient to avoid writing to disk too often.
(I assumed you are using Innodb from your settings)