I'm trying to configure mysql on an Amazon EC2 micro instance with 613MB memory. This instance will only be used for running the mysql, so i want to use as much memory as possible. We have another instance of the same DB running on a different host, so i can easily compare results.
Performing a mediocre query on the original db took less than 3 seconds. Before my changes on the EC2, it took 46 seconds, but now, after changing the settings, the same query only took 4 seconds. However, executing the same query once more, it seems to take forever.
This are the settings i use in my MySql my.cnf:
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
innodb_buffer_pool_size = 128M
query_cache_size = 128M
query_cache_limit = 8M
key_buffer_size = 128M
table_open_cache = 2048
table_definition_cache = 2048
read_buffer_size = 64M
join_buffer_size = 64M
sort_buffer_size = 64M
myisam_sort_buffer_size = 64M
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
I don't think the myisam param should be included tho, since it should all be using innodb, but i just gave it some extra memory just in case it is needed.
It would be logical if it was slower at first run, and after that go faster, but this just seems illogical.
Any idea's would be very welcome.
EC2 micro has a generally terrible performance. You could get Linode/Rackspace for the same price.
Amazon Micro instances "may opportunistically increase CPU capacity in short bursts when additional cycles are available."
This means you're allocated X units of computation but it can jump to X + N when other Micro instances are not using their allocation and your instance will automatically scavenge it.
This is likely why you are seeing fluctuations in performance.
Related
I have deployed 64 core 300 GB+ RAM Amazon server and installed virtualmin on it. This parent server is to be used as a database server. It stores a Laravel jobs queue with more than 2,000,000 jobs.
I am having trouble configuring MySql server in the parent server . Currently the parent server is connected to AWS Auto Scaling Group with many child servers. The child servers in Scaling Group read data from parent server, process the data and store the result back in parent server. Each server, on average, completes 3 jobs from the parent server.
I want to connect about 1000 child server. The problem arises when there are more than 500 server connected. At that time, the MYSql server in parent server becomes very slow. The child server receive data very slowly. It doesnt crash or give connection limit error.
I have tried various variable settings and increased limits, but so far I am unable to solve the issue, My current /etc/my.cnf config is as below:
symbolic-links=0
innodb_file_per_table = 1
myisam_sort_buffer_size = 64M
read_rnd_buffer_size = 32M
net_buffer_length = 12M
read_buffer_size = 128M
sort_buffer_size = 128M
table_open_cache = 64
max_allowed_packet = 5M
key_buffer_size = 512M
max_connections = 100000
innodb_buffer_pool_size=64G
tmp_table_size= 4095M
max_heap_table_size = 20G
Rest of the settings are default.
myisam_sort_buffer_size = 64M
key_buffer_size = 512M
These settings are for MyISAM only. You shouldn't be using MyISAM. It's not the '90s any more.
table_open_cache = 64
This is absurdly low. It is almost certainly why everything grinds to a halt when you have many machines connected.
tmp_table_size= 4095M
max_heap_table_size = 20G
Are you really going to have temporary tables of anywhere near that size?
innodb_buffer_pool_size=64G
How big is your data? If you are anywhere near justifying 300GB of RAM, I presume you must have about a terabyte of data. Unless you have much less data than the server size would imply, this should probably be more like 250GB (and tmp and heap table sizes should be much smaller).
sort_buffer_size = 128M
This is absurdly high. sort_buffer_size is one of many settings that you almost certainly shouldn't be touching.
If you aren't 100% sure you know what each setting does, you should leave it at defaults. What you have here is guaranteed to completely cripple your immensely powerful server.
For an social sciences research project I'm using MySQL (5.5) on a dedicated Linux server with 8 GB of memory. The data consists of some 30 million records, resulting in a MyISAM source table of about 4GB (with MyISAM since the data is stable and transaction are not useful). My question is this: how can I prevent memory from being an unnecessary bottleneck?
At the current settings only about 20% of memory is ever used, but the right balance of my.ini settings is difficult to find, since many variables are interdependent. How can I allow MySQL from using as much memory as possible (reserving enough to prevent Linux from swapping out).
current settings:
[mysqld]
max_connections = 3
performance_schema=on
default-storage-engine=MYISAM
local-infile=1
myisam_sort_buffer_size = 2048M
key_buffer_size = 2048M
tmp_table_size = 2048M
max_heap_table_size = 2048M
sort_buffer_size = 128M
read_buffer_size = 256M
read_rnd_buffer_size = 128M
join_buffer_size = 512M
thread_stack = 256KB
query_cache_size = 64M
query_cache_limit = 32M
table_open_cache= 256
table_definition_cache = 512
myisam_max_sort_file_size = 75G
Your best bet is to run MySQL Tuner on your database after it been running and in use for a while. No one on SO will be able to give you relevant information for optimizing your memory usage without knowing the shape of your database and usage patterns. MySQL Tuner automates this.
I want to increase the performance of MySQL. So I have done the configuration level changes to MySQL. I used innodb_flush_method = O_DIRECT, but insert rate is not increasing much. Normally, insertion rate is 650 inserts/sec. How do I know weather O_DIRECT is working properly.
I am using Ubuntu 14.04.1 server and MySQL v5.6. CPU Memory and Disk I/O rates are normal (I use RAID, 16 GB RAM, 8 CPU cores) I use WSO2 CEP for insertion. I have implement that part and measured using MySQL workbench. But I couldn't get much more performance though I increase the insertion rate through wSO2 CEP.
I have used following my.cnf.
my.cnf
[mysqld]
innodb_buffer_pool_size = 9G
query_cache_size = 128M
innodb_log_file_size = 1768M
innodb_flush_log_at_trx_commit = 0
innodb_io_capacity = 1000
innodb_flush_method = O_DIRECT
max_heap_table_size = 536870912
innodb_lock_wait_timeout = 1
max_connections = 400
sort_buffer_size = 128M
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
skip-host-cache
skip-name-resolve
event_scheduler=on
In this case if you are using Event tables, older CEP/siddhi version does not perform batch insertions.. That could be the cause for above.. In latest SNAPSHOT source (of Siddhi) we have fixed this.. And you can gain considerably good numbers in next release..
I have a Drupal application which has been running on a single MySQL database server for 12 months, and has been performing relatively well (apart from peak load events). We needed to be able to support much higher spikes than the current DB server allowed, and at 32GB there was not much gain to be had from simply vertically scaling the single DB server.
We decided to set up a new MariaDB Galera cluster with 2x 32GB instances. We matched the configuration as far as possible with the soon-to-be-obselete DB server.
After migrating to the new database servers, we noticed that the CPU usage on those instances was constantly at 100%, and load was steadily increasing. Over the course of 1 hour, load average went from 0.1 to 150.
Initially we thought it might have something to do with the synchronisation between servers, but even with 1 server turned off and no sync occurring the it was still maxing out CPU as long as the web application was making requests to it.
After a lot of experimentation I found that reducing a few of the configuration options had a profound effect on the CPU usage and load. After making the below changes, the load average has stabilised between 4 and 6 on both instances.
The questions
What are some possible reasons for such a dramatic difference in CPU usage between the old and new servers, despite essentially migrating the configuration from the old server?
Load is currently hovering between 4 and 6 (and this is a low traffic period for our website). What should I be looking at to try and reduce this value, and ensure that when the site gets hit with some real traffic it wont fall over?
Config changes
innodb_buffer_pool_instances
Original value: 500 (there are 498 tables total in all databases)
New value: 92
table_cache
Original value: 8
New value: 4
max_connections
Original value: 1000
New value: 400
Current configuration
Here is the full configuration file from one of the servers /etc/mysql/my.cnf
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
query_cache_type=1
bind-address=0.0.0.0
max_connections = 400
wait_timeout = 600
key_buffer_size = 16M
max_allowed_packet = 16777216
max_heap_table_size = 512M
table_cache = 92
thread_stack = 196608
thread_cache_size = 8
myisam-recover = BACKUP
query_cache_limit = 1048576
query_cache_size = 128M
expire_logs_days = 10
general_log = 0
max_binlog_size = 10485760
server-id = 0
innodb_file_per_table
innodb_buffer_pool_size = 25G
innodb_buffer_pool_instances = 4
innodb_log_buffer_size = 8388608
innodb_additional_mem_pool_size = 8388608
innodb_thread_concurrency = 16
net_buffer_length = 16384
sort_buffer_size = 2097152
myisam_sort_buffer_size = 8388608
read_buffer_size = 131072
join_buffer_size = 131072
read_rnd_buffer_size = 262144
tmp_table_size = 512M
long_query_time = 1
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
# Galera Provider Configuration
wsrep_provider=/usr/lib/galera/libgalera_smm.so
#wsrep_provider_options="gcache.size=32G"
# Galera Cluster Configuration
wsrep_cluster_name="xxx"
wsrep_cluster_address="gcomm://xxx.xxx.xxx.107,xxx.xxx.xxx.108"
# Galera Synchronization Congifuration
wsrep_sst_method=rsync
#wsrep_sst_auth=user:pass
# Galera Node Configuration
wsrep_node_address="xxx.xxx.xxx.107"
wsrep_node_name="xxx01"
[mysqldump]
quick
quote-names
max_allowed_packet = 16777216
[isamchk]
key_buffer_size = 16777216
We ended up getting a Percona consultant to assist with this problem. The main issue they identified was a large number of EXPLAIN queries were being executed. Turns out this was some debugging code that was left enabled (devel.module query logging for drupal devs). Disabling this saw CPU usage fall off a cliff.
There were a number of additional fixes which they recommended we implement.
Add a third node to the cluster to act as an observer and maintain the integrity of the cluster.
Add primary keys to tables that do not have one.
Change MyISAM tables to InnoDB.
Change wsrep_sst_method from rsync to xtrabackup-v2.
Set innodb_log_file_size to 512M.
Set innodb_flush_log_at_trx_commit to 2 as the cluster maintains the integrity of the data.
I hope this information helps anyone who runs into similar issues.
innodb_buffer_pool_instances should not be a function of the number of tables. The manual advocates that each instance be no smaller than 1GB. So, I suggest that even 92 is much too high. But my.cnf says only innodb_buffer_pool_instances = 4??
table_cache = 92
Maybe your comments are messed up? 500 would be more reasonable for table_open_cache. (table_cache is the old name.)
This may be the problem:
query_cache_size = 128M
Whenever a write occurs, all entries in the QC for the table(s) involved are purged from the QC. Recommend no more than 50M. Or, better yet, turn the QC off completely.
You have the slowlog turned on. What does pt-query-digest say are the top couple of queries? (This may be your best way to get a handle on the problem.)
I Have a dedicated server - Intel Xeon L5320 with 8GB of RAM and 2 x 500GB 7200RMP HDD
I need to optimize mysql to cope with a large 5Gb MyISAM table + around 25 - 30 smaller databases currently it looks like this:
key_buffer = 3G
thread_cache_size = 16
table_cache = 8192
query_cache_size = 512M
As it is the server really struggles and I get continues tmp disk full warnings could you please help me out / suggest the best my.cnf configuration for my server and or any other settings changes that would improve performance.
Thanks in advance
I recommend you use mytop and mysqltuner to analyze using mysql resources (RAM and CPU).
Too enable the option to log slow queries:
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
And check out this post about ntpd service:
MySQL high CPU usage
Finally I leave you in a setting that I have a dedicated server for a high rate of transactions.
max_allowed_packet=16M
key_buffer_size=8M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=512M
join_buffer_size=40M
table_open_cache=1024
query_cache_size=40M
table_definition_cache=256
innodb_additional_mem_pool_size=10M
key_buffer_size=16M
max_allowed_packet=32M
max_connections = 300
query_cache_limit = 10M
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 3
Greetings.
If /tmp is filling up, you are running some large, inefficient queries somewhere which are falling back to FILESORT. Well-written, efficient queries should typically not need this -- turn on slow query logging (if it isn't already) and check the log to see what needs optimizing.