I am currently debugging some performance issues related to my cloudsql mySQL database, and am looking to investigate its configuration parameters.
In the cloud sql docs the following note can be found:
"Maximum concurrent connections - Note:
To determine the maximum value that you can set for this flag, the
basic formulas are:
Available RAM = Global Buffers + (Thread Buffers x max_connections)
max_connections = (Available RAM - Global Buffers) / Thread Buffers
To get the list of buffers and their values: SHOW VARIABLES LIKE '%buffer%'; Allow some headroom for other processes requiring
memory."
However, running the sql command produces the following output:
mysql> SHOW VARIABLES LIKE '%buffer%';
+-------------------------------------+----------------+
| Variable_name | Value |
+-------------------------------------+----------------+
| bulk_insert_buffer_size | 8388608 |
| innodb_buffer_pool_chunk_size | 55574528 |
| innodb_buffer_pool_dump_at_shutdown | ON |
| innodb_buffer_pool_dump_now | OFF |
| innodb_buffer_pool_dump_pct | 25 |
| innodb_buffer_pool_filename | ib_buffer_pool |
| innodb_buffer_pool_in_core_file | ON |
| innodb_buffer_pool_instances | 1 |
| innodb_buffer_pool_load_abort | OFF |
| innodb_buffer_pool_load_at_startup | ON |
| innodb_buffer_pool_load_now | OFF |
| innodb_buffer_pool_size | 55574528 |
| innodb_change_buffer_max_size | 25 |
| innodb_change_buffering | all |
| innodb_log_buffer_size | 16777216 |
| innodb_sort_buffer_size | 1048576 |
| join_buffer_size | 262144 |
| key_buffer_size | 8388608 |
| myisam_sort_buffer_size | 8388608 |
| net_buffer_length | 16384 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 262144 |
| sql_buffer_result | OFF |
+-------------------------------------+----------------+
25 rows in set (0.02 sec)
I do not understand which values to plug into the formula. Which values correspond to Global Buffers and Thread Buffers?
I found this blog article that lists which are the Global Buffer and Thread Buffer variables, which are:
Global Buffer:
key_buffer_size
innodb_buffer_pool_size
innodb_log_buffer_size
innodb_additional_mem_pool_size
net_buffer_size
query_cache_size
Thread Buffer:
sort_buffer_size
myisam_sort_buffer_size
read_buffer_size
join_buffer_size
read_rnd_buffer_size
thread_stack
Related
I did following changes for caching mysql queries in server
SET GLOBAL query_cache_size = 8388608;
SET GLOBAL query_cache_limit = 1048576;
SET GLOBAL query_cache_type = 1;
and I am getting following result in mysql
mysql> show variables like 'query%';
+------------------------------+---------+
| Variable_name | Value |
+------------------------------+---------+
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 8388608 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
+------------------------------+---------+
Is there anything I need to change in my php code to achieve caching for all the mysql queries ? Please check this
I have a server on Debian 6 using Mysql 5.1 and I'm having problems with big tables on Innodb. As Mysql was installed on its default configurations I'd like to know some suggestion about how to improve the performance of my server.
DB Mysql 5.1
Operative System: Debian6
I have 4G available on free memory according free -g command on Debian.
I have 6GB on swap on Debian OS.
mysql> SHOW VARIABLES LIKE '%buffer%';
+-------------------------+------------+
| Variable_name | Value |
+-------------------------+------------+
| bulk_insert_buffer_size | 8388608 |
| innodb_buffer_pool_size | 2097152000 |
| innodb_log_buffer_size | 1048576 |
| join_buffer_size | 131072 |
| key_buffer_size | 16777216 |
| myisam_sort_buffer_size | 8388608 |
| net_buffer_length | 16384 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 2097144 |
| sql_buffer_result | OFF |
+-------------------------+------------+
12 rows in set
I have to mention that the only thing I have changed on my cnf file is innodb_buffer_pool_size to 2097152000 but I have no get good result about performance.
By the way on my server I have big tables with partitions and big tables on MyIsam as well.
Dose any one knows why information_schema.tables.data_free in InnoDB always be 8388608, no matter how many rows in tables;
table_schema table_name table_rows data_free engine
g33v1 appraise 0 8388608 InnoDB
g33v1 areatype 12403 8388608 InnoDB
g33v1 atype 581982 8388608 InnoDB
g33v1 atype2 579700 8388608 InnoDB
thanks.
I think it is the temporary table maximum size allocated for doing sorting and other things which requires the space on HDD. As the same things appears in other cases also
mysql> show global variables like '%tmp%';
+----------------+-----------------------+
| Variable_name | Value |
+----------------+-----------------------+
| bdb_tmpdir | /usr/local/mysql/tmp/ |
| max_tmp_tables | 32 |
| tmp_table_size | 8388608 |
| tmpdir | /usr/local/mysql/tmp |
+----------------+-----------------------+
mysql> show global variables like '%myisam%';
+---------------------------------+---------------+
| Variable_name | Value |
+---------------------------------+---------------+
| myisam_data_pointer_size | 4 |
| myisam_max_extra_sort_file_size | 2147483648 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_recover_options | OFF |
| myisam_repair_threads | 1 |
| myisam_sort_buffer_size | 4194304 |
| myisam_stats_method | nulls_unequal |
+---------------------------------+---------------
I want to check the log in MySQL to see the queries that are being run by my application. How can I do this? I am using XAMPP and the directory to MySQL is C:\xampp\mysql.
This is what I get when I do show variables like '%log%';
mysql> show variables like '%log%';
+---------------------------------+------------------------------------+
| Variable_name | Value |
+---------------------------------+------------------------------------+
| back_log | 50 |
| binlog_cache_size | 32768 |
| binlog_format | STATEMENT |
| expire_logs_days | 0 |
| general_log | OFF |
| general_log_file | C:/xampp/mysql/data/mysql.log |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_locks_unsafe_for_binlog | OFF |
| innodb_log_buffer_size | 8388608 |
| innodb_log_file_size | 5242880 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | C:\xampp\mysql\data\ |
| innodb_mirrored_log_groups | 1 |
| log | OFF |
| log_bin | OFF |
| log_bin_trust_function_creators | OFF |
| log_bin_trust_routine_creators | OFF |
| log_error | C:\xampp\mysql\data\mysql.err |
| log_output | FILE |
| log_queries_not_using_indexes | OFF |
| log_slave_updates | OFF |
| log_slow_queries | OFF |
| log_warnings | 1 |
| max_binlog_cache_size | 4294963200 |
| max_binlog_size | 1073741824 |
| max_relay_log_size | 0 |
| relay_log | |
| relay_log_index | |
| relay_log_info_file | relay-log.info |
| relay_log_purge | ON |
| relay_log_space_limit | 0 |
| slow_query_log | OFF |
| slow_query_log_file | C:/xampp/mysql/data/mysql-slow.log |
| sql_log_bin | ON |
| sql_log_off | OFF |
| sql_log_update | ON |
| sync_binlog | 0 |
+---------------------------------+------------------------------------+
37 rows in set (0.00 sec)
Enable general query log by the following query in mysql command line
SET GLOBAL general_log = 'ON';
Now open C:/xampp/mysql/data/mysql.log and check query log
If it fails, open your my.cnf file. For windows its my.ini file and enable it there. Just make sure its in the [mysqld] section
[mysqld]
general_log = 1
Note: In xampp my.ini file can be either found in xampp\mysql or in c:\windows directory
Set up the General Query Log.
Its given on OFFICIAL MYSQL website.
SET GLOBAL general_log = 'ON';
You can also use custom path:
[mysqld]
# Set Slow Query Log
long_query_time = 1
slow_query_log = 1
slow_query_log_file = "C:/slowquery.log"
#Set General Log
log = "C:/genquery.log"
Seems like the general query log is the file that you need. A good introduction to this is at http://dev.mysql.com/doc/refman/5.1/en/query-log.html
For me, general_log didn't worked. But adding this to my.ini worked
[mysqld]
log-output=FILE
slow_query_log = 1
slow_query_log_file = "d:/temp/developer.log"
My mysql only caters read requests. I thought, it will be a good idea to make use of the cache completely. I am running MySQL in a VM and that is the only application running inside the VM. I am allocating 2GB memory for that VM. I am using a 64 bit centos on the VM. If you think already it is using the maximum memory that can be used, I can also allocate more to that VM. I am not very good in understanding the mysql settings and finding out the memory footprint used by a process but I am interested in learning how to. Thanks a lot for the help.
These are some information regarding my MYSQL :
mysql> show global variables like "%cache%";
+------------------------------+----------------------+
| Variable_name | Value |
+------------------------------+----------------------+
| binlog_cache_size | 32768 |
| have_query_cache | YES |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| max_binlog_cache_size | 18446744073709547520 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 0 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| table_cache | 64 |
| thread_cache_size | 0 |
+------------------------------+----------------------+
13 rows in set (0.00 sec)
mysql> show global variables like "%buffer%";
+-------------------------------+---------+
| Variable_name | Value |
+-------------------------------+---------+
| bulk_insert_buffer_size | 8388608 |
| innodb_buffer_pool_awe_mem_mb | 0 |
| innodb_buffer_pool_size | 8388608 |
| innodb_log_buffer_size | 1048576 |
| join_buffer_size | 131072 |
| key_buffer_size | 8384512 |
| myisam_sort_buffer_size | 8388608 |
| net_buffer_length | 16384 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 2097144 |
+-------------------------------+---------+
12 rows in set (0.00 sec)
mysql> show table status where name="items"
-> ;
+-------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-------------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |
+-------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-------------------+----------+----------------+---------+
| items | MyISAM | 10 | Dynamic | 42667 | 346 | 14775916 | 281474976710655 | 1970176 | 0 | 341337 | 2009-07-22 13:31:00 | 2010-10-20 15:37:18 | NULL | latin1_swedish_ci | NULL | | |
+-------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-------------------+----------+----------------+---------+
This is the output of my ulimit -a
[sethu#work13 root]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 8191
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 8191
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Please let me know if you need more information.
You can achieve a quick performance boost by enabling the query cache!
Add this to your my.cnf:
query_cache_type=1
query_cache_limit=1M
query_cache_size=32M
For basic statistics and recommendations you can start with mysqltuner.pl script. Do not apply recommendations blindly, as it might decrease performance.
MySQLTuner Github page
One-liner to fetch latest version of script and run it:
curl -sSL mysqltuner.pl | perl
Perhaps you could consider using memcached? However, going to InnoDB table should give an immediate improvement (as N.B. said in a comment)