Increasing the caching capability of MySQL - mysql

My mysql only caters read requests. I thought, it will be a good idea to make use of the cache completely. I am running MySQL in a VM and that is the only application running inside the VM. I am allocating 2GB memory for that VM. I am using a 64 bit centos on the VM. If you think already it is using the maximum memory that can be used, I can also allocate more to that VM. I am not very good in understanding the mysql settings and finding out the memory footprint used by a process but I am interested in learning how to. Thanks a lot for the help.
These are some information regarding my MYSQL :
mysql> show global variables like "%cache%";
+------------------------------+----------------------+
| Variable_name | Value |
+------------------------------+----------------------+
| binlog_cache_size | 32768 |
| have_query_cache | YES |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| max_binlog_cache_size | 18446744073709547520 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 0 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| table_cache | 64 |
| thread_cache_size | 0 |
+------------------------------+----------------------+
13 rows in set (0.00 sec)
mysql> show global variables like "%buffer%";
+-------------------------------+---------+
| Variable_name | Value |
+-------------------------------+---------+
| bulk_insert_buffer_size | 8388608 |
| innodb_buffer_pool_awe_mem_mb | 0 |
| innodb_buffer_pool_size | 8388608 |
| innodb_log_buffer_size | 1048576 |
| join_buffer_size | 131072 |
| key_buffer_size | 8384512 |
| myisam_sort_buffer_size | 8388608 |
| net_buffer_length | 16384 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 2097144 |
+-------------------------------+---------+
12 rows in set (0.00 sec)
mysql> show table status where name="items"
-> ;
+-------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-------------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |
+-------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-------------------+----------+----------------+---------+
| items | MyISAM | 10 | Dynamic | 42667 | 346 | 14775916 | 281474976710655 | 1970176 | 0 | 341337 | 2009-07-22 13:31:00 | 2010-10-20 15:37:18 | NULL | latin1_swedish_ci | NULL | | |
+-------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-------------------+----------+----------------+---------+
This is the output of my ulimit -a
[sethu#work13 root]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 8191
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 8191
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Please let me know if you need more information.

You can achieve a quick performance boost by enabling the query cache!
Add this to your my.cnf:
query_cache_type=1
query_cache_limit=1M
query_cache_size=32M

For basic statistics and recommendations you can start with mysqltuner.pl script. Do not apply recommendations blindly, as it might decrease performance.
MySQLTuner Github page
One-liner to fetch latest version of script and run it:
curl -sSL mysqltuner.pl | perl

Perhaps you could consider using memcached? However, going to InnoDB table should give an immediate improvement (as N.B. said in a comment)

Related

How to determine Global and Thread Buffer size in MySQL?

I am currently debugging some performance issues related to my cloudsql mySQL database, and am looking to investigate its configuration parameters.
In the cloud sql docs the following note can be found:
"Maximum concurrent connections - Note:
To determine the maximum value that you can set for this flag, the
basic formulas are:
Available RAM = Global Buffers + (Thread Buffers x max_connections)
max_connections = (Available RAM - Global Buffers) / Thread Buffers
To get the list of buffers and their values: SHOW VARIABLES LIKE '%buffer%'; Allow some headroom for other processes requiring
memory."
However, running the sql command produces the following output:
mysql> SHOW VARIABLES LIKE '%buffer%';
+-------------------------------------+----------------+
| Variable_name | Value |
+-------------------------------------+----------------+
| bulk_insert_buffer_size | 8388608 |
| innodb_buffer_pool_chunk_size | 55574528 |
| innodb_buffer_pool_dump_at_shutdown | ON |
| innodb_buffer_pool_dump_now | OFF |
| innodb_buffer_pool_dump_pct | 25 |
| innodb_buffer_pool_filename | ib_buffer_pool |
| innodb_buffer_pool_in_core_file | ON |
| innodb_buffer_pool_instances | 1 |
| innodb_buffer_pool_load_abort | OFF |
| innodb_buffer_pool_load_at_startup | ON |
| innodb_buffer_pool_load_now | OFF |
| innodb_buffer_pool_size | 55574528 |
| innodb_change_buffer_max_size | 25 |
| innodb_change_buffering | all |
| innodb_log_buffer_size | 16777216 |
| innodb_sort_buffer_size | 1048576 |
| join_buffer_size | 262144 |
| key_buffer_size | 8388608 |
| myisam_sort_buffer_size | 8388608 |
| net_buffer_length | 16384 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 262144 |
| sql_buffer_result | OFF |
+-------------------------------------+----------------+
25 rows in set (0.02 sec)
I do not understand which values to plug into the formula. Which values correspond to Global Buffers and Thread Buffers?
I found this blog article that lists which are the Global Buffer and Thread Buffer variables, which are:
Global Buffer:
key_buffer_size
innodb_buffer_pool_size
innodb_log_buffer_size
innodb_additional_mem_pool_size
net_buffer_size
query_cache_size
Thread Buffer:
sort_buffer_size
myisam_sort_buffer_size
read_buffer_size
join_buffer_size
read_rnd_buffer_size
thread_stack

How do I check the value of 'binlog_do_db' in MySQL?

I am trying to debug why my DML statements aren't ending up in the binlogs. The master and slave appear to be talking to eachother and the slave status indicates that it caught up properly to the most recent log file.
I did a show variables LIKE '%binlog%'; with this output, but binlog_do_db is not there.
+-----------------------------------------+----------------------+
| Variable_name | Value |
+-----------------------------------------+----------------------+
| binlog_cache_size | 32768 |
| binlog_checksum | CRC32 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_error_action | ABORT_SERVER |
| binlog_format | ROW |
| binlog_group_commit_sync_delay | 0 |
| binlog_group_commit_sync_no_delay_count | 0 |
| binlog_gtid_simple_recovery | ON |
| binlog_max_flush_queue_time | 0 |
| binlog_order_commits | ON |
| binlog_row_image | FULL |
| binlog_rows_query_log_events | OFF |
| binlog_stmt_cache_size | 32768 |
| innodb_api_enable_binlog | OFF |
| innodb_locks_unsafe_for_binlog | OFF |
| log_statements_unsafe_for_binlog | ON |
| max_binlog_cache_size | 18446744073709547520 |
| max_binlog_size | 104857600 |
| max_binlog_stmt_cache_size | 18446744073709547520 |
| sync_binlog | 1 |
+-----------------------------------------+----------------------+
After much effort and grief, I found that in the previous administrator must have been trying to get replication to work and put these lines into the /etc/mysql/mysql.conf.d/mysqld.cnf file
# Setup Incremental Backups
log-bin = /srv/ddisk/backups/mysql/mysql_inc
expire_logs_days = 3
max_binlog_size = 100M
server-id = 1
binlog_do_db = include_database_name
The last line is probably the culprit (though I will likely clean out that whole config block since I have them in the my.cnf config file already. In any case, I am now trying to verify that this configuration is actually being read by MySQL, but I'm unable to figure out what query to run to find this variable.
Would someone happen to know how to find binlog_do_db?
Thanks ahead of time
You can check using mysql prompt on the master node below:
mysql> SHOW MASTER STATUS;
It will show Binary log file, position, Binlog_Do_DB, Binlog_Ignore_DB parameter and so on.

How can I cache all mysql queries and results in my webpage instead of caching each query?

I did following changes for caching mysql queries in server
SET GLOBAL query_cache_size = 8388608;
SET GLOBAL query_cache_limit = 1048576;
SET GLOBAL query_cache_type = 1;
and I am getting following result in mysql
mysql> show variables like 'query%';
+------------------------------+---------+
| Variable_name | Value |
+------------------------------+---------+
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 8388608 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
+------------------------------+---------+
Is there anything I need to change in my php code to achieve caching for all the mysql queries ? Please check this

Mysql 5.1 Improve Performance for InnoDB

I have a server on Debian 6 using Mysql 5.1 and I'm having problems with big tables on Innodb. As Mysql was installed on its default configurations I'd like to know some suggestion about how to improve the performance of my server.
DB Mysql 5.1
Operative System: Debian6
I have 4G available on free memory according free -g command on Debian.
I have 6GB on swap on Debian OS.
mysql> SHOW VARIABLES LIKE '%buffer%';
+-------------------------+------------+
| Variable_name | Value |
+-------------------------+------------+
| bulk_insert_buffer_size | 8388608 |
| innodb_buffer_pool_size | 2097152000 |
| innodb_log_buffer_size | 1048576 |
| join_buffer_size | 131072 |
| key_buffer_size | 16777216 |
| myisam_sort_buffer_size | 8388608 |
| net_buffer_length | 16384 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 2097144 |
| sql_buffer_result | OFF |
+-------------------------+------------+
12 rows in set
I have to mention that the only thing I have changed on my cnf file is innodb_buffer_pool_size to 2097152000 but I have no get good result about performance.
By the way on my server I have big tables with partitions and big tables on MyIsam as well.

why information_schema.tables.data_free always be 8388608?

Dose any one knows why information_schema.tables.data_free in InnoDB always be 8388608, no matter how many rows in tables;
table_schema table_name table_rows data_free engine
g33v1 appraise 0 8388608 InnoDB
g33v1 areatype 12403 8388608 InnoDB
g33v1 atype 581982 8388608 InnoDB
g33v1 atype2 579700 8388608 InnoDB
thanks.
I think it is the temporary table maximum size allocated for doing sorting and other things which requires the space on HDD. As the same things appears in other cases also
mysql> show global variables like '%tmp%';
+----------------+-----------------------+
| Variable_name | Value |
+----------------+-----------------------+
| bdb_tmpdir | /usr/local/mysql/tmp/ |
| max_tmp_tables | 32 |
| tmp_table_size | 8388608 |
| tmpdir | /usr/local/mysql/tmp |
+----------------+-----------------------+
mysql> show global variables like '%myisam%';
+---------------------------------+---------------+
| Variable_name | Value |
+---------------------------------+---------------+
| myisam_data_pointer_size | 4 |
| myisam_max_extra_sort_file_size | 2147483648 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_recover_options | OFF |
| myisam_repair_threads | 1 |
| myisam_sort_buffer_size | 4194304 |
| myisam_stats_method | nulls_unequal |
+---------------------------------+---------------