Data are similar on two servers.
Server A :
mysql> SHOW VARIABLES LIKE "%version%";
+-------------------------+-------------------------+
| Variable_name | Value |
+-------------------------+-------------------------+
| innodb_version | 5.7.26 |
| protocol_version | 10 |
| slave_type_conversions | |
| tls_version | TLSv1,TLSv1.1 |
| version | 5.7.26-0ubuntu0.16.04.1 |
| version_comment | (Ubuntu) |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+-------------------------+-------------------------+
Server B
MariaDB> SHOW VARIABLES LIKE "%version%";
+-------------------------+--------------------------+
| Variable_name | Value |
+-------------------------+--------------------------+
| innodb_version | 5.6.42-84.2 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 10.1.38-MariaDB-0+deb9u1 |
| version_comment | Debian 9.8 |
| version_compile_machine | x86_64 |
| version_compile_os | debian-linux-gnu |
| version_malloc_library | system jemalloc |
| version_ssl_library | YaSSL 2.4.4 |
| wsrep_patch_version | wsrep_25.24 |
+-------------------------+--------------------------+
When I make an EXPLAIN on a query, results are differents :
Server A return
+----+-------------+-------+------------+--------+----------------------+----------+---------+--------------------+------+----------+-----------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+--------+----------------------+----------+---------+--------------------+------+----------+-----------------+
| 1 | SIMPLE | f1_ | NULL | index | IDX_90291C2D2D710CF2 | date_idx | 5 | NULL | 40 | 100.00 | Using temporary |
| 1 | SIMPLE | u0_ | NULL | eq_ref | PRIMARY,closed_idx | PRIMARY | 4 | smailsf.f1_.poster | 1 | 50.00 | Using where |
+----+-------------+-------+------------+--------+----------------------+----------+---------+--------------------+------+----------+-----------------+
Server B return
+------+-------------+-------+--------+----------------------+---------+---------+------------------+--------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+--------+----------------------+---------+---------+------------------+--------+---------------------------------+
| 1 | SIMPLE | f1_ | ALL | IDX_90291C2D2D710CF2 | NULL | NULL | NULL | 657064 | Using temporary; Using filesort |
| 1 | SIMPLE | u0_ | eq_ref | PRIMARY,closed_idx | PRIMARY | 4 | smail.f1_.poster | 1 | Using where |
+------+-------------+-------+--------+----------------------+---------+---------+------------------+--------+---------------------------------+
Why, for the same query, on Server A, number of rows is 40 and Server B, number of rows is 657064 ?
It's like Maria don't considere indexes or cache ?
What kind of configuration options can impact this behaviour ?
Thanks,
Ok,
innodb_buffer_pool_size settings is my solution ;) Thanks.
Related
MySQL 5.7.17 MGR deploy in single-primary mode, 3 node all one one machine, the same configuration.
And then we test insert on the primary node and observe that the slave nodes have high delay with the write node, even primary node finish the insert test, slave node data is incresing!
Why MySQL Group Replication slave nodes have high delay with write node?
here is the my.cnf:
[mysqld]
datadir=/dba/mysql/data/s1
basedir=/dba/mysql/mysql-5.7/
port=24801
socket=/dba/mysql/data/s1/s1.sock
server_id=1
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_checksum=NONE
log_slave_updates=ON
log_bin=binlog
binlog_format=ROW
transaction_write_set_extraction=XXHASH64
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
loose-group_replication_start_on_boot=off
loose-group_replication_local_address= "127.0.0.1:24901"
loose-group_replication_group_seeds= "127.0.0.1:24901,127.0.0.1:24902,127.0.0.1:24903"
loose-group_replication_bootstrap_group= off
loose-group_replication_single_primary_mode=true
loose-group_replication_enforce_update_everywhere_checks=false
slave_parallel_type=LOGICAL_CLOCK
slave_preserve_commit_order=1
slave_parallel_workers=4
and the MGR config :
mysql> show variables like '%group_replication%';
+----------------------------------------------------+-------------------------------------------------+
| Variable_name | Value |
+----------------------------------------------------+-------------------------------------------------+
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_allow_local_lower_version_join | OFF |
| group_replication_auto_increment_increment | 7 |
| group_replication_bootstrap_group | OFF |
| group_replication_components_stop_timeout | 31536000 |
| group_replication_compression_threshold | 1000000 |
| group_replication_enforce_update_everywhere_checks | OFF |
| group_replication_flow_control_applier_threshold | 25000 |
| group_replication_flow_control_certifier_threshold | 25000 |
| group_replication_flow_control_mode | QUOTA |
| group_replication_force_members | |
| group_replication_group_name | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa |
| group_replication_group_seeds | 127.0.0.1:24901,127.0.0.1:24902,127.0.0.1:24903 |
| group_replication_gtid_assignment_block_size | 1000000 |
| group_replication_ip_whitelist | AUTOMATIC |
| group_replication_local_address | 127.0.0.1:24901 |
| group_replication_poll_spin_loops | 0 |
| group_replication_recovery_complete_at | TRANSACTIONS_APPLIED |
| group_replication_recovery_reconnect_interval | 60 |
| group_replication_recovery_retry_count | 10 |
| group_replication_recovery_ssl_ca | |
| group_replication_recovery_ssl_capath | |
| group_replication_recovery_ssl_cert | |
| group_replication_recovery_ssl_cipher | |
| group_replication_recovery_ssl_crl | |
| group_replication_recovery_ssl_crlpath | |
| group_replication_recovery_ssl_key | |
| group_replication_recovery_ssl_verify_server_cert | OFF |
| group_replication_recovery_use_ssl | OFF |
| group_replication_single_primary_mode | ON |
| group_replication_ssl_mode | DISABLED |
| group_replication_start_on_boot | OFF |
+----------------------------------------------------+-------------------------------------------------+
32 rows in set (0.01 sec)
When running an order by query with a limit on MariaDB 10.1.18, I get back a wrong order.
Observe the query without the LIMIT statement:
select advert_id, published, id from vacancies order by published asc;
+-----------+-----------+----+
| advert_id | published | id |
+-----------+-----------+----+
| 328377 | 0 | 70 |
| 328844 | 0 | 80 |
| 325263 | 0 | 41 |
| 325774 | 0 | 40 |
| 325775 | 0 | 39 |
| 325929 | 0 | 38 |
| 325885 | 0 | 37 |
| 325901 | 0 | 36 |
| 325920 | 0 | 35 |
| 325917 | 0 | 34 |
| 325922 | 0 | 33 |
| 325889 | 0 | 32 |
| 325927 | 0 | 31 |
| 325238 | 0 | 43 |
| 325244 | 0 | 45 |
| 328365 | 0 | 71 |
| 328446 | 0 | 72 |
| 328362 | 0 | 68 |
| 323602 | 0 | 55 |
| 324250 | 0 | 54 |
| 324254 | 0 | 53 |
| 324911 | 0 | 52 |
With the LIMIT statement:
select advert_id, published, id from vacancies order by published asc limit 10;
+-----------+-----------+----+
| advert_id | published | id |
+-----------+-----------+----+
| 327830 | 0 | 1 |
| 326865 | 0 | 18 |
| 327328 | 0 | 9 |
| 326877 | 0 | 16 |
| 326783 | 0 | 21 |
| 326779 | 0 | 17 |
| 326774 | 0 | 15 |
| 326864 | 0 | 20 |
| 326788 | 0 | 14 |
| 326767 | 0 | 19 |
+-----------+-----------+----+
The order by on published is different in both queries.
For comparison, I ran the same queries on MariaDB 5.5.50 and found the order by + limit correctly returns the same result as the order by query. So from what I understand is that this issue is MariaDB specific, and only exists on newer versions.
Additionally I also ran the same queries, but ordering on a varchar field with a lot of different values, in that case the order was correct. So I'm thinking the problem only applies to ordering with limit on a field that has a lot of the same values.
Does anyone know if there is a way around this? Perhaps a setting in MariaDB?
FYI:
Table structure:
+------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| advert_id | int(11) | YES | | NULL | |
| published | tinyint(1) | NO | | 0 | |
| (other fields omitted)
Explain on query:
explain select advert_id, published, id from vacancies order by published asc;
+------+-------------+-----------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | vacancies | ALL | NULL | NULL | NULL | NULL | 52 | Using filesort |
+------+-------------+-----------+------+---------------+------+---------+------+------+----------------+
explain select advert_id, published, id from vacancies order by published asc limit 10;
+------+-------------+-----------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | vacancies | ALL | NULL | NULL | NULL | NULL | 52 | Using filesort |
+------+-------------+-----------+------+---------------+------+---------+------+------+----------------+
Version with order by issue:
mysql Ver 15.1 Distrib 10.1.18-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
Version without order by issue:
mysql Ver 15.1 Distrib 5.5.50-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
You do not specify which ten first rows you want to obtain. As there are a lot of rows for which published equals 0 MariaDB is free to choose some of them. If you want a specific order try:
SELECT advert_id, published, id FROM vacancies ORDER BY published asc, id LIMIT 10;
I just dont understand one thing. When I type:
SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE';
I get:
+----------------------------------------------+
| table_name |
+----------------------------------------------+
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
41 rows in set (0.23 sec)
but selecting seems not to work at all:
mysql> select * from db;
ERROR 1109 (42S02): Unknown table 'db' in information_schema
mysql>
How is that possible? I mean, SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE'; showed that there IS a table named 'db' ...
Secondly, when I type: SELECT table_name FROM INFORMATION_SCHEMA.TABLES; it gives me :
+----------------------------------------------+
| table_name |
+----------------------------------------------+
| CHARACTER_SETS |
| COLLATIONS |
| COLLATION_CHARACTER_SET_APPLICABILITY |
| COLUMNS |
| COLUMN_PRIVILEGES |
| ENGINES |
| EVENTS |
| FILES |
| GLOBAL_STATUS |
| GLOBAL_VARIABLES |
| KEY_COLUMN_USAGE |
| PARAMETERS |
| PARTITIONS |
| PLUGINS |
| PROCESSLIST |
| PROFILING |
| REFERENTIAL_CONSTRAINTS |
| ROUTINES |
| SCHEMATA |
| SCHEMA_PRIVILEGES |
| SESSION_STATUS |
| SESSION_VARIABLES |
| STATISTICS |
| TABLES |
| TABLESPACES |
| TABLE_CONSTRAINTS |
| TABLE_PRIVILEGES |
| TRIGGERS |
| USER_PRIVILEGES |
| VIEWS |
| INNODB_BUFFER_PAGE |
| INNODB_TRX |
| INNODB_BUFFER_POOL_STATS |
| INNODB_LOCK_WAITS |
| INNODB_CMPMEM |
| INNODB_CMP |
| INNODB_LOCKS |
| INNODB_CMPMEM_RESET |
| INNODB_CMP_RESET |
| INNODB_BUFFER_PAGE_LRU |
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
81 rows in set (0.00 sec)
and Im able to do: mysql> select * from events; and Im getting some results. Why is that? Why can I do select only on upper-case-names? And further, how can I select from information_schema tables given only in uppercase? Cheers
If it matters: Im logged as root to my db.
You select only table_name, but what about table_schema:
MariaDB [(none)]> SELECT table_name, table_schema FROM INFORMATION_SCHEMA.TABLES WHERE
TABLE_TYPE = 'BASE TABLE';
+----------------------------------------------+--------------------+
| table_name | table_schema |
+----------------------------------------------+--------------------+
| columns_priv | mysql |
| db | mysql |
| event | mysql |
| func | mysql |
| general_log | mysql |
| help_category | mysql |
| help_keyword | mysql |
| help_relation | mysql |
| help_topic | mysql |
| host | mysql |
| ndb_binlog_index | mysql |
| plugin | mysql |
| proc | mysql |
| procs_priv | mysql |
| proxies_priv | mysql |
| servers | mysql |
| slow_log | mysql |
| tables_priv | mysql |
| time_zone | mysql |
| time_zone_leap_second | mysql |
| time_zone_name | mysql |
| time_zone_transition | mysql |
| time_zone_transition_type | mysql |
| user | mysql |
| cond_instances | performance_schema |
| events_waits_current | performance_schema |
| events_waits_history | performance_schema |
| events_waits_history_long | performance_schema |
| events_waits_summary_by_instance | performance_schema |
| events_waits_summary_by_thread_by_event_name | performance_schema |
| events_waits_summary_global_by_event_name | performance_schema |
| file_instances | performance_schema |
| file_summary_by_event_name | performance_schema |
| file_summary_by_instance | performance_schema |
| mutex_instances | performance_schema |
| performance_timers | performance_schema |
| rwlock_instances | performance_schema |
| setup_consumers | performance_schema |
| setup_instruments | performance_schema |
| setup_timers | performance_schema |
| threads | performance_schema |
+----------------------------------------------+--------------------+
And you get:
MariaDB [(none)]> select * from mysql.db;
Empty set (0.00 sec)
'information_schema' has ONLY SERVICE INFORMATION. It has info about table 'db' is exist, but it table IS NOT IN 'information_schema' DATABASE -- somewhere, but not in 'information_schema'.
Info about database which has needed table saved in TABLE_SCHEMA field
I have a highly relational set of tables that I've flattened into two InnoDB tables:
mysql> desc WPropertyCube;
+--------------------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+---------------------+------+-----+---------+----------------+
| id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| lineOfBusinessId | bigint(20) unsigned | NO | MUL | NULL | |
| txtProperty1 | varchar(125) | YES | | NULL | |
| txtProperty2 | varchar(125) | YES | | NULL | |
| txtProperty3 | varchar(125) | YES | | NULL | |
...
| txtProperty20 | varchar(125) | YES | | NULL | |
| lookupPropertyId1 | bigint(20) unsigned | YES | | NULL | |
| lookupPropertyId2 | bigint(20) unsigned | YES | | NULL | |
| lookupPropertyId3 | bigint(20) unsigned | YES | | NULL | |
...
| lookupPropertyId30 | bigint(20) unsigned | YES | | NULL | |
+--------------------+---------------------+------+-----+---------+----------------+
mysql> show indexes from WPropertyCube;
+---------------+------------+---------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+---------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| WPropertyCube | 0 | PRIMARY | 1 | id | A | 379383 | NULL | NULL | | BTREE | | |
| WPropertyCube | 1 | WPropertyCube_idx01 | 1 | lineOfBusinessId | A | 204 | NULL | NULL | | BTREE | | |
+---------------+------------+---------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> desc WMeasureCube;
+----------------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+---------------------+------+-----+---------+-------+
| propertyCubeId | bigint(20) unsigned | NO | PRI | NULL | |
| actionDate | datetime | NO | PRI | NULL | |
| measure1 | decimal(15,6) | YES | | NULL | |
| measure2 | decimal(15,6) | YES | | NULL | |
| measure3 | decimal(15,6) | YES | | NULL | |
...
| measure30 | decimal(15,6) | YES | | NULL | |
+----------------+---------------------+------+-----+---------+-------+
mysql> show indexes from WMeasureCube;
+--------------+------------+-------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+-------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| WMeasureCube | 0 | PRIMARY | 1 | propertyCubeId | A | 18 | NULL | NULL | | BTREE | | |
| WMeasureCube | 0 | PRIMARY | 2 | actionDate | A | 81680372 | NULL | NULL | | BTREE | | |
| WMeasureCube | 1 | WMeasureCube_idx1 | 1 | actionDate | A | 18 | NULL | NULL | | BTREE | | |
| WMeasureCube | 1 | WMeasureCube_idx1 | 2 | propertyCubeId | A | 81680372 | NULL | NULL | | BTREE | | |
+--------------+------------+-------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
You can see I've been playing around with indexes but haven't hit upon the magic combination yet. WPropertyCube has ~100K records, and WMeasureCube has ~80M. My query is this:
mysql> explain SELECT wmc.actionDate,
-> SUM(wmc.measure7),
-> SUM(wmc.measure8),
-> SUM(wmc.measure9)
-> FROM WMeasureCube wmc
-> INNER JOIN WPropertyCube wpc
-> ON wmc.propertyCubeId = wpc.id
-> WHERE wpc.lineOfBusinessId IN ( 1, 2, 3, 4 )
-> AND wmc.actionDate BETWEEN '2010-06-28' AND '2010-09-26'
-> GROUP BY wmc.actionDate
-> ORDER BY wmc.actionDate;
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+----------------------------------------------+
| 1 | SIMPLE | wmc | ALL | PRIMARY,WMeasureCube_idx1 | NULL | NULL | NULL | 81680372 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | wpc | eq_ref | PRIMARY,WPropertyCube_idx01 | PRIMARY | 8 | db.wmc.propertyCubeId | 1 | Using where |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+----------------------------------------------+
This query is aggregating ~30K records into ~1,000 records. Notice that no indices are used to access WMeasureCube, and worse, there's a filesort which is slooooowww. Query time ranges between 30 - 150 seconds. Confoundingly, if I remove the GROUP BY:
mysql> explain SELECT wmc.actionDate,
-> SUM(wmc.measure7),
-> SUM(wmc.measure8),
-> SUM(wmc.measure9)
-> FROM WMeasureCube wmc
-> INNER JOIN WPropertyCube wpc
-> ON wmc.propertyCubeId = wpc.id
-> WHERE wpc.lineOfBusinessId IN ( 1, 2, 3, 4 )
-> AND wmc.actionDate BETWEEN '2010-06-28' AND '2010-09-26'
-> ORDER BY wmc.actionDate;
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+-------------+
| 1 | SIMPLE | wmc | ALL | PRIMARY,WMeasureCube_idx1 | NULL | NULL | NULL | 81680372 | Using where |
| 1 | SIMPLE | wpc | eq_ref | PRIMARY,WPropertyCube_idx01 | PRIMARY | 8 | db.wmc.propertyCubeId | 1 | Using where |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+----------+-------------+
Still no index use, but at least there's no filesort. This query runs consistently in less than a second. This is especially weird in that ordering by a set of columns should be congruent, in terms of execution time, to grouping by the same set.
What can I do to speed up my query?
I've tried to add all the server variables, but they take up too much space. So, I've added the ones I think will help the most.
mysql> show variables;
+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------+
| big_tables | OFF |
| binlog_cache_size | 4194304 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_format | STATEMENT |
| bulk_insert_buffer_size | 8388608 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| default_storage_engine | InnoDB |
| innodb_adaptive_flushing | ON |
| innodb_adaptive_hash_index | ON |
| innodb_additional_mem_pool_size | 16777216 |
| innodb_autoextend_increment | 8 |
| innodb_autoinc_lock_mode | 1 |
| innodb_buffer_pool_instances | 1 |
| innodb_buffer_pool_size | 6442450944 |
| innodb_change_buffering | all |
| innodb_checksums | ON |
| innodb_commit_concurrency | 0 |
| innodb_concurrency_tickets | 500 |
| innodb_data_file_path | ibdata1:10M:autoextend |
| innodb_data_home_dir | |
| innodb_doublewrite | ON |
| innodb_fast_shutdown | 1 |
| innodb_file_format | Barracuda |
| innodb_file_format_check | ON |
| innodb_file_format_max | Barracuda |
| innodb_file_per_table | ON |
| innodb_flush_log_at_trx_commit | 0 |
| innodb_flush_method | |
| innodb_force_recovery | 0 |
| innodb_io_capacity | 200 |
| innodb_lock_wait_timeout | 120 |
| innodb_locks_unsafe_for_binlog | OFF |
| innodb_log_buffer_size | 16777216 |
| innodb_log_file_size | 268435456 |
| innodb_log_files_in_group | 3 |
| innodb_log_group_home_dir | ./ |
| innodb_max_dirty_pages_pct | 90 |
| innodb_max_purge_lag | 0 |
| innodb_mirrored_log_groups | 1 |
| innodb_old_blocks_pct | 37 |
| innodb_old_blocks_time | 0 |
| innodb_open_files | 300 |
| innodb_purge_batch_size | 20 |
| innodb_purge_threads | 0 |
| innodb_read_ahead_threshold | 56 |
| innodb_read_io_threads | 8 |
| innodb_replication_delay | 0 |
| innodb_rollback_on_timeout | OFF |
| innodb_spin_wait_delay | 6 |
| innodb_stats_on_metadata | ON |
| innodb_stats_sample_pages | 8 |
| innodb_strict_mode | OFF |
| innodb_support_xa | OFF |
| innodb_sync_spin_loops | 30 |
| innodb_table_locks | ON |
| innodb_thread_concurrency | 0 |
| innodb_thread_sleep_delay | 10000 |
| innodb_use_native_aio | ON |
| innodb_use_sys_malloc | ON |
| innodb_version | 1.1.2 |
| innodb_write_io_threads | 8 |
| join_buffer_size | 8388608 |
| key_buffer_size | 8388608 |
| large_files_support | ON |
| large_page_size | 0 |
| large_pages | OFF |
| last_insert_id | 0 |
| max_binlog_cache_size | 18446744073709547520 |
| max_binlog_size | 1073741824 |
| max_heap_table_size | 536870912 |
| max_join_size | 18446744073709551615 |
| max_length_for_sort_data | 1024 |
| max_sort_length | 1024 |
| max_sp_recursion_depth | 0 |
| max_tmp_tables | 32 |
| optimizer_prune_level | 1 |
| optimizer_search_depth | 62 |
| optimizer_switch | index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on |
| preload_buffer_size | 32768 |
| profiling | OFF |
| profiling_history_size | 15 |
| protocol_version | 10 |
| pseudo_thread_id | 13 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 8388608 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 134217728 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
| rand_seed1 | 0 |
| rand_seed2 | 0 |
| range_alloc_block_size | 4096 |
| read_buffer_size | 8388608 |
| sort_buffer_size | 16777216 |
| sql_big_selects | ON |
| sql_big_tables | OFF |
| sql_max_join_size | 18446744073709551615 |
| storage_engine | InnoDB |
| tmp_table_size | 536870912 |
| tmpdir | /dev/shm |
| tx_isolation | REPEATABLE-READ |
| version | 5.5.6-rc-log |
| version_comment | MySQL Community Server (GPL) |
| version_compile_machine | x86_64 |
| version_compile_os | linux2.6 |
+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------+
[EDIT] Thanks to a suggestion by Daniel, I've changed the DATETIME field to DATE. Happily, the index is now being used. Unhappily, we're still filesorting.
+----+-------------+-------+-------+-----------------------------+---------------------+---------+---------------+--------+-----------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+-----------------------------+---------------------+---------+---------------+--------+-----------------------------------------------------------+
| 1 | SIMPLE | wpc | index | PRIMARY,WPropertyCube_idx01 | WPropertyCube_idx01 | 8 | NULL | 377500 | Using where; Using index; Using temporary; Using filesort |
| 1 | SIMPLE | wmc | ref | PRIMARY,WMeasureCube_idx1 | PRIMARY | 8 | db.wpc.id | 49 | Using where |
+----+-------------+-------+-------+-----------------------------+---------------------+---------+---------------+--------+-----------------------------------------------------------+
i think your problem is the datetime column actiondate. try to set this column to date column if you dont need the time information! then grouping should be faster and your index should work.
I have the following query:
SELECT mdg.id AS mdg_id, vdk.id AS vdisk_id, vdk.name AS vdisk_name, vdk.objKey AS vdisk_key,
vdc.type AS copy_type, vde.copy_id AS copy_id, mdk.id AS mdisk_id, vde.number_extents AS extent_count,
IFNULL(ter.name,'generic_hdd') AS mdisk_tier, mdg.snapKey, vde.objKey AS row_key
FROM mdisk_grp AS mdg
INNER JOIN vdiskcopy AS vdc ON mdg.snapKey = vdc.snapKey AND mdg.id = vdc.mdisk_grp_id
INNER JOIN vdisk AS vdk ON vdc.owner = vdk.objKey
INNER JOIN vdiskextent AS vde ON vdk.snapKey=vde.snapKey AND vdk.id = vde.vdisk_id AND vdc.copy_id = vde.copy_id
INNER JOIN mdisk AS mdk ON vde.id = mdk.id AND vde.snapKey = mdk.snapKey
LEFT JOIN tier_mdisk AS tmk ON tmk.mdisk_objKey = mdk.objKey
LEFT JOIN tier AS ter ON tmk.tier_objKey = ter.objKey
WHERE mdg.snapKey= '333';
I'm seeing it frequently in the slow query log (witch different values of mdg.snapKey, of course); for example:
# User#Host: svcControl[svcControl] # localhost []
# Query_time: 2577 Lock_time: 0 Rows_sent: 11469 Rows_examined: 354942843
The fact that it is running long isn't surprising, given the number of rows being examined.
However, when I EXPLAIN the query, I see
+----+-------------+-------+--------+---------------------------------------------------------+--------------------+---------+------------------------+------+---------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------------------------------+--------------------+---------+------------------------+------+---------------------+
| 1 | SIMPLE | tmk | system | mdisk_objKey_idx | NULL | NULL | NULL | 0 | const row not found |
| 1 | SIMPLE | ter | const | PRIMARY | NULL | NULL | NULL | 1 | |
| 1 | SIMPLE | mdg | ref | mdisk_grp_id_idx,snapIdx | snapIdx | 2 | const | 11 | |
| 1 | SIMPLE | vdc | ref | snapIdx,mdgIdIdx,ownerIdx | mdgIdIdx | 5 | svcObjects.mdg.id | 1 | Using where |
| 1 | SIMPLE | vdk | eq_ref | PRIMARY,vdisk_id_idx,snapIdx | PRIMARY | 3 | svcObjects.vdc.owner | 1 | |
| 1 | SIMPLE | mdk | ref | mdisk_id_idx,snapKey_idx | snapKey_idx | 2 | svcObjects.vdk.snapKey | 16 | |
| 1 | SIMPLE | vde | ref | vdiskextent_id_idx,vdiskextent_vdisk_id_idx,snapKey_idx | vdiskextent_id_idx | 5 | svcObjects.mdk.id | 23 | Using where |
+----+-------------+-------+--------+---------------------------------------------------------+--------------------+---------+------------------------+------+---------------------+
Multiplying all of those rows together comes to a little over 4000 (ignoring the 0 in the column)
It appears that the execution plan is being ignored.
Is there a way to run a query and then get a report and what execution plan was actually followed? Or am I misunderstanding this output?
UPDATE
I noticed that the Cardinality for some of my indexes was NULL, so I ANALYZEd all of the tables. The latest EXPLAIN reports:
+----+-------------+-------+--------+---------------------------------------------------------+--------------------------+---------+----------------------+------+---------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------------------------------+--------------------------+---------+----------------------+------+---------------------+
| 1 | SIMPLE | tmk | system | mdisk_objKey_idx | NULL | NULL | NULL | 0 | const row not found |
| 1 | SIMPLE | ter | const | PRIMARY | NULL | NULL | NULL | 1 | |
| 1 | SIMPLE | mdg | ref | mdisk_grp_id_idx,snapIdx | snapIdx | 2 | const | 8 | |
| 1 | SIMPLE | vdc | ref | snapIdx,mdgIdIdx,ownerIdx | snapIdx | 2 | const | 1580 | Using where |
| 1 | SIMPLE | vdk | eq_ref | PRIMARY,vdisk_id_idx,snapIdx | PRIMARY | 3 | svcObjects.vdc.owner | 1 | |
| 1 | SIMPLE | vde | ref | vdiskextent_id_idx,vdiskextent_vdisk_id_idx,snapKey_idx | vdiskextent_vdisk_id_idx | 5 | svcObjects.vdk.id | 300 | Using where |
| 1 | SIMPLE | mdk | ref | mdisk_id_idx,snapKey_idx | mdisk_id_idx | 5 | svcObjects.vde.id | 14 | Using where |
+----+-------------+-------+--------+---------------------------------------------------------+--------------------------+---------+----------------------+------+---------------------+
That multiplies up to 53,088,000, which is much better than the 354,942,843 that were examined in the slow query.
I guess my question now is this... does the behaviour that I've seen get explained by what I did... if the index cardinalities are NULL then the execution plan will appear over-optimistic and may also result in 6-7 times the number of necessary rows being examined?
Look if the article answers your question: http://www.mysqlperformanceblog.com/2011/10/13/when-explain-estimates-can-go-wrong/