I have a table
CREATE TABLE `pd` (
`product_id` int(11) NOT NULL AUTO_INCREMENT,
`language_id` int(11) NOT NULL,
`name` varchar(255) NOT NULL,
`description` text NOT NULL,
`meta_description` varchar(255) NOT NULL,
`meta_keyword` varchar(255) NOT NULL,
`seo_title` varchar(255) NOT NULL,
`seo_h1` varchar(255) NOT NULL,
PRIMARY KEY (`product_id`,`language_id`),
KEY `language_id` (`description`(128),`language_id`),
KEY `name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=47019 DEFAULT CHARSET=utf8;
When I run this query
EXPLAIN SELECT * FROM `pd` ORDER BY product_id;
I get this result:
+----+-------------+-------+-------+---------------+---------+---------+------+--------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+---------+---------+------+--------+-------+
| 1 | SIMPLE | pd | index | NULL | PRIMARY | 8 | NULL | 139551 | |
+----+-------------+-------+-------+---------------+---------+---------+------+--------+-------+
When I run this query
EXPLAIN SELECT * FROM `pd` ORDER BY name;
I get this result:
+----+-------------+-------+------+---------------+------+---------+------+--------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+--------+----------------+
| 1 | SIMPLE | pd | ALL | NULL | NULL | NULL | NULL | 137762 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+--------+----------------+
Why in second case index isn't used? Only difference I see is product_id is part of primary key and name is non-unique index.
As soon as the name isn't a clustered key - it will be much more expensive to read the index first and seek for every row one by one in data.
So mysql (and I think that any other would DBMS) decdies to read the data and sort it in-memory.
Related
I have a single table book_log Mysql 5.7
+------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| book_id | int(11) | YES | MUL | NULL | |
| type | int(11) | NO | MUL | NULL | |
| value | int(11) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
+------------+----------+------+-----+---------+----------------+
Book table makes connection with series (One series can have many books)
Create table info :
book_log | CREATE TABLE `book_log` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`book_id` int(11) DEFAULT NULL,
`type` int(11) NOT NULL,
`value` int(11) DEFAULT NULL,
`created_at` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `IDX_7E42115316A2B381` (`book_id`),
KEY `IDX_TYPE` (`type`),
KEY `IDX_ME` (`book_id`,`type`) USING BTREE,
CONSTRAINT `FK_7E42115316A2B381` FOREIGN KEY (`book_id`) REFERENCES `book` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1158962 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci |
book | CREATE TABLE `book` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`series_id` int(11) DEFAULT NULL,
`language_id` int(11) DEFAULT NULL,
`position` int(11) NOT NULL,
`dir` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`title` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `IDX_CBE5A3315278319C` (`series_id`),
KEY `IDX_CBE5A33182F1BAF4` (`language_id`),
CONSTRAINT `FK_CBE5A3315278319C` FOREIGN KEY (`series_id`) REFERENCES `series` (`id`),
CONSTRAINT `FK_CBE5A33182F1BAF4` FOREIGN KEY (`language_id`) REFERENCES `language` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=55022 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci |
I make the avg value for a given series
select AVG(value)
from book_log
join book b on book_log.book_id = b.id
where type = 20 and b.series_id = ?;
Explain :
+----+-------------+----------+------------+------+------------------------------+----------------------+---------+----------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+----------+------------+------+------------------------------+----------------------+---------+----------------+------+----------+-------------+
| 1 | SIMPLE | b | NULL | ref | PRIMARY,IDX_CBE5A3315278319C | IDX_CBE5A3315278319C | 5 | const | 212 | 100.00 | Using index |
| 1 | SIMPLE | book_log | NULL | ref | IDX_7E42115316A2B381,IDX_ME | IDX_7E42115316A2B381 | 5 | bdd.b.id | 33 | 100.00 | NULL |
+----+-------------+----------+------------+------+------------------------------+----------------------+---------+----------------+------+----------+-------------+
Or
select AVG(value)
from book_log
where type = 20 AND book_id IN (
select id from book where series_id = ?
);
Explain :
+----+-------------+----------+------------+------+--------------------------------------+----------------------+---------+-------------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+----------+------------+------+--------------------------------------+----------------------+---------+-------------------+------+----------+-------------+
| 1 | SIMPLE | book | NULL | ref | PRIMARY,IDX_CBE5A3315278319C | IDX_CBE5A3315278319C | 5 | const | 212 | 100.00 | Using index |
| 1 | SIMPLE | book_log | NULL | ref | IDX_7E42115316A2B381,IDX_TYPE,IDX_ME | IDX_7E42115316A2B381 | 5 | bdd.book.id | 33 | 3.72 | Using where |
+----+-------------+----------+------------+------+--------------------------------------+----------------------+---------+-------------------+------+----------+-------------+
I have 10 973 results for these query, 42ms for a count(*) but more than 1 sec for the avg query.
I don't understand why is it so long.
Any idea ?
Thx.
You can expect to COUNT(*) be as fast or faster than SUM(somecol) or AVG(othercolumn). Why? The database server is, by the rules of SQL, to apply any optimization that yields the correct anwer. COUNT(*) has some serious optimization to it.
But the aggregate functions that do arithmetic ; they must instead examine every record. So, slower.
You can create a purpose-built index for your query.
It is this:
ALTER TABLE book_log
ADD INDEX type_id_val
(type, book_id, val)
I chose these columns for the index because your query searches for a particuler type in the index. Upon finding the first row of the chosen type, MySQL can range-scan through just the index and not the table. So, faster. It's called a covering index.
I have this query:
EXPLAIN EXTENDED
SELECT DISTINCT
PMS_STAGIONI.DINIZVAL,
PMS_STAGIONI.DFINEVAL,
PMS_DISPO.DDATA
FROM
PMS_DISPO JOIN PMS_STAGIONI
HAVING
PMS_DISPO.DDATA BETWEEN PMS_STAGIONI.DINIZVAL AND PMS_STAGIONI.DFINEVAL
The output of explain is:
+----+-------------+--------------+-------+---------------+------------------------------+---------+------+------+----------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------+-------+---------------+------------------------------+---------+------+------+----------+--------------------------------+
| 1 | SIMPLE | PMS_STAGIONI | index | NULL | IDX_INIZFINEVAL_PMS_STAGIONI | 6 | NULL | 3 | 100.00 | Using index; Using temporary |
| 1 | SIMPLE | PMS_DISPO | index | NULL | IDX_DDATA_PMS_DISPO | 3 | NULL | 1199 | 100.00 | Using index; Using join buffer |
+----+-------------+--------------+-------+---------------+------------------------------+---------+------+------+----------+--------------------------------+
My question is how to calculate the product of the join using explain. For example, in this case are performed 3597 (1199x3) scans or only 1199?
1)If I add "ORDER BY DDATA" lines scanned in the table "PMS_DISPO" become 1130.
2)If I use the "WHERE" clause instead of "HAVING" clause scan no longer uses the indexes. How is it possible?
3)If i want show PMS_STAGIONI.CSTAGIONI (primary key) explain show me that:
+----+-------------+--------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------------+
| 1 | SIMPLE | PMS_STAGIONI | ALL | NULL | NULL | NULL | NULL | 3 | 100.00 | Using temporary |
| 1 | SIMPLE | PMS_DISPO | index | NULL | IDX_DDATA_PMS_DISPO | 3 | NULL | 1130 | 100.00 | Using index; Using join buffer |
+----+-------------+--------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------------+
How can I force the use of the other index?
Thanks in advance.
Edit:
The structure of "PMS_DISPO" is:
CREATE TABLE IF NOT EXISTS `PMS_DISPO` (
`ID` int(11) NOT NULL AUTO_INCREMENT ,
`CPRENOTA` int(11) NOT NULL,
`DDATA` date NOT NULL,
`CCATRIS` int(4) NOT NULL,
`NQUANT` int(4) NOT NULL,
`CAZIENDA` int(4) NOT NULL,
`CAFFILIATO` int(4) NOT NULL,
PRIMARY KEY (`ID`),
KEY `IDX_DDATA_PMS_DISPO` (`DDATA`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1084 ;
And "PMS_STAGIONI" is:
CREATE TABLE IF NOT EXISTS `PMS_STAGIONI` (
`CSTAGIONE` int(11) NOT NULL,
`NVALIDI` tinyint(2) NOT NULL,
`BECCEZIONE` tinyint(1) NOT NULL,
`AGGSET` varchar(7) DEFAULT NULL,
`DINIZVAL` date NOT NULL,
`DFINEVAL` date NOT NULL,
`CAZIENDA` int(4) NOT NULL,
`ID` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`ID`),
KEY `CSTAGIONE` (`CSTAGIONE`),
KEY `IDX_INIZFINEVAL_PMS_STAGIONI` (`DINIZVAL`,`DFINEVAL`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=4 ;
A query of this sort would normally be written as follows, with indexes just as you have them...
SELECT DISTINCT s.dinizval
, s.dfineval
, d.ddata
FROM pms_dispo d
JOIN pms_stagioni s
ON d.ddata BETWEEN s.dinizval AND s.dfineval
I have two tables with matches and users.
I'm trying to find the way to get the top countries playing matches, and I have this SQL:
select
distinct(user.country),
count(*) as counter
from matches
left join user on matches.user_id = user.id
where
matches.`date` between '2014-01-01' and '2014-03-15'
group by user.country
order by counter DESC
limit 10
The problem is that I'm getting "Using where; Using temporary; Using file sort" and the sql takes about 8s in a m3.medium RDS Amazon server (not bad one!)
I have user.country indexed. Both tables are InnoDB.
Any ideas to improve it ?
Tables:
CREATE TABLE `user` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`nick` varchar(32) DEFAULT NULL,
`email` varchar(128) DEFAULT NULL,
`password` varchar(40) DEFAULT NULL,
`country` char(2) DEFAULT '',
PRIMARY KEY (`id`),
KEY `country` (`country`),
) ENGINE=InnoDB AUTO_INCREMENT=254183 DEFAULT CHARSET=utf8;
CREATE TABLE `matches` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(11) unsigned DEFAULT NULL,
`date` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `date` (`date`)
) ENGINE=InnoDB AUTO_INCREMENT=2593195 DEFAULT CHARSET=utf8;
EXPLAIN gives:
+----+-------------+---------+--------+-----------------+---------+---------+----------------------------+---------+------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+--------+-----------------+---------+---------+----------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | matches | ALL | date | NULL | NULL | NULL | 2386708 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | user | eq_ref | PRIMARY,country | PRIMARY | 4 | matches.user_id | 1 | NULL |
+----+-------------+---------+--------+-----------------+---------+---------+----------------------------+---------+----------------------------------------------+
EDIT: Changing to inner join:
+----+-------------+----------+-------+------------------------------+-----------------+---------+------------------+--------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+------------------------------+-----------------+---------+------------------+--------+----------------------------------------------+
| 1 | SIMPLE | user | index | PRIMARY,country | country | 7 | NULL | 234262 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | matches | ref | user_id,date | user_id | 5 | user.id | 5 | Using where |
+----+-------------+----------+-------+------------------------------+-----------------+---------+------------------+--------+----------------------------------------------+
I'm trying to order by and use an index but it seems to still be using filesort
Table
CREATE TABLE `teachers` (
`id` int(10) unsigned NOT NULL,
`name` VARCHAR(10) NOT NULL,
`lastName` tinytext NOT NULL,
KEY `lastName` (lastName(10))
)
Query
mysql> EXPLAIN SELECT name,lastName FROM teachers ORDER BY lastName DESC;
+----+-------------+----------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | teachers | ALL | NULL | NULL | NULL | NULL | 1546 | Using filesort |
+----+-------------+----------+------+---------------+------+---------+------+------+----------------+
You need to create a covering index last name, name
CREATE TABLE `teachers` (
`id` int(10) unsigned NOT NULL,
`name` varchar(10) NOT NULL,
`lastName` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
KEY `last_first` (`lastName`,`name`)
)
Now that query
EXPLAIN SELECT `name`, lastName FROM teachers ORDER BY lastName DESC;
Produces the following result
+----+-------------+----------+-------+---------------+------------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+---------------+------------+---------+------+------+-------------+
| 1 | SIMPLE | teachers | index | NULL | last_first | 269 | NULL | 1 | Using index |
+----+-------------+----------+-------+---------------+------------+---------+------+------+-------------+
Your problem is in key definition: KEY lastName (lastName(10))
Especially that you have defined a KEY length here
You index only a prefix of a column named in the ORDER BY clause. In this case, the index cannot be used to fully resolve the sort order. For example, if you have a CHAR(20) column, but index only the first 10 bytes, the index cannot distinguish values past the 10th byte and a filesort will be needed.
Proof link
You can write KEY lastName (lastName) and it will work as you need.
First, I'm creating table tag:
CREATE TABLE `tag` (
`id` smallint(6) NOT NULL AUTO_INCREMENT,
`total` int(11) DEFAULT NULL,
`total_question` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_sort` (`total`,`total_question`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=gb2312;
mysql> explain select * from tag order by total;
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------------+
| 1 | SIMPLE | tag | index | NULL | idx_sort | 10 | NULL | 1 | Using index |
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------------+
Sort using index, not using filesort.
When I add column name to tag table:
CREATE TABLE `tag` (
`id` smallint(6) NOT NULL AUTO_INCREMENT,
`total` int(11) DEFAULT NULL,
`total_question` int(11) DEFAULT NULL,
`name` char(20) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_sort` (`total`,`total_question`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=gb2312;
mysql> explain select * from tag order by total;
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | tag | ALL | NULL | NULL | NULL | NULL | 1 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
Sort using filesort, not using index.
When I create index only on total:
CREATE TABLE `tag` (
`id` smallint(6) NOT NULL AUTO_INCREMENT,
`total` int(11) DEFAULT NULL,
`total_question` int(11) DEFAULT NULL,
`name` char(20) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_sort` (`total`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=gb2312;
mysql> explain select * from tag order by total;
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | tag | ALL | NULL | NULL | NULL | NULL | 1 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
Sort using filesort! Why? I only use the total column for sort.
you can run desc select * from tag force index (idx_sort) order by total ;
you can see the output :
mysql> desc select * from tag force index (idx_sort) order by total ;
+----+-------------+-------+-------+---------------+----------+---------+------+------+--- ----+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--- ----+
| 1 | SIMPLE | tag | index | NULL | idx_sort | 5 | NULL | 1 | |
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------+