Order by text field with an index - using filesort - mysql

I'm trying to order by and use an index but it seems to still be using filesort
Table
CREATE TABLE `teachers` (
`id` int(10) unsigned NOT NULL,
`name` VARCHAR(10) NOT NULL,
`lastName` tinytext NOT NULL,
KEY `lastName` (lastName(10))
)
Query
mysql> EXPLAIN SELECT name,lastName FROM teachers ORDER BY lastName DESC;
+----+-------------+----------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | teachers | ALL | NULL | NULL | NULL | NULL | 1546 | Using filesort |
+----+-------------+----------+------+---------------+------+---------+------+------+----------------+

You need to create a covering index last name, name
CREATE TABLE `teachers` (
`id` int(10) unsigned NOT NULL,
`name` varchar(10) NOT NULL,
`lastName` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
KEY `last_first` (`lastName`,`name`)
)
Now that query
EXPLAIN SELECT `name`, lastName FROM teachers ORDER BY lastName DESC;
Produces the following result
+----+-------------+----------+-------+---------------+------------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+---------------+------------+---------+------+------+-------------+
| 1 | SIMPLE | teachers | index | NULL | last_first | 269 | NULL | 1 | Using index |
+----+-------------+----------+-------+---------------+------------+---------+------+------+-------------+

Your problem is in key definition: KEY lastName (lastName(10))
Especially that you have defined a KEY length here
You index only a prefix of a column named in the ORDER BY clause. In this case, the index cannot be used to fully resolve the sort order. For example, if you have a CHAR(20) column, but index only the first 10 bytes, the index cannot distinguish values past the 10th byte and a filesort will be needed.
Proof link
You can write KEY lastName (lastName) and it will work as you need.

Related

Use of MySQL explain

I have this query:
EXPLAIN EXTENDED
SELECT DISTINCT
PMS_STAGIONI.DINIZVAL,
PMS_STAGIONI.DFINEVAL,
PMS_DISPO.DDATA
FROM
PMS_DISPO JOIN PMS_STAGIONI
HAVING
PMS_DISPO.DDATA BETWEEN PMS_STAGIONI.DINIZVAL AND PMS_STAGIONI.DFINEVAL
The output of explain is:
+----+-------------+--------------+-------+---------------+------------------------------+---------+------+------+----------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------+-------+---------------+------------------------------+---------+------+------+----------+--------------------------------+
| 1 | SIMPLE | PMS_STAGIONI | index | NULL | IDX_INIZFINEVAL_PMS_STAGIONI | 6 | NULL | 3 | 100.00 | Using index; Using temporary |
| 1 | SIMPLE | PMS_DISPO | index | NULL | IDX_DDATA_PMS_DISPO | 3 | NULL | 1199 | 100.00 | Using index; Using join buffer |
+----+-------------+--------------+-------+---------------+------------------------------+---------+------+------+----------+--------------------------------+
My question is how to calculate the product of the join using explain. For example, in this case are performed 3597 (1199x3) scans or only 1199?
1)If I add "ORDER BY DDATA" lines scanned in the table "PMS_DISPO" become 1130.
2)If I use the "WHERE" clause instead of "HAVING" clause scan no longer uses the indexes. How is it possible?
3)If i want show PMS_STAGIONI.CSTAGIONI (primary key) explain show me that:
+----+-------------+--------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------------+
| 1 | SIMPLE | PMS_STAGIONI | ALL | NULL | NULL | NULL | NULL | 3 | 100.00 | Using temporary |
| 1 | SIMPLE | PMS_DISPO | index | NULL | IDX_DDATA_PMS_DISPO | 3 | NULL | 1130 | 100.00 | Using index; Using join buffer |
+----+-------------+--------------+-------+---------------+---------------------+---------+------+------+----------+--------------------------------+
How can I force the use of the other index?
Thanks in advance.
Edit:
The structure of "PMS_DISPO" is:
CREATE TABLE IF NOT EXISTS `PMS_DISPO` (
`ID` int(11) NOT NULL AUTO_INCREMENT ,
`CPRENOTA` int(11) NOT NULL,
`DDATA` date NOT NULL,
`CCATRIS` int(4) NOT NULL,
`NQUANT` int(4) NOT NULL,
`CAZIENDA` int(4) NOT NULL,
`CAFFILIATO` int(4) NOT NULL,
PRIMARY KEY (`ID`),
KEY `IDX_DDATA_PMS_DISPO` (`DDATA`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1084 ;
And "PMS_STAGIONI" is:
CREATE TABLE IF NOT EXISTS `PMS_STAGIONI` (
`CSTAGIONE` int(11) NOT NULL,
`NVALIDI` tinyint(2) NOT NULL,
`BECCEZIONE` tinyint(1) NOT NULL,
`AGGSET` varchar(7) DEFAULT NULL,
`DINIZVAL` date NOT NULL,
`DFINEVAL` date NOT NULL,
`CAZIENDA` int(4) NOT NULL,
`ID` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`ID`),
KEY `CSTAGIONE` (`CSTAGIONE`),
KEY `IDX_INIZFINEVAL_PMS_STAGIONI` (`DINIZVAL`,`DFINEVAL`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=4 ;
A query of this sort would normally be written as follows, with indexes just as you have them...
SELECT DISTINCT s.dinizval
, s.dfineval
, d.ddata
FROM pms_dispo d
JOIN pms_stagioni s
ON d.ddata BETWEEN s.dinizval AND s.dfineval

why index is not using in order by (foreign key)

when i use id (primary key) with order by clause it uses index named PRIMARY but when i use countrycode (foreign key) with order by clause it does't uses index. my output is below.
mysql> SHOW CREATE TABLE City;
+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| City | CREATE TABLE `City` (
| | `ID` int(11) NOT NULL AUTO_INCREMENT,
| | `Name` char(35) NOT NULL DEFAULT '',
| | `CountryCode` char(3) NOT NULL DEFAULT '',
| | `District` char(20) NOT NULL DEFAULT '',
| | `Population` int(11) NOT NULL DEFAULT '0',
| | PRIMARY KEY (`ID`),
| | KEY `CountryCode` (`CountryCode`),
| | CONSTRAINT `city_ibfk_1` FOREIGN KEY (`CountryCode`) REFERENCES `Country` (`Code`)
| | ) ENGINE=InnoDB AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
mysql> EXPLAIN SELECT * FROM City ORDER BY ID;
+----+-------------+-------+-------+---------------+---------+---------+------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+---------+---------+------+------+-------+
| 1 | SIMPLE | City | index | NULL | PRIMARY | 4 | NULL | 4321 | |
+----+-------------+-------+-------+---------------+---------+---------+------+------+-------+
1 row in set (0.00 sec)
mysql> EXPLAIN SELECT * FROM City ORDER BY COUNTRYCODE;
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | City | ALL | NULL | NULL | NULL | NULL | 4321 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
1 row in set (0.00 sec)
In innodb primary keys are so called "clustered indexes".
It means that the rows physically ordered according the PK values.
Because of that the rows are naturally sorted so it's cheap to read them sorted ASC or DESC.
Another story is when you order by another column.
To use that mysql would have to read both index and data pages, which dramatically increases IO. So mysql decides to sort it in memory instead (because according to its heuristics memory sort is faster than increased IO). If you want to see mysql using that index for sorting you need:
Increase total number of rows to, say several dozens of thousands
Select only a small subset, like LIMIT 10
Then mysql might decide to use index.

SQL grouping and counting, and try to avoid temporary and filesort

I have two tables with matches and users.
I'm trying to find the way to get the top countries playing matches, and I have this SQL:
select
distinct(user.country),
count(*) as counter
from matches
left join user on matches.user_id = user.id
where
matches.`date` between '2014-01-01' and '2014-03-15'
group by user.country
order by counter DESC
limit 10
The problem is that I'm getting "Using where; Using temporary; Using file sort" and the sql takes about 8s in a m3.medium RDS Amazon server (not bad one!)
I have user.country indexed. Both tables are InnoDB.
Any ideas to improve it ?
Tables:
CREATE TABLE `user` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`nick` varchar(32) DEFAULT NULL,
`email` varchar(128) DEFAULT NULL,
`password` varchar(40) DEFAULT NULL,
`country` char(2) DEFAULT '',
PRIMARY KEY (`id`),
KEY `country` (`country`),
) ENGINE=InnoDB AUTO_INCREMENT=254183 DEFAULT CHARSET=utf8;
CREATE TABLE `matches` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(11) unsigned DEFAULT NULL,
`date` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `date` (`date`)
) ENGINE=InnoDB AUTO_INCREMENT=2593195 DEFAULT CHARSET=utf8;
EXPLAIN gives:
+----+-------------+---------+--------+-----------------+---------+---------+----------------------------+---------+------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+--------+-----------------+---------+---------+----------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | matches | ALL | date | NULL | NULL | NULL | 2386708 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | user | eq_ref | PRIMARY,country | PRIMARY | 4 | matches.user_id | 1 | NULL |
+----+-------------+---------+--------+-----------------+---------+---------+----------------------------+---------+----------------------------------------------+
EDIT: Changing to inner join:
+----+-------------+----------+-------+------------------------------+-----------------+---------+------------------+--------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+------------------------------+-----------------+---------+------------------+--------+----------------------------------------------+
| 1 | SIMPLE | user | index | PRIMARY,country | country | 7 | NULL | 234262 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | matches | ref | user_id,date | user_id | 5 | user.id | 5 | Using where |
+----+-------------+----------+-------+------------------------------+-----------------+---------+------------------+--------+----------------------------------------------+

MySQL index isn't used in simple select query

I have a table
CREATE TABLE `pd` (
`product_id` int(11) NOT NULL AUTO_INCREMENT,
`language_id` int(11) NOT NULL,
`name` varchar(255) NOT NULL,
`description` text NOT NULL,
`meta_description` varchar(255) NOT NULL,
`meta_keyword` varchar(255) NOT NULL,
`seo_title` varchar(255) NOT NULL,
`seo_h1` varchar(255) NOT NULL,
PRIMARY KEY (`product_id`,`language_id`),
KEY `language_id` (`description`(128),`language_id`),
KEY `name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=47019 DEFAULT CHARSET=utf8;
When I run this query
EXPLAIN SELECT * FROM `pd` ORDER BY product_id;
I get this result:
+----+-------------+-------+-------+---------------+---------+---------+------+--------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+---------+---------+------+--------+-------+
| 1 | SIMPLE | pd | index | NULL | PRIMARY | 8 | NULL | 139551 | |
+----+-------------+-------+-------+---------------+---------+---------+------+--------+-------+
When I run this query
EXPLAIN SELECT * FROM `pd` ORDER BY name;
I get this result:
+----+-------------+-------+------+---------------+------+---------+------+--------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+--------+----------------+
| 1 | SIMPLE | pd | ALL | NULL | NULL | NULL | NULL | 137762 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+--------+----------------+
Why in second case index isn't used? Only difference I see is product_id is part of primary key and name is non-unique index.
As soon as the name isn't a clustered key - it will be much more expensive to read the index first and seek for every row one by one in data.
So mysql (and I think that any other would DBMS) decdies to read the data and sort it in-memory.

mysql use the index optimization sortingļ¼

First, I'm creating table tag:
CREATE TABLE `tag` (
`id` smallint(6) NOT NULL AUTO_INCREMENT,
`total` int(11) DEFAULT NULL,
`total_question` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_sort` (`total`,`total_question`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=gb2312;
mysql> explain select * from tag order by total;
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------------+
| 1 | SIMPLE | tag | index | NULL | idx_sort | 10 | NULL | 1 | Using index |
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------------+
Sort using index, not using filesort.
When I add column name to tag table:
CREATE TABLE `tag` (
`id` smallint(6) NOT NULL AUTO_INCREMENT,
`total` int(11) DEFAULT NULL,
`total_question` int(11) DEFAULT NULL,
`name` char(20) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_sort` (`total`,`total_question`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=gb2312;
mysql> explain select * from tag order by total;
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | tag | ALL | NULL | NULL | NULL | NULL | 1 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
Sort using filesort, not using index.
When I create index only on total:
CREATE TABLE `tag` (
`id` smallint(6) NOT NULL AUTO_INCREMENT,
`total` int(11) DEFAULT NULL,
`total_question` int(11) DEFAULT NULL,
`name` char(20) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_sort` (`total`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=gb2312;
mysql> explain select * from tag order by total;
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | tag | ALL | NULL | NULL | NULL | NULL | 1 | Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
Sort using filesort! Why? I only use the total column for sort.
you can run desc select * from tag force index (idx_sort) order by total ;
you can see the output :
mysql> desc select * from tag force index (idx_sort) order by total ;
+----+-------------+-------+-------+---------------+----------+---------+------+------+--- ----+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--- ----+
| 1 | SIMPLE | tag | index | NULL | idx_sort | 5 | NULL | 1 | |
+----+-------------+-------+-------+---------------+----------+---------+------+------+-------+