How MySQL implements the loose index scan - mysql

Question

Recently,I face a question how mysql implements the loose index scan?
For example:
the test table structure is:
CREATE TABLE test (
id int(11) NOT NULL default '0',
v1 int(10) unsigned NOT NULL default '0',
v2 int(10) unsigned NOT NULL default '0',
v3 int(10) unsigned NOT NULL default '0',
PRIMARY KEY (id),
KEY v1_v2_v3 (v1,v2,v3)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
select * from test;
+----+----+-----+----+
| id | v1 | v2 | v3 |
+----+----+-----+----+
| 1 | 1 | 0 | 1 |
| 2 | 3 | 1 | 2 |
| 10 | 4 | 10 | 10 |
| 0 | 4 | 100 | 0 |
| 3 | 4 | 100 | 3 |
| 5 | 5 | 9 | 5 |
| 8 | 7 | 3 | 8 |
| 7 | 7 | 4 | 7 |
| 30 | 8 | 15 | 30 |
+----+----+-----+----+
Now let's see two sql:
first one:
mysql> explain select v1,v2 from test group by v1,v2;
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
| 1 | SIMPLE | test | range | NULL | v1_v2_v3 | 8 | NULL | 3 | Using index for group-by |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)
I know that Using index for group-by means MySQL use the loose index scan to query the sql.But why the explain output column rows is 3?I wonder how MySQL only scan three rows and get the query result.
second one:
mysql> explain select max(v3) from test where v1>3 group by v1,v2;
+----+-------------+-------+-------+---------------+----------+---------+------+------+---------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+---------------------------------------+
| 1 | SIMPLE | test | range | v1_v2_v3 | v1_v2_v3 | 8 | NULL | 1 | Using where; Using index for group-by |
+----+-------------+-------+-------+---------------+----------+---------+------+------+---------------------------------------+
1 row in set (0.00 sec)
mysql> explain select max(v2) from test where v1>3 group by v1,v2;
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
| 1 | SIMPLE | test | range | v1_v2_v3 | v1_v2_v3 | 4 | NULL | 4 | Using where; Using index |
+----+-------------+-------+-------+---------------+----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)
the only difference between the above two sql is in the select list,one is max(v3),another one is max(v2).But why the max(v3) uses the loose index scan,the max(v2) don't use the loose index scan? I don't unnderstand the GROUP BY Optimization says:
The only aggregate functions used in the select list (if any) are MIN() and MAX(), and all of them refer to the same column. The column must be in the index and must immediately follow the columns in the GROUP BY.
why the column must immediately follow the columns in the GROUP BY?
I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.
Thanks!

Answer 1

This is too long for a comment.
Essentially, when asking "why does the optimizer behave a certain way", the answer is because the designers implemented it that way. If you want to know "why", you would have to ask them . . . that is not an appropriate question for a general-purpose forum.
I want to point out a few things, though. If you think that that the max(v2) is a bug, then you can report it at bugs.mysql.com. I don't think it is a bug for two reasons:
The documentation explicitly states how the optimization works, and this query is not documented to use the index ("v2" does not follow the keys in the group by).
Even if it were documented differently, the use of an aggregation function on a group by key is, shall I say, non-sensical. It is valid SQL, but it is simply verbose and unnecessary. Such constructs are way down on the list of priorities for database implementors.
Finally, MySQL does not really use statistics (very well?) when creating the query plan. However, in most databases, validating a query plan on 9 rows (which fit on a single data page) often results in a query plan that does a full table scan and "inefficient" algorithms. As an example, an algorithm such as bubble sort is quite inefficient on large numbers of rows, but it can be the most efficient sorting algorithm on a (very) small number of rows.

Answer 2

Is there any reason to use max (v2) in the query? The result is the same even if you do not use the max () function. If you change the query to "select v2 from test where v1> 3 group by v1, v2 ", it will be done by loose index scan method.
And here are the reasons why the column must immediately follow the columns in the GROUP BY.
v1 v2 v3
1 1 1
1 1 2
1 1 10
1 2 1
1 2 2
1 2 8
In this case, select max (v3) from t1 group by v1, v2 to perform loose index scan. This is done as shown in the following figure.
v1 v2 v3
1 1 1
1 1 2
1 1 10 ------------------> 10 return
1 2 1
1 2 2
1 2 8 ------------------> 8 return
However, if you perform select max (v3) from t1 group by v1, loose index scan is not possible. Because you have to access all the keys to find the maximum value(=10).
v1 v2 v3
1 1 1 ------------------> (x)
1 1 2 ------------------> (x)
1 1 10 ------------------> 10 return
1 2 1 ------------------> (x)
1 2 2 ------------------> (x)
1 2 8 ------------------> (x)
Note that you can use the following command to see how many records are accessed using loose index scan (or tight index scan).
flush status;
select max(v3) from t1 group by v1,v2; -- perform loose index scan
show session status like 'Handler_read_key%';
flush status;
select max(v3) from t1 group by v1; -- perform tight index scan
show session status like 'Handler_read_key%';