Make MySQL read from multiple indexes? - mysql

Question

Let's start off with a simple example:
CREATE TABLE `test` (
`id` INT UNSIGNED NOT NULL,
`value` CHAR(12) NOT NULL,
INDEX (`id`),
INDEX (`value`)
) ENGINE = InnoDB;
So 2 columns, both indexed. What I thought this meant was that MySQL would never have to read the actual table anymore, since all the data is stored in an index.
mysql> EXPLAIN SELECT id FROM test WHERE id = 1;
+----+-------------+-------+------+---------------+------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+-------+------+-------------+
| 1 | SIMPLE | test | ref | id | id | 4 | const | 1 | Using index |
+----+-------------+-------+------+---------------+------+---------+-------+------+-------------+
"Using index", very nice. To my understanding this means that it is reading data from the index and not from the actual table. But what I really want is the "value" column.
mysql> EXPLAIN SELECT value FROM test WHERE id = 1;
+----+-------------+-------+------+---------------+------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+-------+------+-------+
| 1 | SIMPLE | test | ref | id | id | 4 | const | 1 | |
+----+-------------+-------+------+---------------+------+---------+-------+------+-------+
Hmm, no "using index" this time.
I thought it might help if I add an index that covers both columns.
ALTER TABLE `test` ADD INDEX `id_value` (`id`,`value`);
Now let's run that previous select-statement again and tell it to use the new index.
mysql> EXPLAIN SELECT id, value FROM test USE INDEX (id_value) WHERE id = 1;
+----+-------------+-------+------+---------------+----------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+----------+---------+-------+------+-------------+
| 1 | SIMPLE | test | ref | id_value | id_value | 4 | const | 1 | Using index |
+----+-------------+-------+------+---------------+----------+---------+-------+------+-------------+
Praise the Lord, it's reading from the index.
But actually I don't really need the combined index for anything else. Is it possible to make MySQL read from 2 separate indexes?
Any insights would be greatly appreciated.
EDIT: Ok, yet another example. This one is with the original table definition (so an index on each column).
mysql> EXPLAIN SELECT t1.value
-> FROM test AS t1
-> INNER JOIN test AS t2
-> ON t1.id <> t2.id AND t1.value = t2.value
-> WHERE t1.id = 1;
+----+-------------+-------+------+---------------+-------+---------+----------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+-------+---------+----------+------+-------------+
| 1 | SIMPLE | t1 | ref | id,value | id | 4 | const | 1 | |
| 1 | SIMPLE | t2 | ref | value | value | 12 | t1.value | 1 | Using where |
+----+-------------+-------+------+---------------+-------+---------+----------+------+-------------+
This must certainly read from both indexes (since both fields are used in the join condition) yet it STILL reads the data from the actual record, right? Why doesn't it just use the data it has read from the index? Or does it actually use that data without saying "using index"?
Thanks again

Answer 1

The key, ref and rows columns are more telling for this purpose. In each case, they indicate that MySQL has selected an index, has a value to lookup in that index, and is retrieving only one row from the table as a result. This is what you were after.
In your second query, MySQL still needs to retrieve the value from the record even though it has located the record on id via an index. If your WHERE criterion looked up based on value, then that index would have been used and there would have been no need to retrieve the record.
The manual on Using index Extra information:
The column information is retrieved from the table using only information in the index tree without having to do an additional seek to read the actual row. This strategy can be used when the query uses only columns that are part of a single index.
If the Extra column also says Using where, it means the index is being used to perform lookups of key values. Without Using where, the optimizer may be reading the index to avoid reading data rows but not using it for lookups. For example, if the index is a covering index for the query, the optimizer may scan it without using it for lookups.
For InnoDB tables that have a user-defined clustered index, that index can be used even when Using index is absent from the Extra column. This is the case if type is index and key is PRIMARY.

Answer 2

In your first query, MySQL says using index because it can answer your query by looking at the index and the index alone. It does not need to go to the table to look up the corresponding value for the id column, because that's actually the same thing it's already got in the index.
In the second query, MySQL does need to look at the table to fetch the correct value, but it's still using the index, as you can see in the key column of your EXPLAIN statement.
In the third query, MySQL again doesn't have to look at the table anymore, because all the information it needs to answer your query is right there in the multiple-column index.

Answer 3

Just think a bit how indexes works.
Say, you have 10k records in your test table and index on the value column. While you're populating your table with data (or explicitly using ANALYZE command), database is keeping statistics on your table and all indexes.
At the moment you issue your query, there're several ways how to deliver you the data. In the very simplified case of test table and value column, something like:
SELECT * FROM test WHERE value = 'a string';
database query planner has 2 options:
performing a sequential scan on the whole table and filter the results or
performing index scan to lookup the desired data entries.
Querying indexes has some performance penalty, as database must seek for the value in the index. If we take that you have a B-tree index in a "good shape" (i.e. balanced), then you'll find your entry in at most 14 lookups in the index (as 2^14 > 10k, I hope I'm not mistaken here). So, in order to deliver you 1 row with a string value, database will have to perform up to 14 lookups in the index and 1 extra lookup in your table. In the unlucky case, this will mean system will perform 15 random I/O operations to read in custom data portions from your disk.
In the case there's only one value that requires lookup in the index and that your table is quite big in size, index operations will give you a significant performance boost.
But there's a point after which index scan becomes more expensive, then a straightforward sequential scan:
when your table is occupying really small size on the disk;
when your query will require lookup of round 10% of the total number of records in the test table (the number 10% very approximate, don't take it for granted).
Things to consider:
comparison operations for numeric data types are significantly cheaper, then comparing strings;
statistics accuracy;
how often index / table is queried, or which probability it is to find needed data in the database's shared pool.
These all affects performance and also the plans that database chooses to deliver the data.
So, indexes are not always good.
To answer your to read from 2 separate indexes question: feature you're looking for is called Bitmap index, and it is not available in MySQL as far as I know.

Answer 4

New with 5.0, MySQL can utilize more than one index on a table with Index merge, though they're not as speedy (by far) as multi-column covering indexes, so MySQL will only use them in special cases.
So, other than the merge index case, MySQL only uses one index per table.
Don't be too afraid of covering indexes. They can serve double duty. Indexes are left most prefixed, so you can use a multi-column index for just the left most column, or the first and second, and so on.
For example, if you have the multi-column index id_value (id,value), you can delete the index id (id), since it's redundant. The id_value index can also be used for just the id column.
Also, with InnoDB, every index automatically includes the primary key column(s), so if id were your primary key, an index on value provides the same benefit as having a covering index on (id, value).
Every index does negatively affect inserts, and updates against the indexed columns. There's a trade off, and only you (and some testing) can decide if covering indexes are right for you.
Deletes don't have much impact on indexes because they're just "marked for deletion", and they only get purged when your system's load is low.
Indexes also use up memory. Given enough memory, a properly configured MySQL server will have every index loaded in memory. This makes selects that utilize a covering index super fast.