query doesn't use index when using IN subquery - mysql

Question

You will notice the primary query is NOT using an index on school_id, when there is an index. If I remove the subquery and use a hardcoded list, it will use an index.
mysql> explain SELECT year, race, CONCAT(percent,'%') as percent
-> FROM school_data_race_ethnicity as school_data_race_ethnicity_outer
-> WHERE school_id IN(
-> SELECT field_school_id_value
-> FROM field_data_field_school_id
-> WHERE entity_id IN (SELECT entity_id
-> FROM field_data_field_district
-> WHERE field_district_nid =
-> (SELECT entity_id FROM field_data_field_district_id
-> WHERE `field_district_id_value` = 26130106 LIMIT 1))
-> ) ORDER BY year DESC, race;
+----+--------------------+----------------------------------+----------------+------------------------------+-----------+---------+------+-------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+----------------------------------+----------------+------------------------------+-----------+---------+------+-------+-----------------------------+
| 1 | PRIMARY | school_data_race_ethnicity_outer | ALL | NULL | NULL | NULL | NULL | 97116 | Using where; Using filesort |
| 2 | DEPENDENT SUBQUERY | field_data_field_school_id | ALL | NULL | NULL | NULL | NULL | 5325 | Using where |
| 3 | DEPENDENT SUBQUERY | field_data_field_district | index_subquery | entity_id,field_district_nid | entity_id | 4 | func | 1 | Using where |
| 4 | SUBQUERY | field_data_field_district_id | ALL | NULL | NULL | NULL | NULL | 685 | Using where |
+----+--------------------+----------------------------------+----------------+------------------------------+-----------+---------+------+-------+-----------------------------+
4 rows in set (0.00 sec)
mysql> describe school_data_race_ethnicity
-> ;
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| school_id | varchar(255) | NO | MUL | NULL | |
| year | int(11) | NO | MUL | NULL | |
| race | varchar(255) | NO | | NULL | |
| percent | decimal(5,2) | NO | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
mysql>

Answer 1

use INNER JOIN instead of subquery with IN clause:
explain SELECT year, race, CONCAT(percent,'%') as percent
FROM school_data_race_ethnicity a
INNER JOIN(
SELECT field_school_id_value
FROM field_data_field_school_id b
INNER JOIN (SELECT entity_id
FROM field_data_field_district
WHERE field_district_nid =
(SELECT entity_id FROM field_data_field_district_id
WHERE `field_district_id_value` = 26130106 LIMIT 1)) c
ON b. entity_id = c.entity_id
) d
ON a.school_id = d.field_school_id_value
ORDER BY year DESC, race;

Answer 2

The server probably is skipping indexed seek because it's not able to figure out how many items will actually be in the list. The thinking is that since it doesn't have a good estimate of the number of items in the subquery, it's going to choose the most conservative approach and go with a scan (probably).
I'd suggest rewriting the query using JOINs instead:
SELECT DISTINCT o.year, o.race, CONCAT(o.percent,'%') as percent
FROM school_data_race_ethnicity as o
INNER JOIN field_data_field_school_id s
ON s.field_school_id_value = o.school_id
INNER JOIN field_data_field_district d
ON d.entity_id = s.entity_id
WHERE field_district_nid = (
SELECT entity_id
FROM field_data_field_district_id
WHERE `field_district_id_value` = 26130106 LIMIT 1))
ORDER BY o.year DESC, o.race;