Order by not picking up the right indexes - mysql

Question

I have projects table and each project has multiple categories assigned. The category mapping is stored in the project_category table. I want to list all recent projects that are not expired. Here is the schema, indexes and query.
Schema
Create table projects (
project_id Bigint UNSIGNED NOT NULL AUTO_INCREMENT,
project_title Varchar(300) NOT NULL,
date_added Datetime NOT NULL,
is_expired Bit(1) NOT NULL DEFAULT false,
Primary Key (project_id)) ENGINE = InnoDB;
Create table project_category (
project_category_id Int UNSIGNED NOT NULL AUTO_INCREMENT,
cat_id Int UNSIGNED NOT NULL,
project_id Bigint UNSIGNED NOT NULL,
Primary Key (project_category_id)) ENGINE = InnoDB;
Indexes
CREATE INDEX project_listing (is_expired, date_added) ON projects;
Create INDEX category_mapping_IDX ON project_category (project_id,cat_id);
Query
mysql> EXPLAIN
SELECT P.project_id
FROM projects P
INNER JOIN project_category C USING (project_id)
WHERE P.is_expired=false
AND C.cat_id=17
ORDER BY P.date_added DESC LIMIT 27840,10;
+----+-------------+-------+--------+--------------------------------------------+---------+---------+-------------------------+--------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+--------------------------------------------+---------+---------+-------------------------+--------+---------------------------------+
| 1 | SIMPLE | C | ref | project_id,cat_id,category_mapping_IDX | cat_id | 4 | const | 185088 | Using temporary; Using filesort |
| 1 | SIMPLE | P | eq_ref | PRIMARY,is_expired_INX,project_listing_IDX | PRIMARY | 8 | freelancer.C.project_id | 1 | Using where |
+----+-------------+-------+--------+--------------------------------------------+---------+---------+-------------------------+--------+---------------------------------+
I am wondering why MySQL isn't using the index on project_category, and why it is doing a full sort?
I also tried the following query just to avoid file sorting, but it is not working either.
mysql> EXPLAIN
SELECT P.project_id
FROM projects P,
(
SELECT P.project_id
FROM projects P
INNER JOIN project_category C USING (project_id)
WHERE C.cat_id=17
) F
WHERE F.project_id=P.project_id
AND P.is_expired=FALSE
LIMIT 10;
+----+-------------+------------+--------+--------------------------------------------+---------+---------+-------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+--------------------------------------------+---------+---------+-------------------------+--------+-------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 110920 | |
| 1 | PRIMARY | P | eq_ref | PRIMARY,is_expired_INX,project_listing_IDX | PRIMARY | 8 | F.project_id | 1 | Using where |
| 2 | DERIVED | C | ref | project_id,cat_id,category_mapping_IDX | cat_id | 4 | | 185088 | |
| 2 | DERIVED | P | eq_ref | PRIMARY | PRIMARY | 8 | freelancer.C.project_id | 1 | Using index |
+----+-------------+------------+--------+--------------------------------------------+---------+---------+-------------------------+--------+-------------+

Answer 1

Your problem is here:
Create INDEX category_mapping_IDX ON project_category (project_id,cat_id);
This index is not useful when you're trying to subselect a single cat_id, because cat_id is not the first part of the index. Think of the index as a concatenated string and you can see why it can not be used. Swap the order:
Create INDEX category_mapping_IDX ON project_category (cat_id, project_id);