Nested set index issue - mysql

Question

I have a query which is running a bit slow. It takes a table containing the items that result from a search and then gets the categories that contain 1 or more of these items. The categories (~300) are stored in a nested set model with around 4 levels. I need to know the counts at each level (so an item might be in the grand child category, but also needs to be counted in the child and parent categories).
The query to do this is as follows:-
SELECT category_parent.id,
category_parent.depth,
category_parent.name AS item_sub_category,
category_parent.left_index,
category_parent.right_index,
COUNT(DISTINCT item.id) as total
FROM search_enquiries_found
INNER JOIN item ON search_enquiries_found.item_id = item.id
INNER JOIN category ON item.mmg_code = category.mmg_code
INNER JOIN category category_parent ON category.left_index BETWEEN category_parent.left_index AND category_parent.right_index
WHERE search_enquiries_found.search_enquiry_id = 35
AND item.cost_price > 0
GROUP BY category_parent.id,
category_parent.depth,
item_sub_category,
category_parent.left_index,
category_parent.right_index
ORDER BY category_parent.left_index
With around 12k records found this is taking ~1.5 seconds. The explain follows:-
id select_type table partitions type possible_keys key key_len ref rows Extra
1 SIMPLE category NULL ALL left_index NULL NULL NULL 337 Using temporary; Using filesort
1 SIMPLE item NULL ref PRIMARY,id,mmg_code mmg_code 27 em_entaonline.category.mmg_code 43 Using where
1 SIMPLE search_enquiries_found NULL eq_ref search_enquiry_id,item_id,search_enquiry_id_2,search_enquiry_id_relevance search_enquiry_id 8 const,em_entaonline.item.id 1 Using index
1 SIMPLE category_parent NULL ALL left_index NULL NULL NULL 337 Range checked for each record (index map: 0x2)
The important table is the hierachical category table:-
CREATE TABLE `category` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(128) NOT NULL,
`depth` int(11) NOT NULL,
`is_active` tinyint(1) NOT NULL DEFAULT '1',
`left_index` int(4) NOT NULL,
`right_index` int(4) NOT NULL,
`mmg_code` varchar(25) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `left_index` (`left_index`,`right_index`),
UNIQUE KEY `depth` (`depth`,`left_index`,`right_index`),
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
This is not usefully using any index on the join to get parent categories.
Changing this query to remove most of the group by fields and just using the unique id (in effect the other fields in the group by are not required, but I tend to include them to keep to strict standards) reduces the time taken down to ~0.5 seconds:-
SELECT category_parent.id,
category_parent.depth,
category_parent.name AS item_sub_category,
category_parent.left_index,
category_parent.right_index,
COUNT(DISTINCT item.id) as total
FROM search_enquiries_found
INNER JOIN item ON search_enquiries_found.item_id = item.id
INNER JOIN category ON item.mmg_code = category.mmg_code
INNER JOIN category category_parent ON category.left_index BETWEEN category_parent.left_index AND category_parent.right_index
WHERE search_enquiries_found.search_enquiry_id = 35
AND item.cost_price > 0
GROUP BY category_parent.id
ORDER BY category_parent.left_index
The explain for this is very slightly different:-
id select_type table partitions type possible_keys key key_len ref rows Extra?
1 SIMPLE category NULL ALL left_index NULL NULL NULL 337 Using temporary; Using filesort
1 SIMPLE item NULL ref PRIMARY,id,mmg_code mmg_code 27 em_entaonline.category.mmg_code 43 Using where
1 SIMPLE search_enquiries_found NULL eq_ref search_enquiry_id,item_id,search_enquiry_id_2,search_enquiry_id_relevance search_enquiry_id 8 const,em_entaonline.item.id 1 Using index
1 SIMPLE category_parent NULL ALL PRIMARY,left_index,depth,name NULL NULL NULL 337 Using where; Using join buffer (Block Nested Loop)
Now I have 2 questions.
1 - Why does removing the fields from the GROUP BY (which are completely dependent on the primary key left in the group by) change the EXPLAIN this way, with such a large change in performance.
2 - In either case the performance is poor, with no key used for the join to the parent category table. Any suggestions on improving this?
As a further point, I have tried reversing the order of the joins and forcing it using STRAIGHT_JOIN. In this case this further improves the performance (~0.2 seconds), but this is an unusual situation (as it is returning all the records from the parent category table - most times the search will return far less records hence far less categories), so I think this will slow things down in more normal situations, and it still isn't using indexes on some of the joins. It would be good if MySQL could decide which way to join the tables depending on the result sets.
SELECT category_parent.id,
category_parent.depth,
category_parent.name AS item_sub_category,
category_parent.left_index,
category_parent.right_index,
COUNT(DISTINCT item.id) as total
FROM category category_parent
STRAIGHT_JOIN category ON category.left_index BETWEEN category_parent.left_index AND category_parent.right_index
STRAIGHT_JOIN item ON item.mmg_code = category.mmg_code
STRAIGHT_JOIN search_enquiries_found ON search_enquiries_found.item_id = item.id
WHERE search_enquiries_found.search_enquiry_id = 35
GROUP BY category_parent.id,
category_parent.depth,
item_sub_category,
category_parent.left_index,
category_parent.right_index
ORDER BY category_parent.left_index
id select_type table partitions type possible_keys key key_len ref rows Extra
1 SIMPLE category_parent NULL ALL left_index NULL NULL NULL 337 Using temporary; Using filesort
1 SIMPLE category NULL ALL left_index NULL NULL NULL 337 Range checked for each record (index map: 0x2)
1 SIMPLE item NULL ref PRIMARY,id,mmg_code mmg_code 27 em_entaonline.category.mmg_code 43 Using index
1 SIMPLE search_enquiries_found NULL eq_ref search_enquiry_id,item_id,search_enquiry_id_2,search_enquiry_id_relevance search_enquiry_id 8 const,em_entaonline.item.id 1 Using index