what index to use for query - mysql

Question

controller
#from = params[:from] ? 4.hours.since(Time.zone.parse(params[:from])) : today_start_time
#to = params[:to] ? 4.hours.since(Time.zone.parse(params[:to])) : #from
#franchise = current_user_accessible_franchises_filtered
#orders_type = Restaurant::DELIVERY_TYPES.detect { |t| t == params[:orders_type]} || Restaurant::DELIVERY_TYPES.first
#limit = (params[:limit] || 250).to_i
#to = 48.hours.since(#to) if #to == #from && #from == today_start_time
start_time = #from.utc.strftime('%Y-%m-%d %H:%M')
end_time = 24.hours.since(#to).utc.strftime('%Y-%m-%d %H:%M')
#orders = Order
.includes(:address, :franchise, :customer, :ordered_items, :rests, :driver)
.where(
"((orders.created_at >= ? AND orders.created_at <= ?) OR (orders.delivery_target >= ? AND orders.delivery_target <= ?)) AND orders.franchise_id in (?) AND orders.delivery_type = ?",
start_time, end_time, start_time, end_time, #franchise, #orders_type
)
.order("orders.status desc, orders.id desc")
log
(23732.8ms) SELECT COUNT(DISTINCT orders.id) FROM orders LEFT
OUTER JOIN addresses ON addresses.id = orders.address_id
LEFT OUTER JOIN franchises ON franchises.id =
orders.franchise_id LEFT OUTER JOIN customers ON
customers.id = orders.customer_id LEFT OUTER JOIN
ordered_items ON ordered_items.order_id = orders.id LEFT
OUTER JOIN ordered_items ordered_items_orders_join ON
ordered_items_orders_join.order_id = orders.id LEFT OUTER JOIN
restaurants ON restaurants.id =
ordered_items_orders_join.restaurant_id LEFT OUTER JOIN drivers
ON drivers.id = orders.driver_id WHERE (((orders.created_at >=
'2015-03-01 10:00' AND orders.created_at <= '2015-03-04 10:00') OR
(orders.delivery_target >= '2015-03-01 10:00' AND
orders.delivery_target <= '2015-03-04 10:00')) AND orders.franchise_id
in
(3,31,4,22,37,2,36,17,34,30,19,20,21,18,27,13,25,16,35,24,32,33,1,12,28,23,14,26,29,8,11)
AND orders.delivery_type = 'Mr. Delivery Restaurant')
EXPLAIN EXTENDED
mysql> EXPLAIN EXTENDED
SELECT DISTINCT `orders`.id
FROM `orders`
LEFT OUTER JOIN `addresses` ON `addresses`.`id` = `orders`.`address_id`
LEFT OUTER JOIN `franchises` ON `franchises`.`id` = `orders`.`franchise_id`
LEFT OUTER JOIN `customers` ON `customers`.`id` = `orders`.`customer_id`
LEFT OUTER JOIN `ordered_items` ON `ordered_items`.`order_id` = `orders`.`id`
LEFT OUTER JOIN `ordered_items` `ordered_items_orders_join`
ON `ordered_items_orders_join`.`order_id` = `orders`.`id`
LEFT OUTER JOIN `restaurants` ON `restaurants`.`id` = `ordered_items_orders_join`.`restaurant_id`
LEFT OUTER JOIN `drivers` ON `drivers`.`id` = `orders`.`driver_id`
WHERE ( ( (orders.created_at >= '2015-02-01 10:00'
AND orders.created_at <= '2015-02-04 10:00')
OR (orders.delivery_target >= '2015-02-01 10:00'
AND orders.delivery_target <= '2015-02-04 10:00') )
AND orders.franchise_id in (3,31,4,22,37,2,36,17,34,30,
19,20,21,18,27,13,25,16,35,24,32,33,1,12,28,23,14,26,29, 8,11 )
AND orders.delivery_type = 'Mr. Delivery Restaurant'
)
ORDER BY orders.status desc,
orders.id desc
LIMIT 250;
+----+-------------+---------------------------+--------+------------------------------------------------------------------------------------------------------------------------------+---------------------------------+---------+---------------------------------------------------------+--------+----------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------------------------+--------+------------------------------------------------------------------------------------------------------------------------------+---------------------------------+---------+---------------------------------------------------------+--------+----------+----------------------------------------------+
| 1 | SIMPLE | orders | ref | index_orders_on_created_at,index_orders_on_delivery_target,index_orders_on_franchise_id,index_orders_on_delivery_type,my_idx | index_orders_on_delivery_type | 77 | const | 549769 | 100.00 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | addresses | eq_ref | PRIMARY | PRIMARY | 4 | mrd_staging_new.orders.address_id | 1 | 100.00 | Using index; Distinct |
| 1 | SIMPLE | franchises | eq_ref | PRIMARY | PRIMARY | 4 | mrd_staging_new.orders.franchise_id | 1 | 100.00 | Using index; Distinct |
| 1 | SIMPLE | customers | eq_ref | PRIMARY | PRIMARY | 4 | mrd_staging_new.orders.customer_id | 1 | 100.00 | Using index; Distinct |
| 1 | SIMPLE | ordered_items | ref | index_ordered_items_on_order_id | index_ordered_items_on_order_id | 5 | mrd_staging_new.orders.id | 2 | 100.00 | Using index; Distinct |
| 1 | SIMPLE | ordered_items_orders_join | ref | index_ordered_items_on_order_id | index_ordered_items_on_order_id | 5 | mrd_staging_new.orders.id | 2 | 100.00 | Distinct |
| 1 | SIMPLE | restaurants | eq_ref | PRIMARY | PRIMARY | 4 | mrd_staging_new.ordered_items_orders_join.restaurant_id | 1 | 100.00 | Using index; Distinct |
| 1 | SIMPLE | drivers | eq_ref | PRIMARY | PRIMARY | 4 | mrd_staging_new.orders.driver_id | 1 | 100.00 | Using index; Distinct |
+----+-------------+---------------------------+--------+------------------------------------------------------------------------------------------------------------------------------+---------------------------------+---------+---------------------------------------------------------+--------+----------+----------------------------------------------+
8 rows in set, 1 warning (0.00 sec)
indices on Orders table
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+---------------------------------------+--------------+-----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| orders | 0 | PRIMARY | 1 | id | A | 935943 | NULL | NULL | | BTREE | | |
| orders | 1 | index_orders_on_delivered_at | 1 | delivered_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_updated_at | 1 | updated_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_created_at | 1 | created_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_pickup_at | 1 | pickup_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_coupon_id | 1 | coupon_id | A | 14856 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_driver_id | 1 | driver_id | A | 4159 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_delivery_target | 1 | delivery_target | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_franchise_id | 1 | franchise_id | A | 48 | NULL | NULL | YES | BTREE | | |
| orders | 1 | idx_for_customers_report | 1 | customer_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | idx_for_customers_report | 2 | delivered_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | idx_for_customers_report | 3 | created_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | idx_for_customers_report | 4 | status | A | 935943 | NULL | NULL | | BTREE | | |
| orders | 1 | idx_for_customers_report | 5 | franchise_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 1 | id | A | 935943 | NULL | NULL | | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 2 | created_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 3 | delivered_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 4 | status | A | 935943 | NULL | NULL | | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 5 | franchise_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 6 | customer_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | orders_fetch_index_for_revenue_report | 7 | address_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_address_id | 1 | address_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_customer_id | 1 | customer_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | index_orders_on_delivery_type | 1 | delivery_type | A | 4 | NULL | NULL | | BTREE | | |
| orders | 1 | index_orders_on_status | 1 | status | A | 8 | NULL | NULL | | BTREE | | |
| orders | 1 | index_orders_on_should_pay_restaurant | 1 | should_pay_restaurant | A | 4 | NULL | NULL | YES | BTREE | | |
| orders | 1 | my_idx | 1 | created_at | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | my_idx | 2 | delivery_target | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | my_idx | 3 | franchise_id | A | 935943 | NULL | NULL | YES | BTREE | | |
| orders | 1 | my_idx | 4 | delivery_type | A | 935943 | NULL | NULL | | BTREE | | |
| orders | 1 | my_idx | 5 | status | A | 935943 | NULL | NULL | | BTREE | | |
| orders | 1 | my_idx | 6 | id | A | 935943 | NULL | NULL | | BTREE | | |
total quantity
mysql> select count(*) from orders;
+----------+
| count(*) |
+----------+
| 965520 |
+----------+
1 row in set (0.32 sec)
I have tried to add a composite index
create index my_idx on orders(created_at, delivery_target, franchise_id, delivery_type);
even
create index my_idx on orders(created_at, delivery_target, franchise_id, delivery_type, status, id);
but without success.
Any ideas?

Answer 1

INDEX(delivery_type, -- because it is "= constant"
franchise_id) -- the only other viable column
Indexing any column hiding in an OR is virtually futile.
I do question your ORDER BY status while doing DISTINCT id and no mention of status.
You may be able to speed it up by turning the OR into a UNION:
( SELECT orders.status, orders.id
FROM ...
JOIN ...
WHERE orders.created_at ...
AND ...
ORDER BY ...
LIMIT 250
)
UNION DISTINCT
( SELECT orders.status, orders.id
FROM ...
JOIN ...
WHERE orders.delivery_target ... -- note difference
AND ...
ORDER BY ...
LIMIT 250
)
ORDER BY status DESC, id DESC -- yes, again
LIMIT 250; -- yes, again
But for this reformulation to work well, there needs to be new indexes. I am not sure whether these
INDEX(delivery_type, franchise_id, created_at),
INDEX(delivery_type, franchise_id, delivery_target),
or these
INDEX(delivery_type, created_at),
INDEX(delivery_type, delivery_target),
would work better. Add all 4; do EXPLAIN to see which ones it uses.