MySQL Query Optimization for complex query - mysql

Question

I am working with an existing site and I came across the following MySQL query that needs optimization:
select
mo.mmrrc_order_oid,
mo.completed_by_email,
mo.completed_by_name,
mo.completed_by_title,
mo.order_submission_oid,
mo.order_dt,
mo.center_id,
mo.po_num_tx,
mo.mod_dt,
ste_s.state_cd,
group_concat(distinct osr.status_cd order by osr.status_cd) as test,
case group_concat(distinct osr.status_cd order by osr.status_cd)
when 'Fulfilled' then 'Fulfilled'
when 'Fulfilled,N/A' then 'Fulfilled'
when 'N/A' then 'N/A'
when 'Pending' then 'Pending'
else 'In Process'
end as restriction_status,
max(osr.closed_dt) as restriction_update_dt,
ot.milestone,
ot.completed_dt as tracking_update_dt,
dc.first_name,
dc.last_name,
inst.institution_name,
order_search.products as products_ordered,
mo.other_emails,
mo.customer_label,
mo.grant_numbers
from
t_mmrrc_order mo
join ste_state ste_s using(state_id)
left join t_order_contact oc
on oc.mmrrc_order_oid=mo.mmrrc_order_oid and oc.role_cd='Recipient'
left join t_distrib_cont_instn dci using(distrib_cont_instn_oid)
left join t_institution inst using(institution_oid)
left join t_distribution_contact dc using(distribution_contact_oid)
left join t_order_tracking ot
on ot.mmrrc_order_oid=mo.mmrrc_order_oid
and ifnull(ot.order_tracking_oid, '0000-00-00')= ifnull(
(
select max(order_tracking_oid)
from t_order_tracking ot3
where
ot3.mmrrc_order_oid=mo.mmrrc_order_oid
and ot3.completed_dt= (
select max(completed_dt)
from t_order_tracking ot2
where ot2.mmrrc_order_oid=mo.mmrrc_order_oid
)
), '0000-00-00')
left join t_order_strain_restriction osr
on osr.mmrrc_order_oid = mo.mmrrc_order_oid
left join order_search on order_search.mmrrc_order_oid=mo.mmrrc_order_oid
group by
mo.mmrrc_order_oid
LIMIT 0, 5
this query takes 10+ seconds to run regardless of the limit. When run without a limit, there are a total of 5,727 results and runtime is 10.624 seconds.
With "LIMIT 0, 5" it took 18.47 seconds.
I understand that there are a bunch of joins and nested selects, which is why it is so slow. Any ideas on how to optimize this without having to change the database structure?
MySQL version: 5.0.95
Most tables have over 10,000 records.
This simpler query takes about 9 seconds:
select
mo.mmrrc_order_oid,
mo.completed_by_email,
mo.completed_by_name,
mo.completed_by_title,
mo.order_submission_oid,
mo.order_dt,
mo.center_id,
mo.po_num_tx,
mo.mod_dt,
dc.first_name,
dc.last_name,
inst.institution_name,
order_search.products as products_ordered,
mo.other_emails,
mo.customer_label,
mo.grant_numbers
from
t_mmrrc_order mo
join ste_state ste_s using(state_id)
left join t_order_contact oc
on oc.mmrrc_order_oid=mo.mmrrc_order_oid and oc.role_cd='Recipient'
left join t_distrib_cont_instn dci using(distrib_cont_instn_oid)
left join t_institution inst using(institution_oid)
left join t_distribution_contact dc using(distribution_contact_oid)
left join t_order_strain_restriction osr
on osr.mmrrc_order_oid = mo.mmrrc_order_oid
left join order_search on order_search.mmrrc_order_oid=mo.mmrrc_order_oid
group by mo.mmrrc_order_oid
limit 0,5
I suppose the grouping slows it down the most. In this case, without grouping takes only 0.17 seconds. Any help would be appreciated. Thanks.
Additional details - here is what EXPLAIN gives me for the first query:
View Image
I found that order_search is a view that is causing most of the slow down. The query for the view is:
SELECT
t_oi.mmrrc_order_oid AS mmrrc_order_oid,
group_concat(t_im.icc_item_code separator ',') AS products
FROM
t_order_item t_oi
JOIN t_item_master t_im on t_oi.item_master_oid = t_im.item_master_oid
JOIN t_strain_archive on t_im.strain_archive_oid = t_strain_archive.strain_archive_oid
WHERE t_oi.item_status_cd IN (_utf8'Active',_utf8'Modified')
GROUP BY t_oi.mmrrc_order_oid
ORDER BY t_im.icc_item_code

Answer 1

Just assuming you haven't index the coloumns so i create some indexes for your coloumns this would help you and there are still much coloumns to index like in your join conditions you should apply this operation on that coloumns also for better execution
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamemmrrc_order_oid` (`mmrrc_order_oid`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamecompleted_by_email` (`completed_by_email`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamecompleted_by_name` (`completed_by_name`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamecompleted_by_title` (`completed_by_title`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnameorder_submission_oid` (`order_submission_oid`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnameorder_dt` (`order_dt`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamecenter_id` (`center_id`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamepo_num_tx` (`po_num_tx`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamemod_dt` (`mod_dt`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnameother_emails` (`other_emails`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamecustomer_label` (`customer_label`);
ALTER TABLE `t_mmrrc_order` ADD INDEX `Indexnamegrant_numbers` (`grant_numbers`);
ALTER TABLE `t_distribution_contact ` ADD INDEX `Indexnamefirst_name` (`first_name`);
ALTER TABLE `t_distribution_contact ` ADD INDEX `Indexnamelast_name` (`last_name`);
ALTER TABLE `order_search` ADD INDEX `Indexnameproducts` (`products`);

Answer 2

I managed to solve this problem by doing two separate queries from my PHP script.
First, I query the order_search view by itself and save all the data in a PHP array indexed by the mmrrc_order_oid, which then serves as a quick lookup table for products. This resulting lookup table is an array of about about 6000 strings.
Next, I perform the big complex query with order_search table omitted. This only takes about a second now. For each resulting record, I simply use the lookup table by mmrrc_order_oid to get the products for that order.