How do I optimize this mysql query with nested joins? - mysql

Question

Below is my query which I need to optimize.
SELECT
UPPER(IFNULL(op.supplier_payment_method,s.default_payment_method)) AS Payment_Method,
op.supplier_payment_date AS Payment_Date,
Date_format(IFNULL(op.supplier_payment_date,op.ship_date),'%M %Y') AS Payment_Month,
s.supplier_name AS Farm,
op.sub_order_id AS Order_num,
Date_format(op.ship_date,'%b-%d-%Y') AS Ship_Date,
op.farm_credit AS Farm_Credit,
op.credit_memo AS Credit_Memo,
op.credit_description AS Credit_Description,
opb.boxes AS Box_Type,
CONCAT('$',FORMAT(op.box_charge,2)) AS Box_Charge,
op.invoice_num AS Invoice_num,
CONCAT('$',FORMAT(op.invoice_amt,2)) AS Invoice_Amt,
CONCAT('$',FORMAT(op.total_invoice_amt,2)) AS Total_Invoice_Amt,
CONCAT(op.UM_qty,' ',op.UM_type) AS St_Bu_Qty,
op.PO_Product_Name AS Invoice_desc,
CONCAT('$',FORMAT((op.price_um*op.um_qty),2)) AS Cost_product_cms,
op.supplier_invoice_note AS Supplier_Invoice_Notes,
CONCAT('$',FORMAT(op.cms_invoice_cost,2)) AS CMS_Invoice_diff,
CONCAT('$',FORMAT(op.total_farm_cost,2)) AS Farm_Cost
FROM
orders_products op
INNER JOIN
suppliers s ON s.supplier_id = op.supplier_name
LEFT JOIN
(
SELECT
sub_order_id,
GROUP_CONCAT(CONCAT(box_type_qty,' ',bo.box_option_name) SEPARATOR ', ') AS boxes
FROM
order_products_boxes opb
INNER JOIN box_options bo ON bo.id=opb.box_type_id
GROUP BY
opb.sub_order_id
) opb ON opb.sub_order_id = op.sub_order_id
WHERE
op.order_active=0
AND op.ship_date>='2013-03-01'
AND op.ship_date<='2013-04_01'
ORDER BY op.ship_date DESC
As you can see, the query comprises of 4 tables, each containing around 20k-30k rows each. So as soon as I add in the sub-query, the query becomes exceptionally slow. It takes approx 1.5mins to fetch just 500 rows of record. Is there a way to speed things up within a single query?

Answer 1

Your left-join query that pre-concatenates the sub-box options is querying the ENTIRE DATABASE FIRST, then only joins to those that are within the criteria you are limiting for the outer WHERE clause. It may be a little more effort, but pull the same outer criteria into your inner query should significantly help. As for doing a field-level sql-select on a per-every-row basis could be a performance killer.
You should only need to change at the "FROM" clause...
from
orders_products op
INNER JOIN suppliers s
ON op.supplier_name = s.supplier_id
LEFT JOIN
( SELECT
opb2.sub_order_id,
GROUP_CONCAT(CONCAT(box_type_qty,' ',bo.box_option_name) SEPARATOR ', ') AS boxes
FROM
orders_products op2
JOIN order_products_boxes opb2
on op2.sub_order_id = opb2.sub_order_id
INNER JOIN box_options bo
ON opb2.box_type_id = bo.id
WHERE
op2.order_active = 0
AND op2.ship_date >= '2013-03-01'
AND op2.ship_date <= '2013-04_01'
GROUP BY
opb2.sub_order_id ) opb
ON op.sub_order_id = opb.sub_order_id
WHERE
op.order_active = 0
AND op.ship_date >= '2013-03-01'
AND op.ship_date <= '2013-04_01'

Answer 2

SELECT
UPPER(IFNULL(op.supplier_payment_method,s.default_payment_method)) AS Payment_Method,
op.supplier_payment_date AS Payment_Date,
Date_format(IFNULL(op.supplier_payment_date,op.ship_date),'%M %Y') AS Payment_Month,
s.supplier_name AS Farm,
op.sub_order_id AS Order_num,
Date_format(op.ship_date,'%b-%d-%Y') AS Ship_Date,
op.farm_credit AS Farm_Credit,
op.credit_memo AS Credit_Memo,
op.credit_description AS Credit_Description,
(
SELECT
GROUP_CONCAT(CONCAT(box_type_qty,' ',bo.box_option_name) SEPARATOR ', ')
FROM
order_products_boxes opb
INNER JOIN box_options bo ON bo.id=opb.box_type_id
GROUP BY
opb.sub_order_id
) AS Box_Type,
CONCAT('$',FORMAT(op.box_charge,2)) AS Box_Charge,
op.invoice_num AS Invoice_num,
CONCAT('$',FORMAT(op.invoice_amt,2)) AS Invoice_Amt,
CONCAT('$',FORMAT(op.total_invoice_amt,2)) AS Total_Invoice_Amt,
CONCAT(op.UM_qty,' ',op.UM_type) AS St_Bu_Qty,
op.PO_Product_Name AS Invoice_desc,
CONCAT('$',FORMAT((op.price_um*op.um_qty),2)) AS Cost_product_cms,
op.supplier_invoice_note AS Supplier_Invoice_Notes,
CONCAT('$',FORMAT(op.cms_invoice_cost,2)) AS CMS_Invoice_diff,
CONCAT('$',FORMAT(op.total_farm_cost,2)) AS Farm_Cost
FROM
orders_products op
INNER JOIN
suppliers s ON s.supplier_id = op.supplier_name
LEFT JOIN
order_products_boxes opb (
INNER JOIN box_options bo ON bo.id=opb.box_type_id
) opb ON opb.sub_order_id = op.sub_order_id
WHERE
op.order_active=0
AND op.ship_date>='2013-03-01'
AND op.ship_date<='2013-04_01'
ORDER BY op.ship_date DESC