this Query is taking 15 seconds - mysql

Question

i added indexes as well but still it is taking 13 sec
I added compound index for all the columns that i've used here
SELECT carrierbil2_.IDENTITY AS col_0_0_,
carrier4_.CARRIER_NAME AS col_1_0_,
carrier4_.IDENTITY AS col_2_0_,
carrier4_.CARRIER_ID AS col_3_0_,
shipmentor0_.EXTERNAL_REFERENCE_ID AS col_4_0_,
invoicedet5_.INVOICE_NUMBER AS col_5_0_,`enter code here`
shipmentca1_.CARRIER_REFERENCE_NUMBER AS col_6_0_,
SUM(shipmentco9_.RATED_COST) AS col_7_0_,
SUM(shipmentco9_.COST) AS col_8_0_,
invoice6_.TOTAL_PAID_AMOUNT AS col_9_0_,
invoice6_.INVOICE_GENERATED_DATE AS col_10_0_,
shipmentor0_.ACTUAL_SHIP_DATE AS col_11_0_,
bolstatus15_.BOL_STATUS_ID AS col_12_0_,
shipmentlo10_.LOCATION_NAME AS col_13_0_,
country11_.COUNTRY_NAME AS col_14_0_,
postal14_.POSTAL_CODE AS col_15_0_,
state12_.STATE_NAME AS col_16_0_,
city13_.CITY_NAME AS col_17_0_,
shipmentlo16_.LOCATION_NAME AS col_18_0_,
country17_.COUNTRY_NAME AS col_19_0_,
postal20_.POSTAL_CODE AS col_20_0_,
state18_.STATE_NAME AS col_21_0_,
city19_.CITY_NAME AS col_22_0_,
shipmentor0_.IDENTITY AS col_23_0_,
shipmentca1_.IDENTITY AS col_24_0_,
shipmentno7_.NOTE AS col_25_0_
FROM
SHIPMENT_ORDER shipmentor0_
INNER JOIN
SHIPMENT_CARRIER shipmentca1_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentca1_.SHIPMENT_ORDER_ID
AND (shipmentca1_.IS_DELETED = 0)
LEFT OUTER JOIN
CARRIER_BILL_DETAILS carrierbil2_ ON shipmentca1_.SHIPMENT_CARRIER_ID = carrierbil2_.SHIPMENT_CARRIER_ID
LEFT OUTER JOIN
CARRIER_BILLS carrierbil3_ ON carrierbil2_.CARRIER_BILL_ID = carrierbil3_.CARRIER_BILL_ID
INNER JOIN
CARRIER carrier4_ ON shipmentca1_.CARRIER_ID = carrier4_.CARRIER_ID
LEFT OUTER JOIN
INVOICE_DETAILS invoicedet5_ ON shipmentor0_.SHIPMENT_ORDER_ID = invoicedet5_.SHIPMENT_ORDER_ID
LEFT OUTER JOIN
INVOICE invoice6_ ON invoicedet5_.INVOICE_ID = invoice6_.INVOICE_ID
LEFT OUTER JOIN
SHIPMENT_NOTES shipmentno7_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentno7_.SHIPMENT_ORDER_ID
AND (shipmentno7_.NOTE_TYPE = 4)
LEFT OUTER JOIN
SHIPMENT_COST shipmentco8_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentco8_.SHIPMENT_ID
LEFT OUTER JOIN
SHIPMENT_COST_DETAILS shipmentco9_ ON shipmentco8_.SHIPMENT_COST_ID = shipmentco9_.SHIPMENT_COST_ID
AND (shipmentco9_.IS_DELETED = 0)
LEFT OUTER JOIN
SHIPMENT_LOCATION shipmentlo10_ ON shipmentor0_.ORIGIN_ID = shipmentlo10_.SHIPMENT_LOCATION_ID
AND (shipmentlo10_.LOCATION_TYPE_ID = 3)
LEFT OUTER JOIN
COUNTRY country11_ ON shipmentlo10_.COUNTRY_ID = country11_.COUNTRY_ID
LEFT OUTER JOIN
STATE state12_ ON shipmentlo10_.STATE_ID = state12_.STATE_ID
LEFT OUTER JOIN
CITY city13_ ON shipmentlo10_.CITY_ID = city13_.CITY_ID
LEFT OUTER JOIN
POSTAL postal14_ ON shipmentlo10_.POSTAL_ID = postal14_.POSTAL_ID
LEFT OUTER JOIN
BOL_STATUS bolstatus15_ ON shipmentor0_.ORDER_STATUS = bolstatus15_.BOL_STATUS_ID
LEFT OUTER JOIN
SHIPMENT_LOCATION shipmentlo16_ ON shipmentor0_.DESTINATION_LOCATION_ID = shipmentlo16_.SHIPMENT_LOCATION_ID
AND (shipmentlo16_.LOCATION_TYPE_ID = 4)
LEFT OUTER JOIN
COUNTRY country17_ ON shipmentlo16_.COUNTRY_ID = country17_.COUNTRY_ID
LEFT OUTER JOIN
STATE state18_ ON shipmentlo16_.STATE_ID = state18_.STATE_ID
LEFT OUTER JOIN
CITY city19_ ON shipmentlo16_.CITY_ID = city19_.CITY_ID
LEFT OUTER JOIN
POSTAL postal20_ ON shipmentlo16_.POSTAL_ID = postal20_.POSTAL_ID
CROSS JOIN
CLIENT client21_
WHERE
shipmentor0_.CLIENT_ID = client21_.CLIENT_ID
AND bolstatus15_.SEQUENCE_ID >= 700
AND (carrierbil3_.IS_APPROVED = 0
OR carrierbil3_.IS_APPROVED IS NULL)
AND (carrierbil3_.IS_DELETED = 0
OR carrierbil3_.IS_DELETED IS NULL)
AND (carrierbil2_.IS_DELETED = 0
OR carrierbil2_.IS_DELETED IS NULL)
AND (shipmentor0_.IS_DELETED = 0
OR shipmentor0_.IS_DELETED IS NULL)
GROUP BY invoice6_.INVOICE_GENERATED_DATE , shipmentca1_.IDENTITY , invoicedet5_.INVOICE_NUMBER , invoice6_.TOTAL_PAID_AMOUNT , shipmentca1_.CARRIER_REFERENCE_NUMBER , carrier4_.CARRIER_ID , CAST(carrier4_.IDENTITY AS SIGNED) , carrier4_.CARRIER_NAME , CAST(carrierbil2_.IDENTITY AS SIGNED) , shipmentor0_.SHIPMENT_ORDER_ID , shipmentno7_.NOTE , shipmentor0_.EXTERNAL_REFERENCE_ID , shipmentlo10_.LOCATION_NAME , country11_.COUNTRY_NAME , postal14_.POSTAL_CODE , state12_.STATE_NAME , city13_.CITY_NAME , shipmentlo16_.LOCATION_NAME , country17_.COUNTRY_NAME , postal20_.POSTAL_CODE , state18_.STATE_NAME , city19_.CITY_NAME , shipmentor0_.IDENTITY
ORDER BY shipmentor0_.SHIPMENT_ORDER_ID DESC;

Answer 1

The indexes are mostly useless because of OR, as in
AND (carrierbil3_.IS_APPROVED = 0
OR carrierbil3_.IS_APPROVED IS NULL)
The simple way to fix that is to pick either 0 or NULL to represent the flag. Then make sure all the data is consistent, and change the WHERE to just check for the one case.
Do you really mean
CROSS JOIN
CLIENT client21_
That is likely to be a performance-killer and generate a huge resultset.
Never mind. You have the ON in WHERE. Please use ON for relations and WHERE for filtering.
WHERE
shipmentor0_.CLIENT_ID = client21_.CLIENT_ID
I see a mixture of LEFT JOIN and JOIN. Check that the LEFT JOINs really need to be LEFT; that is, the 'right' table might have missing data.
To discuss further, please provide EXPLAIN SELECT ....
Eschew over-normalization:
You have 5 tables to describe a location (name, country, postal, state, city). Instead, I recommend a single table with those 5 columns. This, alone, would get rid of 8 JOINs.
CAST(carrier4_.IDENTITY AS SIGNED) -- Can't you fix the datatype to be SIGNED, or allow the value to be UNSIGNED?
But perhaps the main performance-killer is the "explode-implode" syndrone. First, it does a lot of JOINs, building a huge intermediate table, then it collapses that by doing GROUP BY. The remedy is
SELECT ...
FROM ( SELECT SUM(...), SUM(...) FROM ... GROUP BY ... ) AS a
JOIN ((whatever else is needed));
That is, first devise a minimal "derived table" that does the GROUP BY (and/or ORDER BY and/or LIMIT). Then see what else is needed to complete the query (namely all the normalization lookups).
After you have acted on most of my comments, we can discuss whether you have the optimal indexes. (It is premature to do so now.) If so, please start a new Question; it would be too much clutter to add to this one.

Answer 2

First of all, that's a lot of joins. However, the main reason your query is taking a significant time is because you're adding an order by clause. You need to figure out a way to avoid it, or may be come up with a different strategy