I have the following SQL query
SELECT
DISTINCT
count("SiteTree_Live"."ID")
FROM
"SiteTree_Live"
LEFT JOIN "Page_Live" ON "Page_Live"."ID" = "SiteTree_Live"."ID"
LEFT JOIN "TourPage_Live" ON "TourPage_Live"."ID" = "SiteTree_Live"."ID"
LEFT JOIN "DepartureDate" ON "DepartureDate"."TourID" = "SiteTree_Live"."ID"
WHERE
("SiteTree_Live"."Locale" = 'en_AU')
AND ("SiteTree_Live"."ClassName" IN ('TourPage'))
AND ("DepartureDate"."DepartureDate" LIKE '2012-11%')
but it producing a wrong count as the query result. The total intented result this query is suppose to return should not be more than 245 but currently, its returning more than about "4569" results.
Thats is because of the JOIN on the "DepartureDate" table as the query returns the expected result when i remove the join from the "DepartureDate" table.
What modification do i need to make to my query to count the Macthes between "SiteTree_Live"."ID" and "DepartureDate"."TourID" whiles counting only the "SiteTree_Live"."ID" count excluding the Departure dates?
Any suggestions welcomed :)
THE ANSWER
SELECT
COUNT(DISTINCT SiteTree_Live.ID)
FROM
"SiteTree_Live" LEFT JOIN "Page_Live" ON "Page_Live"."ID" = "SiteTree_Live"."ID"
LEFT JOIN "TourPage_Live" ON "TourPage_Live"."ID" = "SiteTree_Live"."ID"
LEFT JOIN "DepartureDate" ON "DepartureDate"."TourID" = "SiteTree_Live"."ID"
WHERE
("SiteTree_Live"."Locale" = 'en_AU')
AND ("SiteTree_Live"."ClassName" IN ('TourPage'))
AND ("DepartureDate"."DepartureDate" LIKE '2013-03%')
Seems to give me the right result. Thanks for the tip #Michael Berkowski
Minor correction: if DepartureDate is a date-type, then the LIKE '2013-03% will force it to be coerced into a character type (this is a mysql feature) As a result, any indexes on DepartureDate will not be used, IIRC. Better use a plain range-query:
SELECT
COUNT(DISTINCT stl.ID)
FROM
SiteTree_Live stl
LEFT JOIN
DepartureDate dd ON dd.TourID = stl.ID
WHERE
stl.Locale = 'en_AU'
AND stl.ClassName = 'TourPage'
AND dd.DepartureDate >= '2013-03-01'
AND dd.DepartureDate < '2013-04-01'
;
Do this (You have a bunch of unneeded joins)
SELECT
COUNT(DISTINCT SiteTree_Live.ID)
FROM
`SiteTree_Live`
LEFT JOIN
`DepartureDate` ON `DepartureDate`.`TourID` = `SiteTree_Live`.`ID`
WHERE
`SiteTree_Live`.`Locale` = 'en_AU'
AND `SiteTree_Live`.`ClassName` = 'TourPage'
AND `DepartureDate`.`DepartureDate` LIKE '2013-03%'
You could also do a GROUP BY:
SELECT
COUNT(SiteTree_Live.ID)
FROM
`SiteTree_Live`
LEFT JOIN
`DepartureDate` ON `DepartureDate`.`TourID` = `SiteTree_Live`.`ID`
WHERE
`SiteTree_Live`.`Locale` = 'en_AU'
AND `SiteTree_Live`.`ClassName` = 'TourPage'
AND `DepartureDate`.`DepartureDate` LIKE '2013-03%'
GROUP BY
SiteTree_Live.ID
Related
I am having issues with getting this double left join to get the listingspecificsListPrice, but that info exists in the table, cant figure out why it would not include it. This is my sql.
SELECT mls_subject_property.*, mls_images.imagePath, mls_forms_listing_specifics.listingspecificsListPrice
FROM mls_subject_property
LEFT JOIN mls_images ON mls_subject_property.mls_listingID = mls_images.mls_listingID
LEFT JOIN mls_forms_listing_specifics ON mls_forms_listing_specifics.mls_listingID = mls_subject_property.mls_listingID AND mls_images.imgOrder = 0
WHERE userID = 413
GROUP BY mls_subject_property.mls_listingID
The result comes out like this..
All of the other fields come back, but it doesnt seem to want to bring back those two items.
This is a picture of the other table, to show that the data does in fact exist.
The mls_images.imgOrder = 0 condition should be in the join with mls_images, not mls_forms_listing_specifics.
Don't use GROUP BY if you're not using any aggregation functions. Use SELECT DISTINCT to prevent duplicates.
SELECT DISTINCT mls_subject_property.*, mls_images.imagePath, mls_forms_listing_specifics.listingspecificsListPrice
FROM mls_subject_property
LEFT JOIN mls_images ON mls_subject_property.mls_listingID = mls_images.mls_listingID AND mls_images.imgOrder = 0
LEFT JOIN mls_forms_listing_specifics ON mls_forms_listing_specifics.mls_listingID = mls_subject_property.mls_listingID
WHERE userID = 413
i added indexes as well but still it is taking 13 sec
I added compound index for all the columns that i've used here
SELECT carrierbil2_.IDENTITY AS col_0_0_,
carrier4_.CARRIER_NAME AS col_1_0_,
carrier4_.IDENTITY AS col_2_0_,
carrier4_.CARRIER_ID AS col_3_0_,
shipmentor0_.EXTERNAL_REFERENCE_ID AS col_4_0_,
invoicedet5_.INVOICE_NUMBER AS col_5_0_,`enter code here`
shipmentca1_.CARRIER_REFERENCE_NUMBER AS col_6_0_,
SUM(shipmentco9_.RATED_COST) AS col_7_0_,
SUM(shipmentco9_.COST) AS col_8_0_,
invoice6_.TOTAL_PAID_AMOUNT AS col_9_0_,
invoice6_.INVOICE_GENERATED_DATE AS col_10_0_,
shipmentor0_.ACTUAL_SHIP_DATE AS col_11_0_,
bolstatus15_.BOL_STATUS_ID AS col_12_0_,
shipmentlo10_.LOCATION_NAME AS col_13_0_,
country11_.COUNTRY_NAME AS col_14_0_,
postal14_.POSTAL_CODE AS col_15_0_,
state12_.STATE_NAME AS col_16_0_,
city13_.CITY_NAME AS col_17_0_,
shipmentlo16_.LOCATION_NAME AS col_18_0_,
country17_.COUNTRY_NAME AS col_19_0_,
postal20_.POSTAL_CODE AS col_20_0_,
state18_.STATE_NAME AS col_21_0_,
city19_.CITY_NAME AS col_22_0_,
shipmentor0_.IDENTITY AS col_23_0_,
shipmentca1_.IDENTITY AS col_24_0_,
shipmentno7_.NOTE AS col_25_0_
FROM
SHIPMENT_ORDER shipmentor0_
INNER JOIN
SHIPMENT_CARRIER shipmentca1_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentca1_.SHIPMENT_ORDER_ID
AND (shipmentca1_.IS_DELETED = 0)
LEFT OUTER JOIN
CARRIER_BILL_DETAILS carrierbil2_ ON shipmentca1_.SHIPMENT_CARRIER_ID = carrierbil2_.SHIPMENT_CARRIER_ID
LEFT OUTER JOIN
CARRIER_BILLS carrierbil3_ ON carrierbil2_.CARRIER_BILL_ID = carrierbil3_.CARRIER_BILL_ID
INNER JOIN
CARRIER carrier4_ ON shipmentca1_.CARRIER_ID = carrier4_.CARRIER_ID
LEFT OUTER JOIN
INVOICE_DETAILS invoicedet5_ ON shipmentor0_.SHIPMENT_ORDER_ID = invoicedet5_.SHIPMENT_ORDER_ID
LEFT OUTER JOIN
INVOICE invoice6_ ON invoicedet5_.INVOICE_ID = invoice6_.INVOICE_ID
LEFT OUTER JOIN
SHIPMENT_NOTES shipmentno7_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentno7_.SHIPMENT_ORDER_ID
AND (shipmentno7_.NOTE_TYPE = 4)
LEFT OUTER JOIN
SHIPMENT_COST shipmentco8_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentco8_.SHIPMENT_ID
LEFT OUTER JOIN
SHIPMENT_COST_DETAILS shipmentco9_ ON shipmentco8_.SHIPMENT_COST_ID = shipmentco9_.SHIPMENT_COST_ID
AND (shipmentco9_.IS_DELETED = 0)
LEFT OUTER JOIN
SHIPMENT_LOCATION shipmentlo10_ ON shipmentor0_.ORIGIN_ID = shipmentlo10_.SHIPMENT_LOCATION_ID
AND (shipmentlo10_.LOCATION_TYPE_ID = 3)
LEFT OUTER JOIN
COUNTRY country11_ ON shipmentlo10_.COUNTRY_ID = country11_.COUNTRY_ID
LEFT OUTER JOIN
STATE state12_ ON shipmentlo10_.STATE_ID = state12_.STATE_ID
LEFT OUTER JOIN
CITY city13_ ON shipmentlo10_.CITY_ID = city13_.CITY_ID
LEFT OUTER JOIN
POSTAL postal14_ ON shipmentlo10_.POSTAL_ID = postal14_.POSTAL_ID
LEFT OUTER JOIN
BOL_STATUS bolstatus15_ ON shipmentor0_.ORDER_STATUS = bolstatus15_.BOL_STATUS_ID
LEFT OUTER JOIN
SHIPMENT_LOCATION shipmentlo16_ ON shipmentor0_.DESTINATION_LOCATION_ID = shipmentlo16_.SHIPMENT_LOCATION_ID
AND (shipmentlo16_.LOCATION_TYPE_ID = 4)
LEFT OUTER JOIN
COUNTRY country17_ ON shipmentlo16_.COUNTRY_ID = country17_.COUNTRY_ID
LEFT OUTER JOIN
STATE state18_ ON shipmentlo16_.STATE_ID = state18_.STATE_ID
LEFT OUTER JOIN
CITY city19_ ON shipmentlo16_.CITY_ID = city19_.CITY_ID
LEFT OUTER JOIN
POSTAL postal20_ ON shipmentlo16_.POSTAL_ID = postal20_.POSTAL_ID
CROSS JOIN
CLIENT client21_
WHERE
shipmentor0_.CLIENT_ID = client21_.CLIENT_ID
AND bolstatus15_.SEQUENCE_ID >= 700
AND (carrierbil3_.IS_APPROVED = 0
OR carrierbil3_.IS_APPROVED IS NULL)
AND (carrierbil3_.IS_DELETED = 0
OR carrierbil3_.IS_DELETED IS NULL)
AND (carrierbil2_.IS_DELETED = 0
OR carrierbil2_.IS_DELETED IS NULL)
AND (shipmentor0_.IS_DELETED = 0
OR shipmentor0_.IS_DELETED IS NULL)
GROUP BY invoice6_.INVOICE_GENERATED_DATE , shipmentca1_.IDENTITY , invoicedet5_.INVOICE_NUMBER , invoice6_.TOTAL_PAID_AMOUNT , shipmentca1_.CARRIER_REFERENCE_NUMBER , carrier4_.CARRIER_ID , CAST(carrier4_.IDENTITY AS SIGNED) , carrier4_.CARRIER_NAME , CAST(carrierbil2_.IDENTITY AS SIGNED) , shipmentor0_.SHIPMENT_ORDER_ID , shipmentno7_.NOTE , shipmentor0_.EXTERNAL_REFERENCE_ID , shipmentlo10_.LOCATION_NAME , country11_.COUNTRY_NAME , postal14_.POSTAL_CODE , state12_.STATE_NAME , city13_.CITY_NAME , shipmentlo16_.LOCATION_NAME , country17_.COUNTRY_NAME , postal20_.POSTAL_CODE , state18_.STATE_NAME , city19_.CITY_NAME , shipmentor0_.IDENTITY
ORDER BY shipmentor0_.SHIPMENT_ORDER_ID DESC;
The indexes are mostly useless because of OR, as in
AND (carrierbil3_.IS_APPROVED = 0
OR carrierbil3_.IS_APPROVED IS NULL)
The simple way to fix that is to pick either 0 or NULL to represent the flag. Then make sure all the data is consistent, and change the WHERE to just check for the one case.
Do you really mean
CROSS JOIN
CLIENT client21_
That is likely to be a performance-killer and generate a huge resultset.
Never mind. You have the ON in WHERE. Please use ON for relations and WHERE for filtering.
WHERE
shipmentor0_.CLIENT_ID = client21_.CLIENT_ID
I see a mixture of LEFT JOIN and JOIN. Check that the LEFT JOINs really need to be LEFT; that is, the 'right' table might have missing data.
To discuss further, please provide EXPLAIN SELECT ....
Eschew over-normalization:
You have 5 tables to describe a location (name, country, postal, state, city). Instead, I recommend a single table with those 5 columns. This, alone, would get rid of 8 JOINs.
CAST(carrier4_.IDENTITY AS SIGNED) -- Can't you fix the datatype to be SIGNED, or allow the value to be UNSIGNED?
But perhaps the main performance-killer is the "explode-implode" syndrone. First, it does a lot of JOINs, building a huge intermediate table, then it collapses that by doing GROUP BY. The remedy is
SELECT ...
FROM ( SELECT SUM(...), SUM(...) FROM ... GROUP BY ... ) AS a
JOIN ((whatever else is needed));
That is, first devise a minimal "derived table" that does the GROUP BY (and/or ORDER BY and/or LIMIT). Then see what else is needed to complete the query (namely all the normalization lookups).
After you have acted on most of my comments, we can discuss whether you have the optimal indexes. (It is premature to do so now.) If so, please start a new Question; it would be too much clutter to add to this one.
First of all, that's a lot of joins. However, the main reason your query is taking a significant time is because you're adding an order by clause. You need to figure out a way to avoid it, or may be come up with a different strategy
This is my query.
SELECT dr_trans_dtl.dr_ID, jo_trans_dtl.qty
FROM dr_trans_dtl
LEFT JOIN jo_trans_dtl ON dr_trans_dtl.jo_no = jo_trans_dtl.jo_no
WHERE dr_trans_dtl.dr_no = '3329' GROUP BY dr_trans_dtl.dr_ID
Here is the actual result:
What I wanted is the qty should be like this (500,40,1). Because that is the data in jo_trans_dtl.
It seems you need aggregate function as you used group by
SELECT dr_trans_dtl.dr_ID,sum(jo_trans_dtl.qty) as qty
FROM dr_trans_dtl
JOIN jo_trans_dtl ON dr_trans_dtl.jo_no = jo_trans_dtl.jo_no
WHERE dr_trans_dtl.dr_no = '3329'
GROUP BY dr_trans_dtl.dr_ID
I have the following query
SELECT custconcompany, custconfirstname, custconlastname, custconemail, custconphone, shipaddress1, shipaddress2, shipcity, stateabbrv, shipzip, countryname, websitecheck.formfieldfieldvalue websitevalue, excludecheck.formfieldfieldvalue excludevalue
FROM obcisc_customers
JOIN ( (obcisc_shipping_addresses JOIN obcisc_countries
ON obcisc_shipping_addresses.shipcountryid = obcisc_countries.countryid)
LEFT JOIN obcisc_country_states
ON obcisc_shipping_addresses.shipstateid = obcisc_country_states.stateid
LEFT JOIN obcisc_formfieldsessions websitecheck
ON obcisc_shipping_addresses.shipformsessionid = websitecheck.formfieldsessioniformsessionid
LEFT JOIN obcisc_formfieldsessions excludecheck
ON obcisc_shipping_addresses.shipformsessionid = excludecheck.formfieldsessioniformsessionid)
ON obcisc_customers.customerid = obcisc_shipping_addresses.shipcustomerid
WHERE custgroupid = 11
AND websitecheck.formfieldfieldid = 24
AND excludecheck.formfieldfieldid = 30
AND excludecheck.formfieldfieldvalue != 'a:1:{i:0;s:3:"Yes";}'
ORDER BY shipstate, shipcity
This works great except I also need it to return rows where "excludecheck.formfieldfieldid=30" does not exist... right now it's not returning them
When writing a LEFT JOIN, any criteria on the table you're joining with should be put into the ON clause. If you put it into the WHERE clause, you'll filter out the results in the first table that don't have a matching row in the second table, because the NULL value that comes from the outer join will not match the criteria.
SELECT custconcompany, custconfirstname, custconlastname, custconemail, custconphone, shipaddress1, shipaddress2, shipcity, stateabbrv, shipzip, countryname, websitecheck.formfieldfieldvalue websitevalue, excludecheck.formfieldfieldvalue excludevalue
FROM obcisc_customers
JOIN obcisc_shipping_addresses
ON obcisc_customers.customerid = obcisc_shipping_addresses.shipcustomerid
JOIN obcisc_countries
ON obcisc_shipping_addresses.shipcountryid = obcisc_countries.countryid)
LEFT JOIN obcisc_country_states
ON obcisc_shipping_addresses.shipstateid = obcisc_country_states.stateid
LEFT JOIN obcisc_formfieldsessions websitecheck
ON obcisc_shipping_addresses.shipformsessionid = websitecheck.formfieldsessioniformsessionid
AND websitecheck.formfieldfieldid = 24
LEFT JOIN obcisc_formfieldsessions excludecheck
ON obcisc_shipping_addresses.shipformsessionid = excludecheck.formfieldsessioniformsessionid
AND excludecheck.formfieldfieldid = 30
AND excludecheck.formfieldfieldvalue != 'a:1:{i:0;s:3:"Yes";}')
WHERE custgroupid = 11
ORDER BY shipstate, shipcity
Another way to do it is by putting (excludecheck.formfieldfieldid = 30 OR excludecheck.formfieldfieldid IS NULL) in the WHERE clause. But this is more verbose and also I believe it's harder for MySQL to optimize, especially if you have several tables you're joining like this.
I currently have this working using a Sub-query, but as the DB grows this will become HUGELY inefficient. I'm wondering if there is a more efficient way to do what I need to do without sub-queries?
I need to have my final output look like so:
Question, Answer, Responses, Charts included in Response Count
Did this work?, N/A, 26, 30
Did this work?, Yes, 4, 30
This is my current query:
SELECT
bq_text,
ba_a,
bq_id,
COUNT(ba_a) AS ba_aC,
(SELECT COUNT(*) FROM board_done_sheet WHERE sd_b_id = bs.bs_id AND sd_sub = 1) AS sd_chartnumC
FROM board_done_sheet AS sh
LEFT JOIN board_done bd
ON (bd.bd_id = sh.sd_bd_id)
LEFT JOIN boardsubs bs
ON (bd.bd_b_id = bs.bs_id)
LEFT JOIN b_q_answers ba
ON (sh.sd_s_id = ba.ba_s_id)
LEFT JOIN bsquestions bq
ON (bq.bq_id = ba.ba_q_id)
LEFT JOIN multiples m
ON (ba.ba_m_id = m.m_id)
LEFT JOIN users u
ON (u.us_id = bd.bd_d_id)
LEFT JOIN profiles p
ON (p.p_u_id = bd.bd_d_id)
LEFT JOIN users rev
ON (rev.us_id = bd.bd_rev)
WHERE sd_sub = '1' AND bq_text <> 'Date' AND bq_id = 380
GROUP BY bs_id, bq_text, ba_a
That works perfectly, the problem is it has to use sub-queries which as time goes by will get less efficient.
I'm just wondering if there is a better more efficient way to do that summed field without it.
Presumably the subquery you're concerned about is the one in your toplevel SELECT.
That is easy to refactor so it won't get repeated.
Just JOIN it to the rest of the table. You'll want this sort of thing:
SELECT
bq_text, ...
COUNT(ba_a) AS ba_aC,
countup.countup AS sd_chartnumC
FROM board_done_sheet AS sh
LEFT JOIN board_done bd
ON (bd.bd_id = sh.sd_bd_id)
...
LEFT JOIN users rev
ON (rev.us_id = bd.bd_rev)
JOIN (
SELECT COUNT(*) AS countup , sd_b_id
FROM board_done_sheet
WHERE sd_sub = 1
GROUP BY sd_b_id
) AS countup ON countup.sd_b_id = bs.bs_id
WHERE sd_sub = '1'
AND bq_text <> 'Date'
AND bq_id = 380
GROUP BY bs_id, bq_text, ba_a
The countup subquery generates a summary table of counts and ids, and then joins it to the other tables.
A JOIN cascade of this complexity may become inefficient for other reasons as your table grows if you don't structure your indexes correctly.