Mysql Query speed takes 50 seconds - mysql

I have the below Query which takes around 50 seconds to run, is there a way to make it faster? i have not done any indexing yet and will try to do that in the near future but i am still new in this. the problem is not using indexing.
Select stock.StockID As StockID,
stock.BinLocation As BinLocation,
stock.StockCode As StockCode,
stock.Descr As Descr,
stock.Currency As Currency,
stock.Notes As Notes,
stocklocations.Quantity As Quantity,
stocklocations.SLNID As SLNID,
stock.ConditionA As ConditionA,
stocklocationsname.SLN As SLN,
stock.DateAdded As DateAdded,
(Case When (stock.Currency = 'Euro') Then (1.05 * stock.unitprice)
When (stock.Currency = 'GBP') Then (1.24 * stock.unitprice)
When (stock.Currency = 'USD') Then (1 * stock.unitprice) End) As unitprice,
stock.DepName As DepName,
stockhistory.POID As POID,
stock.POID As POID1,
stock.PartNumber As PartNumber,
vender.VendorName As VendorName,
equiptype.EQName As EQName,
groupname.Name As Name
From ((((((stock
Join stocklocations On stock.StockID = stocklocations.StockID)
Join stocklocationsname On stocklocations.SLNID = stocklocationsname.SLNID)
Left Join stockhistory On stockhistory.SHID = (Select Min(stockhistory.SHID)
From stockhistory Where stockhistory.StockID = stock.StockID))
Left Join po On stockhistory.POID = po.POID)
Left Join vender On po.Vender = vender.VendorID)
Left Join equiptype On stock.EquipType = equiptype.EquipTypeID)
Left Join groupname On stock.GroupNameID = groupname.GroupNameID
Where stocklocations.Quantity > 0 And stock.Status <> 'obsolete'

You should probably have an index on stockhistory.StockID:
ALTER TABLE stockhistory ADD INDEX (StockID);
That should speed up the subqueries on stockhistory. Alternatively, since you're retrieving the minimum value of SHID, you might include that column in the index as well:
ALTER TABLE stockhistory ADD INDEX (StockID, SHID);
Adding indexes does not create any restrictions or logical constraints (with the exception of unique indexes). However, as stated in 8.3 Optimization and Indexes from the MySQL documentation:
Although it can be tempting to create an indexes for every possible column used in a query, unnecessary indexes waste space and waste time for MySQL to determine which indexes to use. Indexes also add to the cost of inserts, updates, and deletes because each index must be updated. You must find the right balance to achieve fast queries using the optimal set of indexes.
Figuring out which indexes to create is something of an art. A good start is to investigate query execution with EXPLAIN, as described in 8.8.1 Optimizing Queries with EXPLAIN.

1 ) Remove unuseful () and avoid nested left join
2 ) don't use where fo column involved in left join because this way became inner join
3 ) instead of a subselect try using a join of for the min value related to stockid in stockhistory
Select
stock.StockID As StockID,
stock.BinLocation As BinLocation,
stock.StockCode As StockCode,
stock.Descr As Descr,
stock.Currency As Currency,
stock.Notes As Notes,
stocklocations.Quantity As Quantity,
stocklocations.SLNID As SLNID,
stock.ConditionA As ConditionA,
stocklocationsname.SLN As SLN,
stock.DateAdded As DateAdded,
Case When stock.Currency = 'Euro' Then 1.05 * stock.unitprice
When stock.Currency = 'GBP' Then 1.24 * stock.unitprice
When stock.Currency = 'USD' Then 1 * stock.unitprice End As unitprice,
stock.DepName As DepName,
stockhistory.POID As POID,
stock.POID As POID1,
stock.PartNumber As PartNumber,
vender.VendorName As VendorName,
equiptype.EQName As EQName,
groupname.Name As Name
From stock
Join stocklocations On stock.StockID = stocklocations.StockID
and stocklocations.Quantity > 0
And stock.Status <> 'obsolete'
Join stocklocationsname On stocklocations.SLNID = stocklocationsname.SLNID
Left Join (
Select stockhistory.StockID, Min(stockhistory.SHID)
From stockhistory
group by stockhistory.StockID ) t_h on t_h StockID = stockhistory.StockID
Left Join po On stockhistory.POID = po.POID
Left Join vender On po.Vender = vender.VendorID
Left Join equiptype On stock.EquipType = equiptype.EquipTypeID
Left Join groupname On stock.GroupNameID = groupname.GroupNameID
be sure you have also proper index on the tables and columns involved in join

Try without brackets which you write along joins and FROM.
Select stock.StockID As StockID,
stock.BinLocation As BinLocation,
stock.StockCode As StockCode,
stock.Descr As Descr,
stock.Currency As Currency,
stock.Notes As Notes,
stocklocations.Quantity As Quantity,
stocklocations.SLNID As SLNID,
stock.ConditionA As ConditionA,
stocklocationsname.SLN As SLN,
stock.DateAdded As DateAdded,
(Case When (stock.Currency = 'Euro') Then (1.05 * stock.unitprice)
When (stock.Currency = 'GBP') Then (1.24 * stock.unitprice)
When (stock.Currency = 'USD') Then (1 * stock.unitprice) End) As unitprice,
stock.DepName As DepName,
stockhistory.POID As POID,
stock.POID As POID1,
stock.PartNumber As PartNumber,
vender.VendorName As VendorName,
equiptype.EQName As EQName,
groupname.Name As Name
From stock
Join stocklocations On stock.StockID = stocklocations.StockID
Join stocklocationsname On stocklocations.SLNID = stocklocationsname.SLNID
Left Join stockhistory On stockhistory.SHID = (Select Min(stockhistory.SHID) From stockhistory Where stockhistory.StockID = stock.StockID)
Left Join po On stockhistory.POID = po.POID
Left Join vender On po.Vender = vender.VendorID
Left Join equiptype On stock.EquipType = equiptype.EquipTypeID
Left Join groupname On stock.GroupNameID = groupname.GroupNameID
Where stocklocations.Quantity > 0 And stock.Status <> 'obsolete'

how much time it takes to execute without this?
Left Join stockhistory On stockhistory.SHID = (Select Min(stockhistory.SHID)
From stockhistory Where stockhistory.StockID = stock.StockID))
If it takes 5-6 secs, then you should convert this min SHID group by StockID to a view, and join with the view first, then stockhistory itself to get the stockhistory.POID

Related

How to increase performance of a query against data vault architecture when joins with same select queries are exactly the same but the date?

We need to get the minimum and maximum dates over common location, project and form.
the pseudo-code of such query looks like:
SELECT A.FORM,
A.LOCATION,
A.PROJECT,
MN.START_DATE,
MX.END_DATE
FROM
(
) A
JOIN (
SELECT ...
FROM
-- JOINS ON N TABLES
) MN ON MN.FORM = A.FORM AND MN.LOCATION = A.LOCATION AND MN.PROJECT = A.PROJECT
JOIN (
-- SAME JOINS ON SAME N TABLES
) MX ON MX.FORM = A.FORM AND MX.LOCATION = A.LOCATION AND MX.PROJECT = A.PROJECT
query of MN and MX are exactly the same but the date field whereas MIN(DATE) is coming from MN, and MAX(DATE) is coming from MX join.
is it possible to combine both MN into MX in a way to reduce execution time and increase performance as this is running over thousands of rows of data.
here is the exact query:
SELECT
MX.ACTIVITY_CODE,
MN.FORM_ID_STRING,
MX.PROJECT_CODE,
DATE(MN.MIN_START_DATE) AS MIN_START_DATE,
DATE(MX.MAX_END_DATE) AS MAX_END_DATE
FROM
(
-- Getting the min start date of submissions related to same location, project and activity
SELECT HO.FORM_ID_STRING, HO.LOCATION_NAME, HPC.PROJECT_CODE, HA.ACTIVITY_CODE,
MIN(SO.OBSERVATION_START_DT) MIN_START_DATE
FROM DATA_VAULT.HUB_OBSERVATION HO
JOIN DATA_VAULT.SAT_OBSERVATION SO ON HO.OBSERVATION_HKEY = SO.OBSERVATION_HKEY
JOIN DATA_VAULT.SAT_OBSERVATION_REVIEW SOR ON SOR.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.LNK_OBSERVATION_PROJECT_CODE LOPC ON LOPC.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.HUB_PROJECT_CODE HPC ON HPC.PROJECT_CODE_HKEY = LOPC.PROJECT_CODE_HKEY
JOIN DATA_VAULT.LNK_OBSERVATION_COUNTRY_ACTIVITY LOCA ON LOCA.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.HUB_ACTIVITY HA ON HA.ACTIVITY_HKEY = LOCA.ACTIVITY_HKEY
WHERE
SO.LOAD_DT = (SELECT MAX(LOAD_DT) FROM DATA_VAULT.SAT_OBSERVATION WHERE OBSERVATION_HKEY = HO.OBSERVATION_HKEY)
AND SOR.LOAD_DT = (SELECT MAX(LOAD_DT) FROM DATA_VAULT.SAT_OBSERVATION_REVIEW WHERE OBSERVATION_HKEY = HO.OBSERVATION_HKEY)
AND SOR.REVIEW_STATUS NOT IN ('REJECTED', 'DELETED')
GROUP BY HO.FORM_ID_STRING, HO.LOCATION_NAME, HPC.PROJECT_CODE, HA.ACTIVITY_CODE
) MN
JOIN
(
-- Getting the max end date of submissions related to same location, project and activity
SELECT HO.FORM_ID_STRING as form, HO.LOCATION_NAME as loc, HPC.PROJECT_CODE, HA.ACTIVITY_CODE,
MAX(SO.OBSERVATION_START_DT) as MAX_END_DATE
FROM DATA_VAULT.HUB_OBSERVATION HO
JOIN DATA_VAULT.SAT_OBSERVATION SO ON HO.OBSERVATION_HKEY = SO.OBSERVATION_HKEY
JOIN DATA_VAULT.SAT_OBSERVATION_REVIEW SOR ON SOR.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.LNK_OBSERVATION_PROJECT_CODE LOPC ON LOPC.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.HUB_PROJECT_CODE HPC ON HPC.PROJECT_CODE_HKEY = LOPC.PROJECT_CODE_HKEY
JOIN DATA_VAULT.LNK_OBSERVATION_COUNTRY_ACTIVITY LOCA ON LOCA.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.HUB_ACTIVITY HA ON HA.ACTIVITY_HKEY = LOCA.ACTIVITY_HKEY
WHERE
// HO.FORM_ID_STRING = 'Global_rh_skilled_facility_delivery_v1_0_1_data.csv'
SO.LOAD_DT = (SELECT MAX(LOAD_DT) FROM DATA_VAULT.SAT_OBSERVATION WHERE OBSERVATION_HKEY = HO.OBSERVATION_HKEY)
AND SOR.LOAD_DT = (SELECT MAX(LOAD_DT) FROM DATA_VAULT.SAT_OBSERVATION_REVIEW WHERE OBSERVATION_HKEY = HO.OBSERVATION_HKEY)
AND SOR.REVIEW_STATUS NOT IN ('REJECTED', 'DELETED')
GROUP BY HO.FORM_ID_STRING, HO.LOCATION_NAME, HPC.PROJECT_CODE, HA.ACTIVITY_CODE
) MX
ON MX.form = MN.form_id_string AND MX.loc = MN.location_name AND MN.PROJECT_CODE = MX.PROJECT_CODE
#HoneyBadger is correct. You can fetch both the MIN and MAX start dates in the same subquery, so you don't need to repeat it. That should save some time, IO, and CPU.
SELECT ...,
MIN(SO.OBSERVATION_START_DT) MIN_START_DATE,
MAX(SO.OBSERVATION_START_DT) MAX_END_DATE
And, you have a couple of correlated subqueries in the WHERE clause of your subquery. You can change both of those to a single JOINed subquery and save some more time.
This is what your single big subquery might look like with those two changes. I cannot debug this because I don't have your data and can only guess at your spec.
SELECT HO.FORM_ID_STRING, HO.LOCATION_NAME, HPC.PROJECT_CODE, HA.ACTIVITY_CODE,
MIN(SO.OBSERVATION_START_DT) MIN_START_DATE, /* min and max in */
MAX(SO.OBSERVATION_START_DT) MAX_END_DATE /* just one query */
FROM DATA_VAULT.HUB_OBSERVATION HO
JOIN DATA_VAULT.SAT_OBSERVATION SO ON HO.OBSERVATION_HKEY = SO.OBSERVATION_HKEY
JOIN DATA_VAULT.SAT_OBSERVATION_REVIEW SOR ON SOR.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN ( /* non-correlated subquery to find max LOAD_DT values */
SELECT OBSERVATION_HKEY, MAX(LOAD_DT) LOAD_DT
FROM DATA_VAULT.SAT_OBSERVATION
GROUP BY OBSERVATION_HKEY
) SOMAX ON SO.LOAD_DT = SOMAX.LOAD_DT
AND SO.OBSERVATION_HKEY = SOMAX.OBSERVATION_HKEY
AND SOR.LOAD_DT = SOMAX.LOAD_DT
AND SOR.OBSERVATION_HKEY = SOMAX.OBSERVATION_HKEY
JOIN DATA_VAULT.LNK_OBSERVATION_PROJECT_CODE LOPC ON LOPC.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.HUB_PROJECT_CODE HPC ON HPC.PROJECT_CODE_HKEY = LOPC.PROJECT_CODE_HKEY
JOIN DATA_VAULT.LNK_OBSERVATION_COUNTRY_ACTIVITY LOCA ON LOCA.OBSERVATION_HKEY = HO.OBSERVATION_HKEY
JOIN DATA_VAULT.HUB_ACTIVITY HA ON HA.ACTIVITY_HKEY = LOCA.ACTIVITY_HKEY
WHERE SOR.REVIEW_STATUS NOT IN ('REJECTED', 'DELETED')
GROUP BY HO.FORM_ID_STRING, HO.LOCATION_NAME, HPC.PROJECT_CODE, HA.ACTIVITY_CODE
The next step, if this doesn't give you enough performance, is to look at the execution plan (the EXPLAIN output) and try to work out whether some more indexes or different indexes will help. Searching for MIN and MAX values can be astonishingly fast with the right indexes.
That is a good subject for another question.
Get the max and min without all those joins, if possible.
Get the max and min in the same query, if possible.
Get MAX(LOAD_DT) once, not twice.
Then JOIN to all the other tables.
Some of these indexes may be useful:
HO: INDEX(FORM_ID_STRING, OBSERVATION_HKEY, LOCATION_NAME)
HPC: INDEX(PROJECT_CODE_HKEY, PROJECT_CODE)
HA: INDEX(ACTIVITY_HKEY, ACTIVITY_CODE)
SO: INDEX(LOAD_DT, OBSERVATION_HKEY, OBSERVATION_START_DT)
SOR: INDEX(LOAD_DT, REVIEW_STATUS, OBSERVATION_HKEY)
LOPC: INDEX(OBSERVATION_HKEY, PROJECT_CODE_HKEY)
LOCA: INDEX(OBSERVATION_HKEY, ACTIVITY_HKEY)
HO: INDEX(OBSERVATION_HKEY)
For further discussion, please provide SHOW CREATE TABLE for all the tables; it sounds like they might be over-normalized.

MySQL Find Most Recent/Largest Record Per Group by order by

MySQL Find Most Recent/Largest Record Per Group by order by, and how can i minimize / shorten this query and every time it returns the first row value of the group, whereas i like to select the last row value of a group, and sort the values based on ja.id ? I know this is a bad query can anyone suggest or provide me solution to shorten this query . I have used all the necessary Column Indexes in all the tables. How to shorten the query without the use of union all .both the queries in union all are same expect in the where statement.
SELECT
a.previous_status,
a.rejected_status,
a.rejection_reason_text,
a.rejection_reason,
a.rjaId,
a.refer_applied_status,
a.title,
a.playerId,
a.gameId,
a.gamePostDate,
a.game_referal_amount,
a.country,
a.country_name,
a.state,
a.location,
a.state_abb,
a.game_type,
a.appliedId,
a.appliedStatus,
a.admin_review,
a.is_req_referal_check,
a.memberId,
a.appliedEmail,
a.game_id,
a.referred_id,
a.memStateAbb,
a.memState,
a.memZipcode,
a.memCity,
a.memCountryNme,
a.memCountry,
a.appliedMemberName,
a.first_name,
a.primary_contact,
a.last_name,
a.addressbookImage,
a.userImage,
a.last_login,
a.user_experience_year,
a.user_experience_month,
a.current_designation,
a.current_player,
a.appliedDate,
a.addressbook_id,
a.joiningdate,
a.gameStatus,
a.gameReferalAmountType,
a.gameFreezeStatus,
a.gameFreezeMsg,
a.app_assign_back_to_rp_count,
a.applied_source,
a.max_id,
a.gamesApplied,
a.gamesAppliedId,
SUM(a.totalgameApplied) AS totalgameApplied,
a.application_assign_to_rp_status,
a.rpAppliedSource,
a.applied_on
FROM
(
(
SELECT
ja.previous_status,
ja.rejected_status,
ja.rejection_reason_text,
ja.rejection_reason,
rja.id AS rjaId,
rja. STATUS AS refer_applied_status,
jp.title,
jp.user_user_id AS playerId,
jp.id AS gameId,
jp.posted_on AS gamePostDate,
jp.game_referal_amount,
jp.country,
jp.country_name,
jp.state,
jp.location,
jp.state_abb,
jp.game_type,
ja.id AS appliedId,
IFNULL(ja. STATUS, '') AS appliedStatus,
IFNULL(ja.admin_review, '') AS admin_review,
ja.is_req_referal_check,
usr.id AS memberId,
rja.email AS appliedEmail,
rja.game_id,
rja.referred_id,
mem.state_abb AS memStateAbb,
mem.state AS memState,
mem.zipcode AS memZipcode,
mem.city AS memCity,
mem.country_name AS memCountryNme,
mem.country_code AS memCountry,
usc. NAME AS appliedMemberName,
usc.first_name,
IFNULL(
mem.primary_contact,
usc.phone_number
) AS primary_contact,
usc.last_name,
usc.profileimage_path AS addressbookImage,
usr.profile_image AS userImage,
usr.last_login,
mem.user_experience_year,
mem.user_experience_month,
mem.current_designation,
mem.current_player,
rja.create_date AS appliedDate,
rja.addressbook_id,
IFNULL(ja.joining_date, '') AS joiningdate,
jp. STATUS AS gameStatus,
jp.games_referal_amount_type AS gameReferalAmountType,
jp.game_freeze_status AS gameFreezeStatus,
jp.game_freeze_message AS gameFreezeMsg,
ja.app_assign_back_to_rp_count,
ja.applied_source,
MAX(rja.id) AS max_id,
GROUP_CONCAT(
jp.title
ORDER BY
rja.create_date DESC
) AS gamesApplied,
GROUP_CONCAT(DISTINCT(jp.id)) AS gamesAppliedId,
COUNT(DISTINCT(jp.id)) totalgameApplied,
ja.application_assign_to_rp_status,
1 AS rpAppliedSource,
rja.create_date AS applied_on
FROM
(`refer_gameapplied` AS rja)
JOIN `games_post` AS jp ON `jp`.`id` = `rja`.`game_id`
JOIN `user_socialconnections` AS usc ON `rja`.`addressbook_id` = `usc`.`id`
LEFT JOIN `user_user` AS usr ON `usr`.`email` = `rja`.`email`
LEFT JOIN `user_member` AS mem ON `mem`.`user_id` = `usr`.`id`
LEFT JOIN `game_applied` AS ja ON `ja`.`id` = `rja`.`applied_id`
WHERE
`rja`.`referby_id` = '2389'
GROUP BY
`rja`.`email`
)
UNION ALL
(
SELECT
ja.previous_status,
ja.rejected_status,
ja.rejection_reason_text,
ja.rejection_reason,
jr.id AS rjaId,
jrtm. STATUS AS refer_applied_status,
jp.title,
jp.user_user_id AS playerId,
jp.id AS gameId,
jp.posted_on AS gamePostDate,
jp.game_referal_amount,
jp.country,
jp.country_name,
jp.state,
jp.location,
jp.state_abb,
jp.game_type,
ja.id AS appliedId,
IFNULL(ja. STATUS, '') AS appliedStatus,
IFNULL(ja.admin_review, '') AS admin_review,
ja.is_req_referal_check,
usr.id AS memberId,
jrtm.referto_email AS refappliedEmail,
jr.game_id,
jrtm.id,
mem.state_abb AS memStateAbb,
mem.state AS memState,
mem.zipcode AS memZipcode,
mem.city AS memCity,
mem.country_name AS memCountryNme,
mem.country_code AS memCountry,
usc. NAME AS appliedMemberName,
usc.first_name,
IFNULL(
mem.primary_contact,
usc.phone_number
) AS primary_contact,
usc.last_name,
usc.profileimage_path AS addressbookImage,
usr.profile_image AS userImage,
usr.last_login,
mem.user_experience_year,
mem.user_experience_month,
mem.current_designation,
mem.current_player,
jrtm.refer_on AS appliedDate,
jrtm.referto_addressbookid,
IFNULL(ja.joining_date, '') AS joiningdate,
jp. STATUS AS gameStatus,
jp.games_referal_amount_type AS gameReferalAmountType,
jp.game_freeze_status AS gameFreezeStatus,
jp.game_freeze_message AS gameFreezeMsg,
ja.app_assign_back_to_rp_count,
ja.applied_source,
MAX(jrtm.id) AS max_id,
GROUP_CONCAT(
jp.title
ORDER BY
jr.refer_on DESC
) AS gamesApplied,
GROUP_CONCAT(DISTINCT(jp.id)) AS gamesAppliedId,
COUNT(DISTINCT(jp.id)) totalgameApplied,
ja.application_assign_to_rp_status,
2 AS rpAppliedSource,
jrtm.refer_on AS applied_on
FROM
(`game_refer` AS jr)
JOIN `game_refer_to_member` AS jrtm ON `jrtm`.`rid` = `jr`.`id`
JOIN `games_post` AS jp ON `jp`.`id` = `jr`.`game_id`
JOIN `user_socialconnections` AS usc ON `jrtm`.`referto_addressbookid` = `usc`.`id`
LEFT JOIN `user_user` AS usr ON `usr`.`email` = `jrtm`.`referto_email`
LEFT JOIN `user_member` AS mem ON `mem`.`user_id` = `usr`.`id`
LEFT JOIN `game_applied` AS ja ON `ja`.`referred_by` = `jrtm`.`id`
WHERE
`jrtm`.`status` = '1'
AND `jr`.`referby_user_id` = '2389'
AND `jrtm`.`refer_source` NOT IN ('4')
GROUP BY
`jrtm`.`referto_email`
)
) a
GROUP BY
a.appliedEmail
ORDER BY
a.gamesAppliedId DESC
It sounds like a "groupwise-max" problem. I added a tag that you should research.
At least get rid of the columns that are not relevant to the question.
Try tossing the tables other than jrtm and jr to see if the performance problem persists. (I'm thinking that the LEFT JOINs may be red herrings.)
Try with one part of the UNION, then with the other. This may identify which of the two is more of a burden.
Some indexes to add:
rja: (referby_id, create_date)
jrtm: (status, referto_email)
jrtm: (rid, status, referto_email)
jr: (referby_user_id, refer_on)
DISTINCT is not a function. Don't use parents in DISTINCT(jp.id).

Create One Result set from 3 different Queries

I am trying to create a result set using three queries. I have three tables an inventory table, a form order table, and a form order detail table. I need to be able to input a date range and get how many forms were ordered, how many are in current inventory, and how many were destroyed based on if they contain a destruction date. Ultimately i want a result set that shows:
InventoryId, FormDescription, Product, Ordered, Shipped, Destroyed, Total ending
What would be the best way to get that result set using these queries?
These are my three Queries
SELECT FOD.InventoryId, SUM(FOD.FormOrderAmount) as totalOrdered, FOD.FormShippedAmount FROM tblFormOrder FMO
JOIN tblFormOrderDetails FOD ON FOD.FormOrderId = FMO.FormOrderId
WHERE FMO.OrderDateTime BETWEEN '20110101' and '20120101'
AND FMO.OrderStatus IN ('S')
GROUP BY FOD.InventoryId, FOD.FormShippedAmount -- total shipped by date and inventoryid
SELECT INV.InventoryId, SUM(INV.CurrentAmount) as currentAmount, SUM(INV.OrderAmount) as OrderAmount,
(SUM(INV.OrderAmount) - SUM(INV.CurrentAmount)) as InventoryUsed
FROM tblInventory INV
where INV.CreatedOn
BETWEEN '20110101' and '20120101'
GROUP BY INV.InventoryId -- current amount based off ordered and used
select INV.InventoryId, count(*) as total
, FMO.OrderDateTime as OrderDate, Inv.FormNo, INV.FormDescription, INV.Product
from [tblinventory] INV
join tblformorderdetails FOD ON FOD.InventoryId = inv.InventoryId
join tblformorder FMO on FMO.FormOrderId = FOD.FormOrderId
where INV.DestructionDate
BETWEEN '20110101' and '20120101'
group by
FMO.OrderDateTime,
Inv.FormNo, INV.FormDescription, INV.Product, INV.InventoryId -- using count to find how many destroyed if they have a destruction date
If you want an inner join you can do this
SELECT *
FROM (...) AS Q1, (...) AS Q2, (...) AS Q3
WHERE Q1.InventoryID = Q2.InventoryID AND Q2.InventoryID = Q3.InventoryID
This will only give you the InventoryIDs that are in all 3 queries.
You probably don't want this you probably want all inventory IDs... so you do this.
SELECT *
FROM (SELECT DISTINCT InventoryID FROM tblFormOrderDetails
UNION ALL
SELECT DISTINCT InventoryID FROM tblInventory) I
LEFT JOIN (...) Q1 ON I.InventoryID = Q1.InventoryId
LEFT JOIN (...) Q2 ON I.InventoryID = Q2.InventoryId
LEFT JOIN (...) Q3 ON I.InventoryID = Q3.InventoryId
Make each query a subquery that you join together.
SELECT *
FROM (
SELECT FOD.InventoryId, SUM(FOD.FormOrderAmount) as totalOrdered, FOD.FormShippedAmount FROM tblFormOrder FMO
JOIN tblFormOrderDetails FOD ON FOD.FormOrderId = FMO.FormOrderId
WHERE FMO.OrderDateTime BETWEEN '20110101' and '20120101'
AND FMO.OrderStatus IN ('S')
GROUP BY FOD.InventoryId, FOD.FormShippedAmount -- total shipped by date and inventoryid
) AS q1
LEFT JOIN (
SELECT INV.InventoryId, SUM(INV.CurrentAmount) as currentAmount, SUM(INV.OrderAmount) as OrderAmount,
(SUM(INV.OrderAmount) - SUM(INV.CurrentAmount)) as InventoryUsed
FROM tblInventory INV
where INV.CreatedOn
BETWEEN '20110101' and '20120101'
GROUP BY INV.InventoryId -- current amount based off ordered and used
) AS q2 ON q1.InventoryId = q2.InventoryId
LEFT JOIN (
select INV.InventoryId, count(*) as total
, FMO.OrderDateTime as OrderDate, Inv.FormNo, INV.FormDescription, INV.Product
from [tblinventory] INV
join tblformorderdetails FOD ON FOD.InventoryId = inv.InventoryId
join tblformorder FMO on FMO.FormOrderId = FOD.FormOrderId
where INV.DestructionDate
BETWEEN '20110101' and '20120101'
group by
FMO.OrderDateTime,
Inv.FormNo, INV.FormDescription, INV.Product, INV.InventoryId -- using count to find how many destroyed if they have a destruction date
) AS q3 ON q1.InventoryId = q3.InventoryId

Mysql tekes too much time to excute sql query, based on multiple join

My Sql query takes more time to execute from mysql database server . There are number of tables are joined with sb_tblproperty table. sb_tblproperty is main table that contain more than 1,00,000 rows . most of table contain 50,000 rows.
How to optimize my sql query to fast execution. I have also used indexing.
indexing Explain - query - structure
SELECT `t1`.`propertyId`, `t1`.`projectId`,
`t1`.`furnised`, `t1`.`ownerID`, `t1`.`subType`,
`t1`.`fors`, `t1`.`size`, `t1`.`unit`,
`t1`.`bedrooms`, `t1`.`address`, `t1`.`dateConfirm`,
`t1`.`dateAdded`, `t1`.`floor`, `t1`.`priceAmount`,
`t1`.`priceRate`, `t1`.`allInclusive`, `t1`.`booking`,
`t1`.`bookingRate`, `t1`.`paidPercetage`,
`t1`.`paidAmount`, `t1`.`is_sold`, `t1`.`remarks`,
`t1`.`status`, `t1`.`confirmedStatus`, `t1`.`source`,
`t1`.`companyName` as company, `t1`.`monthly_rent`,
`t1`.`per_sqft`, `t1`.`lease_duration`,
`t1`.`lease_commencement`, `t1`.`lock_in_period`,
`t1`.`security_deposit`, `t1`.`security_amount`,
`t1`.`total_area_leased`, `t1`.`lease_escalation_amount`,
`t1`.`lease_escalation_years`, `t2`.`propertyTypeName` as
propertyTypeName, `t3`.`propertySubTypeName` subType,
`t3`.`propertySubTypeId` subTypeId, `Owner`.`ContactName`
ownerName, `Owner`.`companyName`, `Owner`.`mobile1`,
`Owner`.`otherPhoneNo`, `Owner`.`mobile2`,
`Owner`.`email`, `Owner`.`address` as caddress,
`Owner`.`contactType`, `P`.`projectName` as project,
`P`.`developerName` as developer, `c`.`name` as city,
if(t1.projectId="", group_concat( distinct( L.locality)),
group_concat( distinct(L2.locality))) as locality, `U`.`firstname`
addedBy, `U1`.`firstname` confirmedBy
FROM `sb_tblproperty` as t1
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
JOIN `sb_propertytype` t2 ON `t1`.`propertyType`= `t2`.`propertyTypeId`
JOIN `sb_propertysubtype` t3 ON `t1`.`subType` =`t3`.`propertySubTypeId`
LEFT JOIN `sb_tbluser` U ON `t1`.`addedBy` = `U`.`userId`
LEFT JOIN`sb_tbluser` U1 ON `t1`.`confirmedBy` = `U1`.`userId`
LEFT JOIN `sb_tblproject` P ON `P`.`id` = `t1`.`projectId` LEFT
JOIN `sb_tblpropertylocality` PL ON `t1`.`propertyId` = `PL`.`propertyId`
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
LEFT JOIN `sb_tblprojectlocality` PROL ON `PROL`.`projectId` = `P`.`id`
LEFT JOIN `sa_localitiez` L2 ON `L2`.`id` = `PROL`.`localityId`
LEFT JOIN `sb_tblfloor` F
ON `F`.`floorName` =`t1`.`floor`
WHERE `t1`.`is_sold` != '1' GROUP BY `t1`.`propertyId`
ORDER BY `t1`.`dateConfirm`
DESC LIMIT 1000
Please provide the EXPLAIN.
Meanwhile, try this:
SELECT ...
FROM (
SELECT propertyId
FROM sb_tblproperty
WHERE `is_sold` = 0
ORDER BY `dateConfirm` DESC
LIMIT 1000
) AS x
JOIN `sb_tblproperty` as t1 ON t1.propertyId = x.propertyId
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
...
LEFT JOIN `sb_tblfloor` F ON `F`.`floorName` =`t1`.`floor`
ORDER BY `t1`.`dateConfirm` DESC -- yes, again
Together with
INDEX(is_sold, dateConfirm)
How can t1.projectId="" ? Isn't projectId the PRIMARY KEY? (This is one of many reasons for needing the SHOW CREATE TABLE.)
If my suggestion leads to "duplicate" rows (that is, multiple rows with the same propertyId), don't simply add back the GROUP BY propertyId. Instead figure out why, and avoid the need for the GROUP BY. (That is probably the performance issue.)
A likely case is the GROUP_CONCAT. A common workaround is to change from
GROUP_CONCAT( distinct( L.locality)) AS Localities,
...
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
to
( SELECT GROUP_CONCAT(distinct locality)
FROM sa_localitiez
WHERE id = PL.localityId ) AS Localities
...
# and remove the JOIN

How to query mysql to select and group by multiple values

I'm trying to select and group by all the contentid values of the table below where the match criteria can be several different values.
the contentid values actually represent cars, so I need to select [and group by] all the contentis where the values are 'GMC' and the values are 'sedan' and the value is 'automatic.
i.e. I'm trying to select all the GMC sedans with an automatic transmission.
a query like this fails [obviously]:
select * from modx_site_tmplvar_contentvalues WHERE
`value` = 'gmc' and
`value` = 'tacoma'
group by contentid
I have no idea how to create a query like that. Any suggestions?
You need to "pivot" these data on "tmplvarid", but unfortunately for you MySQL doesn't have a PIVOT statement like other RDBMS. However, you can pivot it yourself by joining in the table multiple times for each variable you care about:
SELECT
contents.contentid,
transmission.value as transmission,
type.value as type,
make.value as make
FROM
(SELECT DISTINCT contentid FROM modx_site_tmplvar_contentvalues) AS contents
LEFT JOIN
modx_site_tmplvar_contentvalues AS transmission
ON contents.contentid = transmission.contentid
AND transmission.tmplvarid = 33 -- id for transmission
LEFT JOIN
modx_site_tmplvar_contentvalues AS make
ON contents.contentid = make.contentid
AND make.tmplvarid = 13 -- id for make
LEFT JOIN
modx_site_tmplvar_contentvalues AS type
ON contents.contentid = type.contentid
AND type.tmplvarid = 17 -- id for type
WHERE
type.value = 'sedan'
AND make.value = 'GMC'
AND transmission.value = 'automatic'
You can expand this with additional joins for other criteria such as year (id 15) or mileage (id 16).
If you need to use the value only, you could try:
SELECT DISTINCT
contents.contentid,
transmission.value as transmission,
type.value as type,
make.value as make
FROM
(SELECT DISTINCT contentid FROM modx_site_tmplvar_contentvalues) AS contents
INNER JOIN
modx_site_tmplvar_contentvalues AS transmission
ON contents.contentid = transmission.contentid
AND transmission.value = 'automatic'
INNER JOIN
modx_site_tmplvar_contentvalues AS make
ON contents.contentid = make.contentid
AND make.value = 'GMC'
INNER JOIN
modx_site_tmplvar_contentvalues AS type
ON contents.contentid = type.contentid
AND type.value = 'sedan'
In any case, make sure you have an index on the value column; these queries are going to get slow.
please try this:
SELECT *
FROM modx_site_tmplvar_contentvalues t1 INNER JOIN modx_site_tmplvar_contentvalues t2 ON t1.contentid = t2.content_id
WHERE
t1.`value` = 'gmc'
AND t2.`value` = 'tacoma';
You can do this with a group by. This is the most flexible in terms of expressing the conditions. In MySQL, multiple joins will often perform better:
select contentid
from modx_site_tmplvar_contentvalues
group by contentid
having sum(`value` = 'gmc') > 0 and
sum(`value` = 'tacoma') > 0;
This is always false:
`value` = 'gmc' and
`value` = 'tacoma'
Instead, use OR:
`value` = 'gmc' OR
`value` = 'tacoma'
In a condition "and" means "this and this is true at the same time". If you want all foos and all bars, then your condition is "foo OR bar".
EDIT:
To select groups containing your values, you can write subqueries:
SELECT DISTINCT name FROM table WHERE name IN (SELECT name FROM table WHERE value='value1') AND name IN (SELECT name FROM table WHERE value='value2')