how to optimize mysql query with inner join - mysql

select id from customer_details where store_client_id = 2
And
id NOT IN (select customer_detail_id from orders
where store_client_id = 2 and total_spent > 100 GROUP BY customer_detail_id )
Or
id IN (select tcd.id from property_details as pd, customer_details as tcd
where pd.store_client_id = 2 and pd.customer_detail_id = tcd.customer_id and pd.property_key = 'Accepts Marketing'
and pd.property_value = 'no')
And
id IN (select customer_detail_id from orders
where store_client_id = 2 GROUP BY customer_detail_id HAVING count(customer_detail_id) > 0 )
Or
id IN (select tor.customer_detail_id from ordered_products as top, orders as tor
where tor.id = top.order_id and tor.store_client_id = 2
GROUP BY tor.customer_detail_id having sum(top.price) = 1)`
I have this mysql query with inner join so when it run in mysql server it slow down what is the issue cant find.
But after 4-5 minutes it return 15 000 records. This records is not an issue may be.
In some tutorial suggest to use Inner join, Left join,...
But I don't know how to convert this query in Join clause.
Any help will be appreciated. Thanks in advance.

First of all please read relational model and optimizing select statements.

Related

Need count of transactional table based on other tables including zeros where there are no matches

I have four tables, three of which are pretty static: haul_types, dumpster_type_team (the dumpster_type_team has the many-to-many relationship between dumpster_types and teams), and users. The fourth table, hauls, has transactional data.
haul_types:
id
name
dumpster_type_team:
id
dumpster_type_id
team_id
users:
id
first_name
last_name
is_driver
team_id
hauls:
haul_type_id
haul_status_id
set_dumpster_type_id
completed_driver_id
team_id
I would like a query that has a combination of dumpster_types, haul_types, and drivers (users) and a count of the hauls they were involved in. In some cases, there should be a count of zero because some drivers haven't completed hauls for every haul_type / dumpster type combination.
Here's the query I have so far that seems to be behaving as if it is an inner join because the records are getting filtered to only show where there are matches:
SELECT
c.haul_type_id,
c.dumpster_type_id,
c.driver_id,
count(h.id) AS haul_count
FROM
hauls h
RIGHT JOIN ( SELECT DISTINCT
ht.id AS haul_type_id,
dtt.dumpster_type_id AS dumpster_type_id,
dtt.team_id AS team_id,
u.id AS driver_id
FROM
haul_types ht
CROSS JOIN dumpster_type_team dtt
CROSS JOIN users u
WHERE
u.team_id = dtt.team_id
AND u.is_driver = TRUE) c ON c.haul_type_id = h.haul_type_id
AND c.dumpster_type_id = h.set_dumpster_type_id
AND c.driver_id = h.completed_driver_id
AND c.team_id = h.team_id
WHERE
h.team_id = 9
AND h.haul_status_id = 3
AND h.completed_driver_id IS NOT NULL
GROUP BY
c.haul_type_id, c.dumpster_type_id, c.driver_id
When I run the subquery in isolation:
SELECT DISTINCT
ht.id AS haul_type_id,
dtt.dumpster_type_id AS dumpster_type_id,
dtt.team_id AS team_id,
u.id AS driver_id
FROM
haul_types ht
CROSS JOIN dumpster_type_team dtt
CROSS JOIN users u
WHERE
u.team_id = dtt.team_id
AND u.is_driver = TRUE
I get the results I want: a row for each permutation of haul_type, dumpster_type, driver_id, and team_id. However, when I run the entire query, I get filtered results despite the right join.
What I would like to have is the following:
If I have 4 haul_types: delivery, swap, live, pickup
and 2 dumpster_types: 10YD, 15YD
and 2 drivers: 1, 2
I would like a haul count for the combination of haul_type, dumpster_type, and driver. If there are no hauls matching the row, show 0:
Any help is appreciated. Thank you
The description of the question and the query seem to have little to do with each other. I don't know what a "pivot table" is supposed to be.
I would like a query that has a combination of dumpster_types, haul_types, and drivers (users) and a count of the hauls they were involved in.
This sounds like a cross join to generate the rows and then a left join/group by to calculate the results:
select d.dumpster_id, ht.haul_type_id, d.driver_id, count(h.driver_id)
from dumpster_types d cross join
haul_types ht cross join
drivers d left join
hauls h
on h.dumpster_id = d.dumpster_id and
h.haul_type_id = ht.haul_type_id and
h.driver_id = d.driver_id
group by d.dumpster_id, ht.haul_type_id, d.driver_id;
Running the query #GordonLinoff provided, exposed the issue I was facing - when applying a where clause on the top level query, the results were getting filtered to only matches. I moved the where clause to individual subqueries and now I am getting all expected results.
Not sure if this is the most efficient way to write it but it yields the correct results:
SELECT
d.dumpster_type_id,
ht.id AS haul_type_id,
u.id AS driver_id,
count(h.id) AS haul_count
FROM (
SELECT
dumpster_type_id,
team_id
FROM
dumpster_type_team
WHERE
team_id = 9) d
CROSS JOIN haul_types ht
CROSS JOIN (
SELECT
users.id
FROM
users
WHERE
users.is_driver = TRUE
AND users.team_id = 9) u
LEFT JOIN (
SELECT
id, set_dumpster_type_id, haul_type_id, completed_driver_id, team_id
FROM
hauls
WHERE
haul_status_id = 3
AND team_id = 9) h ON h.set_dumpster_type_id = d.dumpster_type_id
AND h.haul_type_id = ht.id
AND h.completed_driver_id = u.id
AND h.team_id = d.team_id
GROUP BY
d.dumpster_type_id,
ht.id,
u.id

Mysql query taking 20 minutes only for 11000 records. how to optimize below mysql query select query in where clause with not exists

SELECT DISTINCT ACA.Application_No, AC.FirstName,AC.Id,AC.LastName,AC.MobileNo,CL.leadId
FROM ABSLI_PAYMENT_TRANSACTION APT
INNER JOIN ABSLI_CUSTOMER_APPLICATION ACA ON ACA.Policy_No=APT.policyId
INNER JOIN ABSLI_CUSTOMER AC ON AC.Id=ACA.CustomerId
LEFT JOIN ABSLI_CUSTOMER_LEAD CL ON CL.policyId = ACA.Policy_No
INNER JOIN ABSLI_Policy_Status_Tracking pst ON pst.policyId = APT.policyId
WHERE APT.paymentStatus='Y'
AND NOT EXISTS (SELECT 1 FROM ABSLI_SERVICE_STATUS WHERE PolicyNo=APT.policyId AND NAME = 'APEX_Validate')
AND ACA.Application_No NOT IN (SELECT RT.ApplicationNumber FROM ABSLI_REFUND_TRANSACTION RT WHERE RT.Status != 'Retain')
ORDER BY pst.updatedDate DESC;
Correlated subqueries can be costly on larger datasets, you might want to try converting
...
AND NOT EXISTS (SELECT 1 FROM ABSLI_SERVICE_STATUS WHERE PolicyNo=APT.policyId AND NAME = 'APEX_Validate')
...
to something like
...
LEFT JOIN ABSLI_SERVICE_STATUS AS ss ON APT.policyId = ss.PolicyNo AND ss.NAME = 'APEX_Validate'
...
AND ss.NAME IS NULL
Normally, I'd suggest an "id" field from ss for the IS NULL check, but NAME obviously exists (from the query you have) and cannot be 'APEX_Validate' and NULL at the same time. Also, if there is a compound index on PolicyNo, Name that index can probably be used without accessing the table itself.)
Considering you have an index on ABSLI_REFUND_TRANSACTION.ApplicationNumber column, you can try to use NOT EXISTS also for the second subquery instead of NOT IN. This way, you benefit this index for that subquery, and usually NOT IN might be problematic for large set of data.
SELECT DISTINCT ACA.Application_No, AC.FirstName, AC.Id, AC.LastName, AC.MobileNo, CL.leadId
FROM ABSLI_PAYMENT_TRANSACTION APT
INNER JOIN ABSLI_CUSTOMER_APPLICATION ACA
ON ACA.Policy_No = APT.policyId
INNER JOIN ABSLI_CUSTOMER AC
ON AC.Id = ACA.CustomerId
LEFT JOIN ABSLI_CUSTOMER_LEAD CL
ON CL.policyId = ACA.Policy_No
INNER JOIN ABSLI_Policy_Status_Tracking pst
ON pst.policyId = APT.policyId
WHERE APT.paymentStatus = 'Y'
AND NOT EXISTS (SELECT 1
FROM ABSLI_SERVICE_STATUS
WHERE PolicyNo = APT.policyId
AND NAME = 'APEX_Validate')
AND NOT EXISTS (SELECT 1
FROM ABSLI_REFUND_TRANSACTION RT
WHERE RT.Status != 'Retain'
AND RT.ApplicationNumber = ACA.Application_No)
ORDER BY pst.updatedDate DESC;
But without knowing the execution plan, hard to tell more about performance.

Mysql tekes too much time to excute sql query, based on multiple join

My Sql query takes more time to execute from mysql database server . There are number of tables are joined with sb_tblproperty table. sb_tblproperty is main table that contain more than 1,00,000 rows . most of table contain 50,000 rows.
How to optimize my sql query to fast execution. I have also used indexing.
indexing Explain - query - structure
SELECT `t1`.`propertyId`, `t1`.`projectId`,
`t1`.`furnised`, `t1`.`ownerID`, `t1`.`subType`,
`t1`.`fors`, `t1`.`size`, `t1`.`unit`,
`t1`.`bedrooms`, `t1`.`address`, `t1`.`dateConfirm`,
`t1`.`dateAdded`, `t1`.`floor`, `t1`.`priceAmount`,
`t1`.`priceRate`, `t1`.`allInclusive`, `t1`.`booking`,
`t1`.`bookingRate`, `t1`.`paidPercetage`,
`t1`.`paidAmount`, `t1`.`is_sold`, `t1`.`remarks`,
`t1`.`status`, `t1`.`confirmedStatus`, `t1`.`source`,
`t1`.`companyName` as company, `t1`.`monthly_rent`,
`t1`.`per_sqft`, `t1`.`lease_duration`,
`t1`.`lease_commencement`, `t1`.`lock_in_period`,
`t1`.`security_deposit`, `t1`.`security_amount`,
`t1`.`total_area_leased`, `t1`.`lease_escalation_amount`,
`t1`.`lease_escalation_years`, `t2`.`propertyTypeName` as
propertyTypeName, `t3`.`propertySubTypeName` subType,
`t3`.`propertySubTypeId` subTypeId, `Owner`.`ContactName`
ownerName, `Owner`.`companyName`, `Owner`.`mobile1`,
`Owner`.`otherPhoneNo`, `Owner`.`mobile2`,
`Owner`.`email`, `Owner`.`address` as caddress,
`Owner`.`contactType`, `P`.`projectName` as project,
`P`.`developerName` as developer, `c`.`name` as city,
if(t1.projectId="", group_concat( distinct( L.locality)),
group_concat( distinct(L2.locality))) as locality, `U`.`firstname`
addedBy, `U1`.`firstname` confirmedBy
FROM `sb_tblproperty` as t1
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
JOIN `sb_propertytype` t2 ON `t1`.`propertyType`= `t2`.`propertyTypeId`
JOIN `sb_propertysubtype` t3 ON `t1`.`subType` =`t3`.`propertySubTypeId`
LEFT JOIN `sb_tbluser` U ON `t1`.`addedBy` = `U`.`userId`
LEFT JOIN`sb_tbluser` U1 ON `t1`.`confirmedBy` = `U1`.`userId`
LEFT JOIN `sb_tblproject` P ON `P`.`id` = `t1`.`projectId` LEFT
JOIN `sb_tblpropertylocality` PL ON `t1`.`propertyId` = `PL`.`propertyId`
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
LEFT JOIN `sb_tblprojectlocality` PROL ON `PROL`.`projectId` = `P`.`id`
LEFT JOIN `sa_localitiez` L2 ON `L2`.`id` = `PROL`.`localityId`
LEFT JOIN `sb_tblfloor` F
ON `F`.`floorName` =`t1`.`floor`
WHERE `t1`.`is_sold` != '1' GROUP BY `t1`.`propertyId`
ORDER BY `t1`.`dateConfirm`
DESC LIMIT 1000
Please provide the EXPLAIN.
Meanwhile, try this:
SELECT ...
FROM (
SELECT propertyId
FROM sb_tblproperty
WHERE `is_sold` = 0
ORDER BY `dateConfirm` DESC
LIMIT 1000
) AS x
JOIN `sb_tblproperty` as t1 ON t1.propertyId = x.propertyId
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
...
LEFT JOIN `sb_tblfloor` F ON `F`.`floorName` =`t1`.`floor`
ORDER BY `t1`.`dateConfirm` DESC -- yes, again
Together with
INDEX(is_sold, dateConfirm)
How can t1.projectId="" ? Isn't projectId the PRIMARY KEY? (This is one of many reasons for needing the SHOW CREATE TABLE.)
If my suggestion leads to "duplicate" rows (that is, multiple rows with the same propertyId), don't simply add back the GROUP BY propertyId. Instead figure out why, and avoid the need for the GROUP BY. (That is probably the performance issue.)
A likely case is the GROUP_CONCAT. A common workaround is to change from
GROUP_CONCAT( distinct( L.locality)) AS Localities,
...
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
to
( SELECT GROUP_CONCAT(distinct locality)
FROM sa_localitiez
WHERE id = PL.localityId ) AS Localities
...
# and remove the JOIN

How to speed up multiple counts query and get a result

I've got a query I'm trying to write to get counts of active users and active contacts associated with each account. I have attempted to run the counts separately and in both cases they run at under 1 sec but when I put them together as seen below I don't get a result. Please let me know if there is anything I can do it enhance the query.
select count(c.c_no) as contacts_count, count(u_no) as user_count, a.*
from accounts a
LEFT JOIN users u on u.a_no = a.a_no and u_status = 1
LEFT JOIN IDP1.contacts c on c.a_no = a.a_no and c_status = 1
where a_status = 1
group by a_no
THANK YOU!
Do not use * and aggregate functions simultaneously
Try this
select count(c.c_no) as contacts_count, count(u_no) as user_count, a_no
from accounts a
LEFT JOIN users u on u.a_no = a.a_no and u_status = 1
LEFT JOIN IDP1.contacts c on c.a_no = a.a_no and c_status = 1
where a_status = 1
group by a_no

combine tables with 1 to N relationship into 1 line of record with the last value of the N record

I need a modification of my previous post regarding
how to combine tables with 1 to many relationship into 1 line of record
how to combine tables with 1 to many relationship into 1 line of record
now my problem is my record has now 1 to many relationship. What I need to show is the last record only and combine it in a single line
tables tbl_equipment and tbl_warranty
and here is the desired output
here is the code I'm trying to implement
SELECT
a.equipmentid,
a.codename,
a.name,
a.labelid,
a.ACQUISITIONDATE,
a.description,
a.partofid,
w1.warrantyid as serviceidwarranty,
w1.startdate,
w1.enddate,
w2.warrantyid as productidwarranty,
w2.startdate,
w2.enddate,
s.equipstatusid,
l.equiplocationid FROM TBL_EQUIPMENTMST a
left JOIN tbl_equipwarranty w1
ON w1.equipmentid=a.equipmentid and w1.serviceproduct = 'service'
left JOIN tbl_equipwarranty w2
ON w2.equipmentid=a.equipmentid and w2.serviceproduct = 'product'
left join tbl_equipstatus s
on a.equipmentid = s.equipmentid
left join tbl_equiplocation l
on a.equipmentid = l.equipmentid WHERE a.equipmentid = '112'
I only want to show 1 record with the last value of warranty product and warranty service in the output. Can anyone guide me how to modify my code so that when I try join all the tables listed above can produce 1 record only with the last record of warranty as an output.
I am using firebird as a database. If you have a solution in mysql kindly tell me and ill try to find the counterpart in firebird.
with summary as(
select e.equipmentid ,e.Codename,e.Name,w.warrantyid ,w.Satartdate ,w.Enddate,w.warrantytype
from Eqp e
join Warranty w
on(w.equipmentid =e.equipmentid )
where w.warrantyid =3)
select *,w.warrantyid,w.Satartdate ,w.Enddate,w.warrantytype
from summary s
join Warranty w
on s.Satartdate =w.Satartdate and s.Enddate =w.Enddate
where w.warrantyid =4
after reading the comment of Barmar at the question for solution. I Figured out subquery can solve my problem. Subquery is a new word for me. I research on how to use subquery and came out with a solution below. you can correct me if my code is wrong or how to improve the performance of the query
SELECT
a.equipmentid,a.codename,a.name,a.labelid,a.ACQUISITIONDATE,a.description,a.partofid,
w1.warrantyid as serviceidwarranty,w1.startdate,w1.enddate,
w2.warrantyid as productidwarranty,w2.startdate,w2.enddate,
s.equipstatusid,
l.equiplocationid
FROM
TBL_EQUIPMENTMST a
left JOIN
(select first 1 *
from tbl_equipwarranty
where equipmentid='112' and serviceproduct = 'service'
order by warrantyid desc) w1 ON w1.equipmentid = a.equipmentid
and w1.serviceproduct = 'service'
left JOIN
(select first 1 *
from tbl_equipwarranty
where equipmentid = '112' and serviceproduct = 'product'
order by warrantyid desc) w2 ON w2.equipmentid = a.equipmentid
and w2.serviceproduct = 'product'
left join
(select first 1 *
from tbl_equipstatus
where equipmentid = '112'
order by equipstatusid desc) s on a.equipmentid = s.equipmentid
left join
(select first 1 *
from tbl_equiplocation
where equipmentid = '112'
order by equiplocationid desc) l on a.equipmentid = l.equipmentid
WHERE
a.equipmentid = '112'