MySQL - Slow Query when adding multiple derived tables - Optimization - mysql

Question

For my query, the two derived tables at the bottom are causing a crazy slow up for this query. The query, as is, takes about 45-55 seconds to execute.. NOW, when i remove just one of those derived tables (it does not matter which one) the query goes down to 0.1 - 0.3 seconds. My questions; Is there an issue with having multiple derived tables? Is there a better way to execute this? My indexes all seem to be correct, I will also include the explain from this query.
select t.name as team, u.name as "REP NAME",
count(distinct activity.id) as "TOTAL VISITS",
count(distinct activity.account_id) as "UNIQUE VISITS",
count(distinct placement.id) as "COMMITMENTS ADDED",
CASE WHEN
count(distinct activity.account_id) = 0 THEN (count(distinct
placement.id) / 1)
else (cast(count(distinct placement.id) as decimal(10,2)) /
cast(count(distinct activity.account_id) as decimal(10,2)))
end as "UNIQUE VISIT TO COMMITMENT %",
case when o.mode='basic' then count(distinct placement.id) else
count(distinct(case when placement.commitmentstatus='fullfilled'
then placement.id else 0 end))
end as "COMMITMENTS FULFILLED",
case when o.mode='basic' then 1 else
(CASE WHEN
count(distinct placement.id) = 0 THEN (count(distinct(case when
placement.commitmentstatus='fullfilled' then placement.id else 0
end)) / 1)
else (cast(count(distinct(case when
placement.commitmentstatus='fullfilled' then placement.id else 0
end)) as decimal(10,2)) / cast(count(distinct placement.id) as
decimal(10,2)))
end) end as "COMMITMENT TO FULFILLMENT %"
from lpmysqldb.users u
left join lpmysqldb.teams t on t.team_id=u.team_id
left join lpmysqldb.organizations o on o.id=t.org_id
left join (select * from lpmysqldb.activity where
org_id='555b918ae4b07b6ac5050852' and completed_at>='2018-05-01' and
completed_at<='2018-06-01' and tag='visit' and accountname is not
null and (status='active' or status='true' or status='1')) as
activity on activity.user_id=u.id
left join (select * from lpmysqldb.placements where
orgid='555b918ae4b07b6ac5050852' and placementdate>='2018-05-01' and
placementdate<='2018-06-01' and (status IN ('1','active','true') or
status is null)) as placement on placement.userid=u.id
where u.org_id='555b918ae4b07b6ac5050852'
and (u.status='active' or u.status='true' or u.status='1')
and istestuser!='1'
group by u.org_id, t.name, u.id, u.name, o.mode
order by count(distinct activity.id) desc
Thank you for assistance!
I have edited below with changing the two bottom joins from joining on subqueries to joining on the table directly. Still yielding the same result.

Answer 1

This is a SLIGHTLY restructured query of your same. Might be simplified as the last two subqueries are all pre-aggregated for your respective counts and count distincts so you can use those column names directly instead of showing all the count( distinct ) embedded throughout the query.
I also tried to simplify the division by multiplying a given count by 1.00 to force decimal-based precision as result.
select
t.name as team,
u.name as "REP NAME",
Activity.DistIdCnt as "TOTAL VISITS",
Activity.UniqAccountCnt as "UNIQUE VISITS",
Placement.DistIdCnt as "COMMITMENTS ADDED",
Placement.DistIdCnt /
CASE WHEN Activity.UniqAccountCnt = 0
THEN 1.00
ELSE Activity.UniqAccountCnt * 1.00
end as "UNIQUE VISIT TO COMMITMENT %",
case when o.mode = 'basic'
then Placement.DistIdCnt
else Placement.DistFulfillCnt
end as "COMMITMENTS FULFILLED",
case when o.mode = 'basic'
then 1
else ( Placement.DistFulfillCnt /
CASE when Placement.DistIdCnt = 0
then 1.00
ELSE Placement.DistIdCnt * 1.00
END TRANSACTION )
END as "COMMITMENT TO FULFILLMENT %"
from
lpmysqldb.users u
left join lpmysqldb.teams t
on u.team_id = t.team_id
left join lpmysqldb.organizations o
on t.org_id = o.id
left join
( select
user_id,
count(*) as AllRecs,
count( distinct id ) DistIdCnt,
count( distinct account_id) as UniqAccountCnt
from
lpmysqldb.activity
where
org_id = '555b918ae4b07b6ac5050852'
and completed_at>='2018-05-01'
and completed_at<='2018-06-01'
and tag='visit'
and accountname is not null
and status IN ( '1', 'active', 'true')
group by
user_id ) activity
on u.id = activity.user_id
left join
( select
userid,
count(*) AllRecs,
count(distinct id) as DistIdCnt,
count(distinct( case when commitmentstatus = 'fullfilled'
then id
else 0 end )) DistFulfillCnt
from
lpmysqldb.placements
where
orgid = '555b918ae4b07b6ac5050852'
and placementdate >= '2018-05-01'
and placementdate <= '2018-06-01'
and ( status is null OR status IN ('1','active','true')
group by
userid ) as placement
on u.id = placement.userid
where
u.org_id = '555b918ae4b07b6ac5050852'
and u.status IN ( 'active', 'true', '1')
and istestuser != '1'
group by
u.org_id,
t.name,
u.id,
u.name,
o.mode
order by
activity.DistIdCnt desc
FINALLY, your inner queries are querying for ALL users. If you have a large count of users that are NOT active, you MIGHT exclude those users from each inner query by adding those join/criteria there too such as...
( ...
from
lpmysqldb.placements
JOIN lpmysqldb.users u2
on placements.userid = u2.id
and u2.status IN ( 'active', 'true', '1')
and u2.istestuser != '1'
where … ) as placement