Use main query condition on sub query - mysql

Question

I have a table "products" and a table "links". every product can have multiple links, each link can have many sales, clicks and impressions, but doesn't necessarily have all of them. I want to get a list of links of a certain product matching some criteria for them. I want to get this data grouped per day and campaign and link banner size.
The following query works correctly, but it's much slower than it could be. The problem is that the subqueries get the data for all link ids and it's just filtered in the end. The overall query would become much faster if the sub queries included something like
where product_id IN (...) but I only know the link_ids from the main query, not before
if I try to add
where link_id = l.id
it's obviously an unknown column, because the sub query doesn't have access to the main queries results.
how can I let the sub queries only look up data for the link_Ids that the main query found? I could split it up to 2 complete separate queries, but is this possible in one query?
select p.id, p.name, l.id, l.banner_size,
coalesce(sum(case when t1.col = 'sales' then ct else 0 end), 0) as total_sales,
coalesce(sum(case when t1.col = 'clicks' then ct else 0 end), 0) as total_clicks,
coalesce(sum(case when t1.col = 'impressions' then ct else 0 end), 0) as total_impressions,
t1.dt
from links l
inner join products p
on p.id = l.product_id
left join
(
select count(1) as ct, link_id, date(clicked) dt, 'sales' as col
from sales
where clicked >= '2020-01-01 00:00:00' and clicked <= '2020-01-31 00:00:00'
group by date(clicked), link_id
union all
select count(1) as ct, link_id, date(created) dt, 'clicks'
from clicks_source1
where created >= '2020-01-01 00:00:00' and created <= '2020-01-31 00:00:00'
group by date(created), link_id
union all
select count(1) as ct, link_id, date(time) dt, 'clicks'
from clicks_source2
where time >= '2020-01-01 00:00:00' and time <= '2020-01-31 00:00:00'
group by date(time), link_id
union all
select count(1) as ct, link_id, date(created) dt, 'impressions'
from impression_source1
where created > '2020-01-01 00:00:00' and created <= '2020-01-31 00:00:00'
group by date(created), link_id
union all
select count(1) as ct, link_id, date(time) dt, 'impressions'
from impression_source2
where time > '2020-01-01 00:00:00' and time <= '2020-01-31 00:00:00'
group by date(time), link_id
) t1 on t1.link_id = l.id
where l.agent_id = 300
and p.id = 3454
and l.banner_size = 48
and p.company NOT IN (13592, 28189)
group by c.id, l.banner_size, t1.dt
having (total_sales + total_clicks + total_impressions) > 0
order by dt DESC, p.id ASC, l.banner_size ASC

Answer 1

What you'd like to use is called lateral joins, but MySQL doesn't feature them.
One solution is to move the subqueries for the counts to the select clause:
select
id,
name,
coalesce((select count(*) from views v where v.product_id = p.product_id), 0)
as total_views,
coalesce((select count(*) from clicks c where c.product_id = p.product_id), 0)
as total_clicks
from products
where status = 1;
It is unnecessary that you combined views and clicks in one subquery. Maybe it's just this, that kept the optimizer from getting a better execution plan. You can try the following and check whether it's already much faster than your original query.
select
p.id,
p.name,
coalesce(v.total, 0) as total_views
coalesce(c.total, 0) as total_clicks
from products
left join (select product_id, count(*) as total from views group by product_id) v
on v.product_id = p.product_id
left join (select product_id, count(*) as total from clicks group by product_id) c
on c.product_id = p.product_id
where p.status = 1;

Answer 2

You say that your example is much simplified. Maybe you can just apply the condition early by repeating it. E.g.:
select p.id, p.name,
coalesce(sum(case when t1.col = 'views' then ct else 0 end), 0) as total_views,
coalesce(sum(case when t1.col = 'clicks' then ct else 0 end), 0) as total_clicks
from products p
left join
(
select count(1) as ct, product_id, 'views' as col
from views
where product_id in (select product_id from products where status = 1)
group by product_id
union all
select count(1) as ct, product_id, 'clicks' as col
from clicks
where product_id in (select product_id from products where status = 1)
group by product id
) t1 on t1.product_id = p.product_id
where p.status = 1;
Or with a WITH clause:
with p as (select * from products where status = 1)
select p.id, p.name,
coalesce(sum(case when t1.col = 'views' then ct else 0 end), 0) as total_views,
coalesce(sum(case when t1.col = 'clicks' then ct else 0 end), 0) as total_clicks
from p
left join
(
select count(1) as ct, product_id, 'views' as col
from views
where product_id in (select product_id from p)
group by product_id
union all
select count(1) as ct, product_id, 'clicks' as col
from clicks
where product_id in (select product_id from p)
group by product id
) t1 on t1.product_id = p.product_id
;