Select monthly counts of data across two tables in mysql - mysql

Question

I have two tables, lastfm_scrobbles and lastfm_annotations. Example data:
mysql> select * from lastfm_scrobbles limit 5;
+---------+---------+-----------+---------------------+
| user_id | item_id | artist_id | scrobble_time |
+---------+---------+-----------+---------------------+
| 1469 | 45651 | 1 | 2010-06-30 13:57:42 |
| 1469 | 45651 | 1 | 2011-03-28 15:43:37 |
| 6872 | 45653 | 1 | 2013-08-03 15:07:44 |
| 7044 | 1370 | 1 | 2007-03-26 17:07:26 |
| 7044 | 1370 | 1 | 2007-08-24 18:41:35 |
+---------+---------+-----------+---------------------+
mysql> select * from lastfm_annotations limit 5;
+---------+---------+-----------+--------+------------+
| user_id | item_id | artist_id | tag_id | tag_month |
+---------+---------+-----------+--------+------------+
| 121 | 1330412 | 1330412 | 475 | 2006-12-01 |
| 121 | 1330412 | 1330412 | 517 | 2006-12-01 |
| 121 | 1330412 | 1330412 | 7280 | 2006-12-01 |
| 121 | 1330412 | 1330412 | 21384 | 2006-12-01 |
| 121 | 1330412 | 1330412 | 27872 | 2006-12-01 |
+---------+---------+-----------+--------+------------+
Furthermore, I have a user information table (lastfm_users). The details of this aren't important, but what is relevant is that the query:
select user_id from lastfm_users where scrobbles_recorded==1;
Returns the users I care about for the purposes of this question.
Ok, with that preamble out of the way: I need a query that will get me, for those users, the total number of entries they have in both the scrobbles and annotations tables for each month. In other words, the result should look something like:
user_id y m scrobble_count anno_count
123 2006 3 100 50
456 2008 11 321 10
... and so on
Make sense? I believe the query I want is a combination of the following:
select year(tag_month) as y, month(tag_month) as m, count(*) as anno_count
from lastfm_annotations where user_id in (select user_id from
lastfm_users where scrobbles_recorded=1)
group by user_id, year(tag_month), month(tag_month);
select year(scrobble_time) as y, month(scrobble_time) as m, count(*) as scrobble_count
from lastfm_scrobbles where user_id in (select user_id from
lastfm_users where scrobbles_recorded=1)
group by user_id, year(scrobble_time), month(scrobble_time);
But I'm not certain of the correct way to generate the join query to get the result I want. Suggestions?

Answer 1

You can try
select user_id, y, m,
coalesce(sum(case when source = 1 then total end), 0) anno_count,
coalesce(sum(case when source = 2 then total end), 0) scrobble_count
from
(
select 1 source, a.user_id, year(tag_month) y, month(tag_month) m, count(*) total
from lastfm_annotations a join lastfm_users u
on a.user_id = u.user_id
where u.scrobbles_recorded = 1
group by user_id, year(tag_month), month(tag_month)
union all
select 2 source, s.user_id, year(scrobble_time), month(scrobble_time), count(*)
from lastfm_scrobbles s join lastfm_users u
on s.user_id = u.user_id
where u.scrobbles_recorded = 1
group by user_id, year(scrobble_time), month(scrobble_time)
) q
group by user_id, y, m
or just
select user_id, y, m,
sum(case when source = 1 then 1 else 0 end) anno_count,
sum(case when source = 2 then 1 else 0 end) scrobble_count
from
(
select 1 source, a.user_id, year(tag_month) y, month(tag_month) m
from lastfm_annotations a join lastfm_users u
on a.user_id = u.user_id
where u.scrobbles_recorded = 1
union all
select 2 source, s.user_id, year(scrobble_time), month(scrobble_time)
from lastfm_scrobbles s join lastfm_users u
on s.user_id = u.user_id
where u.scrobbles_recorded = 1
) q
group by user_id, y, m;
Here is SQLFiddle demo