Show all data in a date range using MYSQL recursive function - mysql

Question

I'm trying to get a list of sales for the past 6 months and get 0 values if I have no data for a specific month. So I'm using recursive_all_dates to generate a date range for the past 6 months which works great:
with recursive all_dates(dt) as (
-- anchor
select DATE_SUB(now(), INTERVAL 6 MONTH) dt
union all
-- recursion with stop condition
select dt + interval 1 month from all_dates where dt + interval 1 month <= DATE(now())
)
select DATE_FORMAT(dt, '%Y-%m') as ym from all_dates
This will return:
ym
------
2019-10
2019-11
2019-12
2020-01
2020-02
2020-03
2020-04
Now I want to left join this with my real data:
with recursive all_dates(dt) as (
-- anchor
select DATE_SUB(now(), INTERVAL 6 MONTH) dt
union all
-- recursion with stop condition
select dt + interval 1 month from all_dates where dt + interval 1 month <= now()
)
SELECT
DATE_FORMAT(ad.dt, '%Y-%m') as ym,
sum(profit) as profit
FROM
all_dates as ad
LEFT JOIN organisation_invoices as i
ON
DATE_FORMAT(ad.dt, '%Y-%m') = DATE_FORMAT(i.issue_date, '%Y-%m')
JOIN (
SELECT
invoice_id,
SUM(value) as profit
FROM organisation_invoice_services isrv
GROUP BY invoice_id
) isrv
ON i.id = isrv.invoice_id
WHERE
i.organisation_id = '4b166dbe-d99d-5091-abdd-95b83330ed3a' AND
i.issue_date >= DATE_SUB(NOW(), INTERVAL 6 MONTH)
GROUP BY `ym`
ORDER BY `ym` ASC
But I still only get the populated months:
ym profit
------------------
2019-12 8791
2020-02 302
2020-04 10452
The desired result:
ym profit
------------------
2019-10 0
2019-11 0
2019-12 8791
2020-01 0
2020-02 302
2020-03 0
2020-04 10452
What am I missing?
Edit: Sample data set and fiddle:
CREATE TABLE `organisation_invoices` (
`id` varchar(255) NOT NULL,
`organisation_id` varchar(255) NOT NULL,
`issue_date` date NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `organisation_invoice_services` (
`id` varchar(255) NOT NULL,
`organisation_id` varchar(255) NOT NULL,
`invoice_id` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`qty` float NOT NULL,
`value` float NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `organisation_invoices` (id, organisation_id, issue_date)
VALUES ('e11cec69-138f-4e20-88e5-5430b6c8d0a1', '4b166dbe-d99d-5091-abdd-95b83330ed3a', '2020-01-20');
INSERT INTO `organisation_invoice_services` (id, organisation_id, invoice_id, qty, `value`)
VALUES ('fe45dfd67-138f-4e20-88e5-5430b6c8d0a1', '4b166dbe-d99d-5091-abdd-95b83330ed3a', 'e11cec69-138f-4e20-88e5-5430b6c8d0a1', 1, 1000);
https://www.db-fiddle.com/f/dibyQi31CBtr2Cr8vjJA8i/0

Answer 1

You can use the following:
with recursive all_dates(dt) as (
-- anchor
select DATE_SUB(now(), INTERVAL 6 MONTH) dt
union all
-- recursion with stop condition
select dt + interval 1 month from all_dates where dt + interval 1 month <= now()
)
SELECT DATE_FORMAT(ad.dt, '%Y-%m') as ym, IFNULL(sum(profit),0) as profit
FROM all_dates as ad
LEFT JOIN organisation_invoices as i
ON DATE_FORMAT(ad.dt, '%Y-%m') = DATE_FORMAT(i.issue_date, '%Y-%m')
LEFT JOIN (
SELECT
invoice_id,
SUM(value) as profit
FROM organisation_invoice_services isrv
GROUP BY invoice_id
) isrv
ON i.id = isrv.invoice_id
WHERE
(i.organisation_id = '4b166dbe-d99d-5091-abdd-95b83330ed3a' AND
i.issue_date >= DATE_SUB(NOW(), INTERVAL 6 MONTH)) OR i.organisation_id IS NULL
GROUP BY `ym`
ORDER BY `ym` ASC
demo on dbfiddle.uk
Changes:
The conditions on the WHERE clause change the behaviour of your LEFT JOIN. Since you check for a specific organization_id, you only get matches between your month table and data (the LEFT JOIN behaves like a INNER JOIN). You need the following WHERE clause instead:
WHERE (i.organisation_id = '4b166dbe-d99d-5091-abdd-95b83330ed3a' AND
i.issue_date >= DATE_SUB(NOW(), INTERVAL 6 MONTH)) OR i.organisation_id IS NULL
You also have to change the second JOIN to a LEFT JOIN.