LEFT JOIN on the same table doubling rows - mysql

I have two tables that I am trying to LEFT join but I am not getting the expected results.
Rooms have multiple Children on different days, however Children are only counted in a Room after they have started and if they have hours allocated on that day. The output I am trying to achieve is this.
Room | MaxNum | Mon(Week1) | Tue(Week1) | Mon(Week2) | Tue(Week2)
Blue | 5 | 4 | 4 | 3 | 2
Green | 10 | 10 | 10 | 9 | 9
Red | 15 | 15 | 15 | 15 | 15
Here is the schema and some data...
create table Rooms(
id INT,
RoomName VARCHAR(10),
MaxNum INT
);
create table Children (
id INT,
RoomID INT,
MonHrs INT,
TueHrs INT,
StartDate DATE
);
INSERT INTO Rooms VALUES (1, 'Blue', 5);
INSERT INTO Rooms VALUES (2, 'Green', 10);
INSERT INTO Rooms VALUES (3, 'Red', 15);
INSERT INTO Children VALUES (1, 1, 5, 0, '2018-12-02');
INSERT INTO Children VALUES (2, 1, 0, 5, '2018-12-02');
INSERT INTO Children VALUES (3, 1, 5, 5, '2018-12-09');
INSERT INTO Children VALUES (4, 1, 0, 5, '2018-12-09');
INSERT INTO Children VALUES (5, 2, 5, 0, '2018-12-09');
INSERT INTO Children VALUES (6, 2, 0, 5, '2018-12-09');
The SQL I am having trouble with is this. It may not be the correct approach.
SELECT R.RoomName, R.MaxNum,
R.MaxNum - SUM(CASE WHEN C1.MonHrs > 0 THEN 1 ELSE 0 END) AS Mon1,
R.MaxNum - SUM(CASE WHEN C1.TueHrs > 0 THEN 1 ELSE 0 END) AS Tue1,
R.MaxNum - SUM(CASE WHEN C2.MonHrs > 0 THEN 1 ELSE 0 END) AS Mon2,
R.MaxNum - SUM(CASE WHEN C2.TueHrs > 0 THEN 1 ELSE 0 END) AS Tue2
FROM Rooms R
LEFT JOIN Children C1
ON R.id = C1.RoomID
AND C1.StartDate <= '2018-12-02'
LEFT JOIN Children C2
ON R.id = C2.RoomID
AND C2.StartDate <= '2018-12-09'
GROUP BY R.RoomName;
There is a double up happening on the Rows in the LEFT JOINs that is throwing the counts way off and I don't know how to prevent them. You can see the effect if you replace the SELECT with *
Any suggestions would help a lot.

This sort of problem usually surfaces from doing an aggregation in a too broad point in the query, which then results in duplicate counting of records. Try aggregating the Children table in a separate subquery:
SELECT
R.RoomName,
R.MaxNum,
R.MaxNum - C.Mon1 AS Mon1,
R.MaxNum - C.Tue1 AS Tue1,
R.MaxNum - C.Mon2 AS Mon2,
R.MaxNum - C.Tue2 AS Tue2
FROM Rooms R
LEFT JOIN
(
SELECT
RoomID,
COUNT(CASE WHEN MonHrs > 0 AND StartDate <= '2018-12-02'
THEN 1 END) AS Mon1,
COUNT(CASE WHEN TueHrs > 0 AND StartDate <= '2018-12-02'
THEN 1 END) AS Tue1,
COUNT(CASE WHEN MonHrs > 0 AND StartDate <= '2018-12-09'
THEN 1 END) AS Mon2,
COUNT(CASE WHEN TueHrs > 0 AND StartDate <= '2018-12-09'
THEN 1 END) AS Tue2
FROM Children
GROUP BY RoomID
) C
ON R.id = C.RoomID;
Note that we can avoid the double left join in your original query by instead using conditional aggregation on the start date.
Late edit: You probably don't even need a subquery at all, q.v. the answer by #Salman. But either of our answers should resolve the double counting problem.

You need to use one LEFT JOIN and move the date filter from JOIN condition to the aggregate:
SELECT R.id, R.RoomName, R.MaxNum
, R.MaxNum - COUNT(CASE WHEN C.StartDate <= '2018-12-02' AND C.MonHrs > 0 THEN 1 END) AS Mon1
, R.MaxNum - COUNT(CASE WHEN C.StartDate <= '2018-12-02' AND C.TueHrs > 0 THEN 1 END) AS Tue1
, R.MaxNum - COUNT(CASE WHEN C.StartDate <= '2018-12-09' AND C.MonHrs > 0 THEN 1 END) AS Mon2
, R.MaxNum - COUNT(CASE WHEN C.StartDate <= '2018-12-09' AND C.TueHrs > 0 THEN 1 END) AS Tue2
FROM Rooms R
LEFT JOIN Children C ON R.id = C.RoomID
GROUP BY R.id, R.RoomName, R.MaxNum

Related

loop over a date list (or any list) and append queries in mysql or snowflake

I am new to sql language and recently snowflake. I have a table that contains all checkin dates for all users for a business
user_id | checkin_date
001 03-06-2018
001 07-07-2018
001 08-01-2018
002 03-19-2018
002 03-27-2018
002 07-11-2018
Now I want to do a query such that I can look back from a query_date to see how many times each user checked in between query_date - 7 and query_date, qyery_date - 90 and query date ... the following snowflake query does the job properly for query_date='2018-08-01'.
with user_checkin_history_sum as (
select
user_id,
sum(iff(datediff(DAY, uc.checkin_date, '2018-08-01') <= 7, 1, 0)) as visits_past_7_days,
sum(iff(datediff(DAY, uc.checkin_date, '2018-08-01') <= 90, 1, 0)) as visits_past_90_days,
from user_checkin as uc
where uc.checkin_date < '2018-08-01'
group by user_id
order by user_id
)
This gives me result
user_id | visits_past_7_days | visits_past_90_days
001 0 2
002 0 1
My question is, if I have more than one day as the query_date, i.e., I have a list of checkin_date, for each checkin_date in the list, I do the query as above and append all them together. Basically, it is a loop over + table append, but I do not find an answer how to do this in sql language. Essentially, what I want to do is like the following
with user_checkin_history_sum as (
select
user_id,
sum(iff(datediff(DAY, uc.checkin_date, query_date) <= 7, 1, 0)) as visits_past_7_days,
sum(iff(datediff(DAY, uc.checkin_date, query_date) <= 90, 1, 0)) as visits_past_90_days,
from user_checkin as uc
where uc.checkin_date < query_date and
LOOP OVER
query_date in ('2018-08-01', '2018-06-01')
group by user_id
order by user_id
)
And hopefully it gives this result
user_id | query_date | visits_past_7_days | visits_past_90_days
001 '08-01-2018' 0 2
002 '08-01-2018' 0 1
001 '06-01-2018' 0 1
002 '06-01-2018' 0 2
You should be able to cross join a table containing all the dates you want to examine:
WITH dates AS (
SELECT '2018-06-01' AS query_date UNION ALL
SELECT '2018-08-01' UNION ALL
... -- maybe other dates as well
),
user_checkin_history_sum AS (
SELECT
uc.user_id,
d.query_date,
SUM(IFF(DATEDIFF(DAY, uc.checkin_date, d.query_date) <= 7, 1, 0)) AS visits_past_7_days,
SUM(IFF(DATEDIFF(DAY, uc.checkin_date, d.query_date) <= 90, 1, 0)) AS visits_past_90_days
FROM dates d
CROSS JOIN user_checkin AS uc
WHERE uc.checkin_date < '2018-08-01'
GROUP BY d.query_date, uc.user_id
ORDER BY d.query_date, uc.user_id
)

How to get only latest record from different ranges?

I am looking at a case in which we have a number of tanks filled with liquid. The amount of liquid is measured and information is stored in a database. This update is done every 5 minutes. Here the following information is stored:
tankId
FillLevel
TimeStamp
Each tank is categorized in one of the following 'fill-level' ranges:
Range A: 0 - 40%
Range B: 40 - 75%
Range C: 75 - 100%
Per range I count the amount of events per tankId.
SELECT sum(
CASE
WHEN filllevel>=0 and filllevel<40
THEN 1
ELSE 0
END) AS 'Range A',
sum(
CASE
WHEN filllevel>=40 and filllevel<=79
THEN 1
ELSE 0
END) AS 'Range B',
sum(
CASE
WHEN filllevel>79 and filllevel<=100
THEN 1
ELSE 0
END) AS 'Range C'
FROM TEST ;
The challenge is to ONLY count the latest record for each tank. So for each tankId there is only one count (and that must be the record with the latest time stamp).
For the following data:
insert into tank_db1.`TEST` (ts, tankId, fill_level) values
('2017-08-11 03:31:18', 'tank1', 10),
('2017-08-11 03:41:18', 'tank1', 45),
('2017-08-11 03:51:18', 'tank1', 95),
('2017-08-11 03:31:18', 'tank2', 20),
('2017-08-11 03:41:18', 'tank2', 30),
('2017-08-11 03:51:18', 'tank2', 80),
('2017-08-11 03:31:18', 'tank3', 30),
('2017-08-11 03:41:18', 'tank3', 45),
('2017-08-11 03:51:18', 'tank4', 55);
I would expect the outcome to be (only the records with the latest timestamp per tankId are counted):
- RANGE A: 0
- RANGE B: 1 (tankdId 3)
- RANGE C: 2 (tankId 1 and tankId2)
Probably easy if you are an expert, but for me it is real hard to see what the options are.
Thanks
You can use the following query to get the latest per group timestamp value:
select tankId, max(ts) as max_ts
from test
group by tankId;
Output:
tankId max_ts
--------------------------------
1 tank1 11.08.2017 03:51:18
2 tank2 11.08.2017 03:51:18
3 tank3 11.08.2017 03:41:18
4 tank4 11.08.2017 03:51:18
Using the above query as a derived table you can extract the latest per group fill_level value. This way you can apply the logic that computes each range level:
select sum(
CASE
WHEN t1.fill_level>=0 and t1.fill_level<40
THEN 1
ELSE 0
END) AS 'Range A',
sum(
CASE
WHEN t1.fill_level>=40 and t1.fill_level<=79
THEN 1
ELSE 0
END) AS 'Range B',
sum(
CASE
WHEN t1.fill_level>79 and t1.fill_level<=100
THEN 1
ELSE 0
END) AS 'Range C'
from test as t1
join (
select tankId, max(ts) as max_ts
from test
group by tankId
) as t2 on t1.tankId = t2.tankId and t1.ts = t2.max_ts
Output:
Range A Range B Range C
---------------------------
1 0 2 2
Demo here
I get a different result (oh, well, same result as GB):
SELECT GROUP_CONCAT(CASE WHEN fill_level < 40 THEN x.tankid END) range_a
, GROUP_CONCAT(CASE WHEN fill_level BETWEEN 40 AND 75 THEN x.tankid END) range_b
, GROUP_CONCAT(CASE WHEN fill_level > 75 THEN x.tankid END) range_c
FROM test x
JOIN (SELECT tankid,MAX(ts) ts FROM test GROUP BY tankid) y
ON y.tankid = x.tankid AND y.ts = x.ts;
+---------+-------------+-------------+
| range_a | range_b | range_c |
+---------+-------------+-------------+
| NULL | tank3,tank4 | tank1,tank2 |
+---------+-------------+-------------+
EDIT:
If I was solving this problem, and wanted to include the tank names in the result, then I'd probably execute the following...
SELECT x.*
FROM test x
JOIN
( SELECT tankid,MAX(ts) ts FROM test GROUP BY tankid) y
ON y.tankid = x.tankid
AND y.ts = x.ts
...and handle all the other problems, concerning counts, ranges, and missing/'0' values in application code.

MYSQL Calculate average of a specific occurance in a column

I need to calculate the average of occurrences in a dataset for a given value in a column. I made an easy example but in my current database contains around 2 inner joins to reduce it to 100k records. I need to perform the following select distinct statement for 10 columns.
My current design forces an inner join for each column. Another constraint is that I need to perform it at least 50-100 rows for each name in this example.
I need to figure out an efficient way to calculate this values without using too many resources while making the query fast.
http://sqlfiddle.com/#!9/c2378/3
My expected Result is:
Name | R Avg dir | L Avg dir 1 | L Avg dir 2 | L Avg dir 3
A 0 .5 .25 .25
Create table query:
CREATE TABLE demo
(`id` int, `name` varchar(10),`hand` varchar(1), `dir` int)
;
INSERT INTO demo
(`id`, `name`, `hand`, `dir`)
VALUES
(1, 'A', 'L', 1),
(2, 'A', 'L', 1),
(3, 'A', 'L', 2),
(4, 'A', 'L', 3),
(5, 'A', 'R', 3),
(6, 'A', 'R', 3)
;
Example Query:
SELECT distinct name,
COALESCE(( (Select count(id) as 'cd' from demo where hand = 'L' AND dir = 1) /(Select count(id) as 'fd' from demo where hand = 'L')),0) as 'L AVG dir'
FROM
demo
where hand = 'L' AND dir = 1 AND name = 'A'
One option is to use conditional aggregation:
SELECT name,
count(case when hand = 'L' and dir = 1 then 1 end) /
count(case when hand = 'L' then 1 end) L1Avg,
count(case when hand = 'L' and dir = 2 then 1 end) /
count(case when hand = 'L' then 1 end) L2Avg,
count(case when hand = 'L' and dir = 3 then 1 end) /
count(case when hand = 'L' then 1 end) L3Avg,
count(case when hand = 'R' and dir = 3 then 1 end) /
count(case when hand = 'R' then 1 end) RAvg
FROM demo
WHERE name = 'A'
GROUP BY name
Updated Fiddle Demo
Please note, I wasn't 100% sure why you wanted your RAvg to be 0 -- I assumed you meant 100%. If not, you can adjust the above accordingly.

How to split SQL query results into columns based on two WHERE conditions and two calculated COUNT fields?

I have the following (simplified) database schema:
Persons:
[Id] [Name]
-------------------
1 'Peter'
2 'John'
3 'Anna'
Items:
[Id] [ItemName] [ItemStatus]
-------------------
10 'Cake' 1
20 'Dog' 2
ItemDocuments:
[Id] [ItemId] [DocumentName] [Date]
-------------------
101 10 'CakeDocument1' '2016-01-01 00:00:00'
201 20 'DogDocument1' '2016-02-02 00:00:00'
301 10 'CakeDocument2' '2016-03-03 00:00:00'
401 20 'DogDocument2' '2016-04-04 00:00:00'
DocumentProcessors:
[PersonId] [DocumentId]
-------------------
1 101
1 201
2 301
I have also set up an SQL fiddle to play with: http://www.sqlfiddle.com/#!3/e6082
The relation logic is the following: every Person can work on zero or infinite number of ItemDocuments (many-to-many); each ItemDocument belongs to exactly one Item (one-to-many). Item has status 1 - Active, 2 - Closed
What I need is a report that fulfills the following requirements:
for each person in Persons table, display count of Items that have ItemDocuments related to this person
the counts should be split in two columns by ItemStatus
the query should be filterable by two optional date periods (using two BETWEEN conditions on ItemDocuments.Date field) and the Item counts should also be split into two periods
if a Person does not have any ItemDocuments assigned, it still should be shown in the results with all count values set to 0
if a Person has more than one ItemDocument for an Item, the Item still should be counted only once
Essentially, here is how the results should look like if I use both periods to NULL (to read all the data):
[PersonName] [Active Items for period 1] [Closed Items for period 1] [Active Items for period 2] [Closed Items for period 2]
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
'Peter' 1 1 1 1
'John' 1 0 1 0
'Anna' 0 0 0 0
While I can create an SQL query for each requirement separately, I have a problem to understand how to combine all of them together into one.
For example, I can split ItemStatus counts in two columns using
COUNT(CASE WHEN t.ItemStatus = 1 THEN 1 ELSE NULL END) AS Active,
COUNT(CASE WHEN t.ItemStatus = 2 THEN 1 ELSE NULL END) AS Closed
and I can filter by two periods (with max/min date constants from MS SQL server specification to avoid NULLs for optional period dates) using
between coalesce(#start1, '1753-01-01') and coalesce(#end1, '9999-12-31')
between coalesce(#start2, '1753-01-01') and coalesce(#end2, '9999-12-31')
but how to combine all of this together, considering also JOINs between tables?
Is there any technique, join or MS SQL Server specific approach to do this in efficient way?
My first attempt seems to work as required but it looks like ugly subquery duplications multiple times:
DECLARE #start1 DATETIME, #start2 DATETIME, #end1 DATETIME, #end2 DATETIME
-- SET #start2 = '2017-01-01'
SELECT
p.Name,
(SELECT COUNT(1)
FROM Items i
WHERE i.ItemStatus = 1 AND EXISTS(
SELECT 1
FROM DocumentProcessors AS dcp
INNER JOIN ItemDocuments AS idc ON dcp.DocumentId = idc.Id
WHERE dcp.PersonId = p.Id AND idc.ItemId = i.Id
AND idc.Date BETWEEN COALESCE(#start1, '1753-01-01') AND COALESCE(#end1, '9999-12-31')
)
) AS Active1,
(SELECT COUNT(*)
FROM Items i
WHERE i.ItemStatus = 2 AND EXISTS(
SELECT 1
FROM DocumentProcessors AS dcp
INNER JOIN ItemDocuments AS idc ON dcp.DocumentId = idc.Id
WHERE dcp.PersonId = p.Id AND idc.ItemId = i.Id
AND idc.Date BETWEEN COALESCE(#start1, '1753-01-01') AND COALESCE(#end1, '9999-12-31')
)
) AS Closed1,
(SELECT COUNT(1)
FROM Items i
WHERE i.ItemStatus = 1 AND EXISTS(
SELECT 1
FROM DocumentProcessors AS dcp
INNER JOIN ItemDocuments AS idc ON dcp.DocumentId = idc.Id
WHERE dcp.PersonId = p.Id AND idc.ItemId = i.Id
AND idc.Date BETWEEN COALESCE(#start2, '1753-01-01') AND COALESCE(#end2, '9999-12-31')
)
) AS Active2,
(SELECT COUNT(*)
FROM Items i
WHERE i.ItemStatus = 2 AND EXISTS(
SELECT 1
FROM DocumentProcessors AS dcp
INNER JOIN ItemDocuments AS idc ON dcp.DocumentId = idc.Id
WHERE dcp.PersonId = p.Id AND idc.ItemId = i.Id
AND idc.Date BETWEEN COALESCE(#start2, '1753-01-01') AND COALESCE(#end2, '9999-12-31')
)
) AS Closed2
FROM Persons p
I'm not absolutely sure if I really got what you want, but you might try this
WITH AllData AS
(
SELECT p.Id AS PersonId
,p.Name AS Person
,id.Date AS DocDate
,id.DocumentName AS DocName
,i.ItemName AS ItemName
,i.ItemStatus AS ItemStatus
,CASE WHEN id.Date BETWEEN COALESCE(#start1, '1753-01-01') AND COALESCE(#end1, '9999-12-31') THEN 1 ELSE 0 END AS InPeriod1
,CASE WHEN id.Date BETWEEN COALESCE(#start2, '1753-01-01') AND COALESCE(#end2, '9999-12-31') THEN 1 ELSE 0 END AS InPeriod2
FROM Persons AS p
LEFT JOIN DocumentProcessors AS dp ON p.Id=dp.PersonId
LEFT JOIN ItemDocuments AS id ON dp.DocumentId=id.Id
LEFT JOIN Items AS i ON id.ItemId=i.Id
)
SELECT PersonID
,Person
,COUNT(CASE WHEN ItemStatus = 1 AND InPeriod1 = 1 THEN 1 ELSE NULL END) AS ActiveIn1
,COUNT(CASE WHEN ItemStatus = 2 AND InPeriod1 = 1 THEN 1 ELSE NULL END) AS ClosedIn1
,COUNT(CASE WHEN ItemStatus = 1 AND InPeriod2 = 1 THEN 1 ELSE NULL END) AS ActiveIn2
,COUNT(CASE WHEN ItemStatus = 2 AND InPeriod2 = 1 THEN 1 ELSE NULL END) AS ClosedIn2
FROM AllData
GROUP BY PersonID,Person

Why MySQL full outer join returns nulls?

Why MySQL full outer join returns nulls?
Hi
I have the following data:
s_id,date,p_id,amount_sold
1, '2015-10-01', 1, 10
2, '2015-10-01', 2, 12
7, '2015-10-01', 1, 11
3, '2015-10-02', 1, 11
4, '2015-10-02', 2, 10
5, '2015-10-15', 1, 22
6, '2015-10-16', 2, 20
8, '2015-10-22', 3, 444
and i want my query to output something like this: (A = sum of amount_sold for p_id=1 for that date,B = sum of amount_sold for p_id=2 for that date)
date,A,B,Difference
'2015-10-01',21,12,9
'2015-10-02',11,10,1
'2015-10-15',22,0,22
'2015-10-01',0,20,-20
I tried with this query, but the order its returning is having NULLS and the output is wrong:
SELECT A.p_id,A.date,sum(A.amount_sold) A,B.Bs, (sum(A.amount_sold) - B.Bs) as difference FROM sales as A
LEFT JOIN (
SELECT SUM( amount_sold ) Bs,p_id,s_id, DATE
FROM sales
WHERE p_id =2
group by date
) as B ON A.s_id = B.s_id
where A.p_id=1 or B.p_id=2
group by A.date, A.p_id
UNION
SELECT A.p_id,A.date,sum(A.amount_sold) A,B.Bs, (sum(A.amount_sold) - B.Bs) as difference FROM sales as A
RIGHT JOIN (
SELECT SUM( amount_sold ) Bs,p_id,s_id, DATE
FROM sales
WHERE p_id =2
group by date
) as B ON A.s_id = B.s_id
where B.p_id=2
group by A.date, A.p_id
It returned:
p_id date A Bs difference
1 2015-10-01 21 NULL NULL
2 2015-10-01 12 12 0
1 2015-10-02 11 NULL NULL
2 2015-10-02 10 10 0
1 2015-10-15 22 NULL NULL
2 2015-10-16 20 20 0
What am i doing wrong here? and what is the correct way of doing it? any help would be appreciated.
A full join isn't needed. You can use conditional aggregation instead:
select
date,
sum(case when p_id = 1 then amount_sold else 0 end) a,
sum(case when p_id = 2 then amount_sold else 0 end) b,
sum(case when p_id = 1 then amount_sold else 0 end)
- sum(case when p_id = 2 then amount_sold else 0 end) difference
from sales
where p_id in (1,2)
group by date