MySQL : Select number of unique days between two dates - mysql

I have a table that shows events that took place during several days in a spatial grid and I want to select the number of unique days for each cell of the grid in order to obtain the number of days where an event happend, here my table structure :
+-----+------------+------------+---------+---------+
| id | start_date | end_date | id_cell | event |
+-----+------------+------------+---------+---------+
| 1 | 2017-03-01 | 2017-03-04 | 250 | envent1 |
| 2 | 2017-03-01 | 2017-03-04 | 251 | envent1 |
| 3 | 2017-03-01 | 2017-03-04 | 307 | envent1 |
| 4 | 2017-03-01 | 2017-03-04 | 308 | envent1 |
| 5 | 2017-03-01 | 2017-03-09 | 250 | event2 |
| 9 | 2017-02-24 | 2017-03-03 | 250 | event3 |
| 13 | 2017-02-24 | 2017-03-24 | 250 | event4 |
| 17 | 2017-02-24 | 2017-03-02 | 250 | event5 |
| 21 | 2017-01-04 | 2017-01-25 | 250 | event6 |
| 25 | 2017-03-26 | 2017-03-28 | 250 | event2 |
+-----+------------+------------+---------+---------+
For example, the expected result for the cell with 250 as id is 51 days :
2017-01-04 -> 2017-01-25 = 21 days
2017-02-24 -> 2017-03-24 = 28 days
2017-03-26 -> 2017-03-28 = 2 days
The other dates are all included between 2017-02-24 and 2017-03-24 so they don't have to be counted so 21 + 28 + 2 = 51 days.
I tried to use DATEDIFF() like this :
select datediff(max(end_date) , min(start_date) ) from cell_date where id_cell = 250
The result is 83 because it counts the number of days between 2017-01-25 and 2017-03-01, the days where no event happened.
I tried some requests with DATEDIFF but I couldn't figure out how to do it. Someone can help me please ? Thanks in advance.

You can achieve this by grouping by cell_id and calculating the sum of the individual differences:
Select cell_id,[other columns],Sum(datediff(days,start_date,end_date)) as Days
From my_table
group by cell_id,[other columns]
Edit:
For your need I think you should use an intermediate table to store individual days(sure it's not the best way to do it), join them with your events and then select the distinct days in your result. Here is the code to achieve it
/*Your example Talbe*/
DECLARE #T
TABLE(ID INT,startDate DATE,EndDate DATE,id_cell INT,evnt NVARCHAR(20) )
INSERT INTO #T
VALUES
(1,'2017-03-01','2017-03-04',250,'event1'),
(2,'2017-03-01','2017-03-04',251,'event1'),
(3,'2017-03-01','2017-03-04',307,'event1'),
(4,'2017-03-01','2017-03-04',308,'event1'),
(5,'2017-03-01','2017-03-09',250,'event2'),
(9,'2017-02-24','2017-03-03',250,'event3'),
(13,'2017-02-24','2017-03-24',250,'event4'),
(17,'2017-02-24','2017-03-02',250,'event5'),
(21,'2017-01-04','2017-01-25',250,'event6'),
(25,'2017-03-26','2017-03-28',250,'event2')
/*Table to store days: ideally get the start and end dates from your table */
DECLARE #STARTDATE DATE='2017-01-04'
DECLARE #ENDDATE DATE='2017-03-28'
DECLARE #DAYS
TABLE(oneday DATE)
WHILE #StartDate <= #endDate
BEGIN
INSERT INTO #days
(
oneday
)
SELECT
#StartDate
SET #StartDate = DATEADD(dd, 1, #StartDate)
END
/*The request */
SELECT id_cell,COUNT(DISTINCT oneday) NUMBER_OF_DAYS
FROM #T t
JOIN #DAYS d ON d.oneday>=t.startDate AND d.oneday<t.EndDate
WHERE id_cell=250
GROUP BY id_cell

You could use max and min date and datediff for obtain the diff in days between the date eg: for event
select event, datediff(max(end_date) , min(star_date) )
from my_table
group by event
or yuo can sum by cell_id for the total for cell_id
select cell_id, sum(datediff(end_date , star_date) )as days
from my_table
group by cell_id
or
Looking to your sample seems you need max end_date for some event occurring starting from the same date
select id_cell, sum(datediff(max_end_date, start_date)) as days
from (
select id_cell, start_date, max(end_date) as max_end_date
from cell_date
group by id_cell, start_date ) t
group by id_cell

Related

GROUP BY custom date intervals per year

Situation: I need a custom interval between dates. The problem I face when I try to GROUP BY the year and the result I get amounts to by the given year. I need a custom interval per year from December 20th with time: 00:00:00 of previous year to December 19th with time: 23:59:59 of said year. Here is some of my data:
Table - History:
id | date | income | spent
--------------------------------------------
1 | 2019-12-21 17:15:00 | 600,00 | NULL
2 | 2019-12-23 12:55:00 | 183,00 | NULL
3 | 2019-12-30 20:05:00 | NULL | 25,00
4 | 2020-01-01 15:35:00 | NULL | 13,00
5 | 2020-01-01 20:25:00 | NULL | 500,50
6 | 2020-12-10 10:25:00 | NULL | 5,50
7 | 2021-05-22 12:45:00 | 1098,00 | NULL
8 | 2021-05-23 10:18:00 | NULL | 186,00
9 | 2021-11-25 12:32:00 | NULL | 10,00
10 | 2021-12-23 10:35:00 | NULL | 10,00
The expected result:
Year | Summary Income | Summary Spent | Difference
--------------------------------------------------
2020 | 783,00 | 544,00 | 239,50
2021 | 1098,00 | 196,00 | 902,00
2022 | 0,00 | 10,00 | -10,00
I have managed to get a result with the help of a loop within a procedure:
...
SET #Aa = (SELECT MIN(date) FROM History);
CREATE TEMPORARY TABLE Yr (Year VARCHAR(4), Income FLOAT(8,2), Spent FLOAT(8,2), differ FLOAT(8,2));
Yearly: LOOP
SET #Aa = #Aa + 1;
SET #From = CONCAT((#Aa - 1), '-12-20 00:00:00');
SET #To = CONCAT(#Aa, '-12-19 23:59:59');
SET #Count = (SELECT SUM(income) FROM History WHERE date >= #From AND date <= #To);
SET #diff = (SELECT SUM(spent) FROM History WHERE date >= #From AND date <= #To);
INSERT INTO Yr (Year, Income, Spent, differ) VALUES (#Aa, #Count, #diff, (#Count - #diff));
IF (#Aa = (SELECT MAX(YEAR(date)) FROM History)) THEN LEAVE Yearly; END IF;
END LOOP;
SELECT * FROM Yr;
...
Question: I wonder if it's possible to get a custom interval for an annual summary with an condensed SQL query without using a loop?
You can simply add 11 days to the date before applying the year function to get this grouping, e.g.
SELECT YEAR(DATE_ADD(date, INTERVAL 11 DAY)) AS Year,
SUM(income) AS income,
SUM(spent) AS Spent,
IFNULL(SUM(income),0) - IFNULL(SUM(spent),0) AS difference
FROM History
GROUP BY YEAR(DATE_ADD(date, INTERVAL 11 DAY));
Example on db-fiddle

How to sum non-consecutive values when using group by in MySQL

I have a data set representing alarms' state at a given timestamp (every 15 minutes). When the value is 1 the alarm is ON, 0 when OFF. I am trying to count the number of times the alarm has been triggered per hour (non-consecutive 1).
I took a look at Count max number of consecutive occurrences of a value in SQL Server but couldn't manage to adapt the answer.
Basically the data set for one alarm looks like this:
| id | value | registered_at |
| -- | ---------|---------------------|
| 1 | 1 | 2012-07-15 06:00 |
| 2 | 0 | 2012-07-15 06:15 |
| 3 | 1 | 2012-07-15 06:30 |
| 4 | 0 | 2012-07-15 06:45 |
| 5 | 1 | 2012-07-15 07:00 |
| 6 | 1 | 2012-07-15 07:15 |
| 7 | 1 | 2012-07-15 07:30 |
| 8 | 0 | 2012-07-15 07:45 |
| 8 | 0 | 2012-07-15 08:00 |
The results I am looking for is the following
| registered_at | alarm_triggered |
|--------------------|-----------------|
| 2012-07-15 06 | 2 |
| 2012-07-15 07 | 1 |
| 2012-07-15 08 | 0 |
To create groups I use EXTRACT(DAY_HOUR from registered_at).
Can you help me create the query?
(First time poster on SO, any feedback about the form of this post would be greatly appreciated as well)
Use LAG() window function to check the value of value of the previous row and if it is different and the current row is 1 then sum:
SELECT registered_at,
SUM(value * flag) alarm_triggered
FROM (
SELECT value,
DATE_FORMAT(registered_at, '%Y-%m-%d %H') registered_at,
value <> LAG(value, 1, 0) OVER (PARTITION BY DATE_FORMAT(registered_at, '%m-%d-%Y %H') ORDER BY registered_at) flag
FROM tablename
) t
GROUP BY registered_at
See the demo.
Results:
registered_at
alarm_triggered
2012-07-15 06
2
2012-07-15 07
1
2012-07-15 08
0
I assume the registered_at field is datetime so you need to use datetime function.
here is a query for this:
SELECT DATE_FORMAT(registered_at, "%Y-%m-%d %H:00:00") AS registered_at, SUM(VALUE) AS alarm_triggered
FROM ALARMS
GROUP BY DATE_FORMAT(registered_at, "%Y-%m-%d %H:00:00")
and sqlfiddle to see example:
example
If you need only notified days
select count(value), date_format(registered_at, '%m-%d-%Y %H') as c_at
from notifications
where value = 1
group by date_format(registered_at, '%m-%d-%Y %H');
Or all days
select sum(value), date_format(registered_at, '%m-%d-%Y %H') as c_at
from notifications
group by date_format(registered_at, '%m-%d-%Y %H');
Try this!
You can select it like this:
SELECT CONCAT(YEAR(registered_at), '-', MONTH(registered_at), '-', DAYOFMONTH(registered_at), ' ' HOUR(registered_at)), count(*)
FROM alarms
WHERE value = 1
GROUP BY YEAR(registered_at), MONT(registered_at), DAYOFMONTH(registered_at), HOUR(registered_at);
Explanation
First, we find the records whose value is 1, then group them by year, month, day of month and hour and finally we find out their count.

Sum datetime difference for values of same column and group by day

I have a table with 'ON' and 'OFF' values in column activity and another column datetime.
id(AUTOINCREMENT) id_device activity datetime
1 a ON 2017-05-26 22:00:00
2 b ON 2017-05-26 05:00:00
3 a OFF 2017-05-27 04:00:00
4 b OFF 2017-05-26 08:00:00
5 a ON 2017-05-28 12:00:00
6 a OFF 2017-05-28 15:00:00
I need to get total ON time by day
day id_device total_minutes_on
2017-05-26 a 120
2017-05-26 b 180
2017-05-27 a 240
2017-05-27 b 0
2017-05-28 a 180
2017-05-28 b 0
i have searched and tried answers for another posts, i tried TimeDifference and i get correct total time.
I don't find the way to get total time grouped by date
i appreciate your help
I'm not posting this as a definite answer rather it's an experiment for me and hopefully you'll find is useful in your case. Also I would like to mention that the MySQL database version I'm working with is quite old so the method I'm using is also very manual to say the least.
First of all lets extract your expected output:
The date value in day need to be repeated twice fro each of id_device a and b.
Minutes are calculated based on the activity; if activity is 'ON' until tomorrow, it needs to be calculated until the day end at 24:00:00 while the next day will calculate minutes until the activity is OFF.
What I come up with is this:
Creating condition (1):
SELECT * FROM
(SELECT DATE(datetime) dtt FROM mytable GROUP BY DATE(datetime)) a,
(SELECT id_device FROM mytable GROUP BY id_device) b
ORDER BY dtt,id_device;
The query above will return the following result:
+------------+-----------+
| dtt | id_device |
+------------+-----------+
| 2017-05-26 | a |
| 2017-05-26 | b |
| 2017-05-27 | a |
| 2017-05-27 | b |
| 2017-05-28 | a |
| 2017-05-28 | b |
+------------+-----------+
*Above will only work with all the dates you have in the table. If you want all date regardless if there's activity or not, I suggest you create a calendar table (refer: Generating a series of dates).
So this become the base query. Then I've added an outer query to left join the query above with the original data table:
SELECT v.*,
GROUP_CONCAT(w.activity ORDER BY w.datetime SEPARATOR ' ') activity,
GROUP_CONCAT(TIME_TO_SEC(TIME(w.datetime)) ORDER BY w.datetime SEPARATOR ' ') tr
FROM
-- this was the first query
(SELECT * FROM
(SELECT DATE(datetime) dtt FROM mytable GROUP BY DATE(datetime)) a,
(SELECT id_device FROM mytable GROUP BY id_device) b
ORDER BY a.dtt,b.id_device) v
--
LEFT JOIN
mytable w
ON v.dtt=DATE(w.datetime) AND v.id_device=w.id_device
GROUP BY DATE(v.dtt),v.id_device
What's new in the query is the addition of GROUP_CONCAT operation on both activity and time value extracted from datetime column which is converted into seconds value. You notice that in both of the GROUP_CONCAT there's a similar ORDER BY condition which is important in order to get the exact corresponding value.
The query above will return the following result:
+------------+-----------+----------+-------------+
| dtt | id_device | activity | tr |
+------------+-----------+----------+-------------+
| 2017-05-26 | a | ON | 79200 |
| 2017-05-26 | b | ON OFF | 18000 28800 |
| 2017-05-27 | a | OFF | 14400 |
| 2017-05-27 | b | (NULL) | (NULL) |
| 2017-05-28 | a | ON OFF | 43200 54000 |
| 2017-05-28 | b | (NULL) | (NULL) |
+------------+-----------+----------+-------------+
From here, I've added another query outside to calculate how many minutes and attempt to get the expected result:
SELECT dtt,id_device,
CASE
WHEN SUBSTRING_INDEX(activity,' ',1)='ON' AND SUBSTRING_INDEX(activity,' ',-1)='OFF'
THEN (SUBSTRING_INDEX(tr,' ',-1)-SUBSTRING_INDEX(tr,' ',1))/60
WHEN activity='ON' THEN 1440-(tr/60)
WHEN activity='OFF' THEN tr/60
WHEN activity IS NULL AND tr IS NULL THEN 0
END AS 'total_minutes_on'
FROM
-- from the last query
(SELECT v.*,
GROUP_CONCAT(w.activity ORDER BY w.datetime SEPARATOR ' ') activity,
GROUP_CONCAT(TIME_TO_SEC(TIME(w.datetime)) ORDER BY w.datetime SEPARATOR ' ') tr
FROM
-- this was the first query
(SELECT * FROM
(SELECT DATE(datetime) dtt FROM mytable GROUP BY DATE(datetime)) a,
(SELECT id_device FROM mytable GROUP BY id_device) b
ORDER BY a.dtt,b.id_device) v
--
LEFT JOIN
mytable w
ON v.dtt=DATE(w.datetime) AND v.id_device=w.id_device
GROUP BY DATE(v.dtt),v.id_device
--
) z
The last part I do is if the activity value have both ON and OFF on the same day then (OFF-ON)/60secs=total minutes. If activity value is only ON then minutes value for '24:00:00' > 24 hr*60 min= 1440-(ON/60secs)= total minutes, and if activity only OFF, I just convert seconds to minutes because the day starts at 00:00:00 anyhow.
+------------+-----------+------------------+
| dtt | id_device | total_minutes_on |
+------------+-----------+------------------+
| 2017-05-26 | a | 120 |
| 2017-05-26 | b | 180 |
| 2017-05-27 | a | 240 |
| 2017-05-27 | b | 0 |
| 2017-05-28 | a | 180 |
| 2017-05-28 | b | 0 |
+------------+-----------+------------------+
Hopefully this will give you some ideas. ;)

UPDATE + SET + WHERE - Dynamic minimum value

This is a follow-up to:
Dynamic minimum value for specfic range (mysql)
I do have the query to fetch the third column (lowest of the last 3 days) "Low_3_days" via SELECT command:
-----------------------------------------
| Date | Unit_ | Lowest_in_last_|
| | price | 3_days |
|----------------------------------------
| 2015-01-01 | 15 | 15 |
| 2015-01-02 | 17 | 15 |
| 2015-01-03 | 21 | 15 |
| 2015-01-04 | 18 | 17 |
| 2015-01-05 | 12 | 12 |
| 2015-01-06 | 14 | 12 |
| 2015-01-07 | 16 | 12 |
|----------------------------------------
select S.Date,Unit_price,
(select S.Date, Unit_price,
(SELECT min(s2.Unit_Price)
FROM table s2
WHERE s2.DATE BETWEEN s.DATE - interval 3 day and
s.DATE - interval 1 day
) as min_price_3_days
FROM table S;
My new challenge is - what is the best way to use UPDATE-SET-WHERE so I could add the ("Lowest_in_last_3_days") values to a new column in a table (instead of having temporary results displayed to me via SELECT).
By following the UPDATE-SET-WHERE syntax, the query would be:
UPDATE table
SET min_price_3_days =
(select S.Date, Unit_price,
(SELECT min(s2.Unit_Price)
FROM table s2
WHERE s2.DATE BETWEEN s.DATE - interval 3 day and
s.DATE - interval 1 day
) as min_price_3_days
but I have difficulties constructing the correct query.
What would be the correct approach to this? I do recognize this one is a tough one to solve.
Your UPDATE should look like:
update table set low_3_days=
(SELECT min(Unit_Price)
FROM (select unit_price, date as date2 from table) as s2
WHERE s2.date2 BETWEEN date - interval 3 day and date - interval 1 day
);
You can check it in SQLFiddle
In Fiddle I used different names for table and column. I prefer not to use SQL keywords as names

Count columns according to dates in SQL

I need help with a SQL statement. The goal is to count the amount of alarms of each date. My table looks something like this:
|----DATE----|---COUNTER---|---ALARM_ID---|
|2012-01-01 | 30 | 1 |
|2012-01-01 | 20 | 2 |
|2012-01-01 | 10 | 3 |
|2012-01-02 | 5 | 1 |
|2012-01-02 | 25 | 2 |
|2012-01-02 | 12 | 3 |
|2012-01-03 | 33 | 1 |
|2012-01-03 | 43 | 2 |
|2012-01-03 | 11 | 3 |
And I'm looking for a SQL statement that gives this result:
|----DATE----|---COUNTER---|
|2012-01-01 | 60 |
|2012-01-02 | 42 |
|2012-01-03 | 87 |
I've been working on this SELECT date, SUM(counter) FROM all_stats but all I get is:
|----DATE----|---COUNTER---|
|2012-01-01 | 60 |
Do I have to create a loop to go through all dates and count?
Thanks in advance, Steve-O
SELECT date, SUM(counter)
FROM all_stats
GROUP BY date
Try this instead
SELECT date, SUM(counter) FROM all_stats GROUP BY date;
"GROUP BY date" puts all the individual dates on a separate line and does the sum separately per date.
select date, sum(counter)
from all_stats
group by date