How to sum non-consecutive values when using group by in MySQL - mysql

I have a data set representing alarms' state at a given timestamp (every 15 minutes). When the value is 1 the alarm is ON, 0 when OFF. I am trying to count the number of times the alarm has been triggered per hour (non-consecutive 1).
I took a look at Count max number of consecutive occurrences of a value in SQL Server but couldn't manage to adapt the answer.
Basically the data set for one alarm looks like this:
| id | value | registered_at |
| -- | ---------|---------------------|
| 1 | 1 | 2012-07-15 06:00 |
| 2 | 0 | 2012-07-15 06:15 |
| 3 | 1 | 2012-07-15 06:30 |
| 4 | 0 | 2012-07-15 06:45 |
| 5 | 1 | 2012-07-15 07:00 |
| 6 | 1 | 2012-07-15 07:15 |
| 7 | 1 | 2012-07-15 07:30 |
| 8 | 0 | 2012-07-15 07:45 |
| 8 | 0 | 2012-07-15 08:00 |
The results I am looking for is the following
| registered_at | alarm_triggered |
|--------------------|-----------------|
| 2012-07-15 06 | 2 |
| 2012-07-15 07 | 1 |
| 2012-07-15 08 | 0 |
To create groups I use EXTRACT(DAY_HOUR from registered_at).
Can you help me create the query?
(First time poster on SO, any feedback about the form of this post would be greatly appreciated as well)

Use LAG() window function to check the value of value of the previous row and if it is different and the current row is 1 then sum:
SELECT registered_at,
SUM(value * flag) alarm_triggered
FROM (
SELECT value,
DATE_FORMAT(registered_at, '%Y-%m-%d %H') registered_at,
value <> LAG(value, 1, 0) OVER (PARTITION BY DATE_FORMAT(registered_at, '%m-%d-%Y %H') ORDER BY registered_at) flag
FROM tablename
) t
GROUP BY registered_at
See the demo.
Results:
registered_at
alarm_triggered
2012-07-15 06
2
2012-07-15 07
1
2012-07-15 08
0

I assume the registered_at field is datetime so you need to use datetime function.
here is a query for this:
SELECT DATE_FORMAT(registered_at, "%Y-%m-%d %H:00:00") AS registered_at, SUM(VALUE) AS alarm_triggered
FROM ALARMS
GROUP BY DATE_FORMAT(registered_at, "%Y-%m-%d %H:00:00")
and sqlfiddle to see example:
example

If you need only notified days
select count(value), date_format(registered_at, '%m-%d-%Y %H') as c_at
from notifications
where value = 1
group by date_format(registered_at, '%m-%d-%Y %H');
Or all days
select sum(value), date_format(registered_at, '%m-%d-%Y %H') as c_at
from notifications
group by date_format(registered_at, '%m-%d-%Y %H');
Try this!

You can select it like this:
SELECT CONCAT(YEAR(registered_at), '-', MONTH(registered_at), '-', DAYOFMONTH(registered_at), ' ' HOUR(registered_at)), count(*)
FROM alarms
WHERE value = 1
GROUP BY YEAR(registered_at), MONT(registered_at), DAYOFMONTH(registered_at), HOUR(registered_at);
Explanation
First, we find the records whose value is 1, then group them by year, month, day of month and hour and finally we find out their count.

Related

GROUP BY custom date intervals per year

Situation: I need a custom interval between dates. The problem I face when I try to GROUP BY the year and the result I get amounts to by the given year. I need a custom interval per year from December 20th with time: 00:00:00 of previous year to December 19th with time: 23:59:59 of said year. Here is some of my data:
Table - History:
id | date | income | spent
--------------------------------------------
1 | 2019-12-21 17:15:00 | 600,00 | NULL
2 | 2019-12-23 12:55:00 | 183,00 | NULL
3 | 2019-12-30 20:05:00 | NULL | 25,00
4 | 2020-01-01 15:35:00 | NULL | 13,00
5 | 2020-01-01 20:25:00 | NULL | 500,50
6 | 2020-12-10 10:25:00 | NULL | 5,50
7 | 2021-05-22 12:45:00 | 1098,00 | NULL
8 | 2021-05-23 10:18:00 | NULL | 186,00
9 | 2021-11-25 12:32:00 | NULL | 10,00
10 | 2021-12-23 10:35:00 | NULL | 10,00
The expected result:
Year | Summary Income | Summary Spent | Difference
--------------------------------------------------
2020 | 783,00 | 544,00 | 239,50
2021 | 1098,00 | 196,00 | 902,00
2022 | 0,00 | 10,00 | -10,00
I have managed to get a result with the help of a loop within a procedure:
...
SET #Aa = (SELECT MIN(date) FROM History);
CREATE TEMPORARY TABLE Yr (Year VARCHAR(4), Income FLOAT(8,2), Spent FLOAT(8,2), differ FLOAT(8,2));
Yearly: LOOP
SET #Aa = #Aa + 1;
SET #From = CONCAT((#Aa - 1), '-12-20 00:00:00');
SET #To = CONCAT(#Aa, '-12-19 23:59:59');
SET #Count = (SELECT SUM(income) FROM History WHERE date >= #From AND date <= #To);
SET #diff = (SELECT SUM(spent) FROM History WHERE date >= #From AND date <= #To);
INSERT INTO Yr (Year, Income, Spent, differ) VALUES (#Aa, #Count, #diff, (#Count - #diff));
IF (#Aa = (SELECT MAX(YEAR(date)) FROM History)) THEN LEAVE Yearly; END IF;
END LOOP;
SELECT * FROM Yr;
...
Question: I wonder if it's possible to get a custom interval for an annual summary with an condensed SQL query without using a loop?
You can simply add 11 days to the date before applying the year function to get this grouping, e.g.
SELECT YEAR(DATE_ADD(date, INTERVAL 11 DAY)) AS Year,
SUM(income) AS income,
SUM(spent) AS Spent,
IFNULL(SUM(income),0) - IFNULL(SUM(spent),0) AS difference
FROM History
GROUP BY YEAR(DATE_ADD(date, INTERVAL 11 DAY));
Example on db-fiddle

Group Rows in group by though it contains NULL value in mysql / postgres

I have a table from where I am getting month names and some quantity measures.
Table Name = Month_Name
SELECT month_name,q1,q2 FROM month_name;
mysql> SELECT * FROM MONTH;
+------------+------+------+
| month_name | q1 | q2 |
+------------+------+------+
| January | 10 | 20 |
| March | 30 | 40 |
| March | 10 | 5 |
+------------+------+------+
Expected Output:
mysql> SELECT month_name ,SUM(q1),SUM(q2) FROM MONTH GROUP BY month_name;
+------------+---------+---------+
| month_name | sum(q1) | sum(q2) |
+------------+---------+---------+
| January | 10 | 20 |
| Febuary | 0 | 0 |
| March | 40 | 45 |
| April | 0 | 0 |
+------------+---------+---------+
Group by month will not print February and April since these 2 months are not present in base table. I do not want to use Union All since there will be performance issues with union All, Is there any other optimised approach to this.
You can use a calendar table which keeps track of all the month names which you want to appear in your report.
SELECT
m1.month_name,
SUM(q1) AS q1_sum,
SUM(q2) AS q2_sum
FROM
(
SELECT 'January' AS month_name UNION ALL
SELECT 'February' UNION ALL
SELECT 'March' UNION ALL
...
SELECT 'December'
) m1
LEFT JOIN month m2
ON m1.month_name = m2.month_name
GROUP BY
m1.month_name;
Note that while this solve your immediate problem, it is still not ideal, because we don't have any easy way to sort the months. A much better table design would be to maintain a date column. The month name is easily derived from the date.

mysql sum group by month and date using a contract start and end date

I have a table full of monthly contracts. There is a monthly price, a start date, and an end date for each. I am trying to graph each month's total revenue and am wondering if it's possible to do this in one query (vs. a query for each month).
I know how to group by month and year in mysql, but this requires a more complex solution that "understands" whether to include in the sum for a given month/year based on the start and end date of the contract.
Shorthand example
| contract_id | price | start_date | end_date |
| 1 | 299 | 1546318800 (1/1/19) | 1554004800 (3/31/19) |
| 2 | 799 | 1551416400 (3/1/19) | 1559275200 (5/31/19) |
With this example, there's an overlap in March. Both contracts are running in March, so the sum returned for that month should be 1098.
I'd like to be able to produce a report that includes every month between two dates, so in this case I'd send 1/1/19 - 12/31/19, the full year of 2019 and would hope to see 0 results as well.
| month | year | price_sum |
| 1 | 2019 | 299 |
| 2 | 2019 | 299 |
| 3 | 2019 | 1098 |
| 4 | 2019 | 799 |
| 5 | 2019 | 799 |
| 6 | 2019 | 0 |
| 7 | 2019 | 0 |
| 8 | 2019 | 0 |
| 9 | 2019 | 0 |
| 10 | 2019 | 0 |
| 11 | 2019 | 0 |
| 12 | 2019 | 0 |
Here is a full working script for your problem, which uses a calendar table approach to represent every month in 2019. Specifically, we represent each month using the first of that month. Then, a given price from your table is applicable to that month if there is overlap with the start and end range.
WITH yourTable AS (
SELECT 1 AS contract_id, 299 AS price, '2019-01-01' AS start_date, '2019-03-31' AS end_date UNION ALL
SELECT 2, 799, '2019-03-01', '2019-05-31'
),
dates AS (
SELECT '2019-01-01' AS dt UNION ALL
SELECT '2019-02-01' UNION ALL
SELECT '2019-03-01' UNION ALL
SELECT '2019-04-01' UNION ALL
SELECT '2019-05-01' UNION ALL
SELECT '2019-06-01' UNION ALL
SELECT '2019-07-01' UNION ALL
SELECT '2019-08-01' UNION ALL
SELECT '2019-09-01' UNION ALL
SELECT '2019-10-01' UNION ALL
SELECT '2019-11-01' UNION ALL
SELECT '2019-12-01'
)
SELECT
d.dt,
SUM(t.price) AS price_sum
FROM dates d
LEFT JOIN yourTable t
ON d.dt < t.end_date
AND DATE_ADD(d.dt, INTERVAL 1 MONTH) > t.start_date
GROUP BY
d.dt;
Demo
Notes:
If your dates are actually stored as UNIX timestamps, then just call FROM_UNIXTIME(your_date) to convert them to dates, and use the same approach I gave above.
I had to use the overlapping date range formula here, because the criteria for overlap in a given month is that the range of that month intersects the range given by a start and end date. Have a look at this SO question for more information on that.
My code is for MySQL 8+, though in practice you may wish to create a bona fide calendar table (the CTE version of which I called dates above), which contains the range of months/years which you want to cover your data set.
I understand that you will be given a range of dates for which you will need to report. My solution requires you to initialize a temporary table, such as date_table with the first day of each month for which you want to report on:
create temporary table date_table (
d date,
primary key(d)
);
set #start_date = '2019-01-01';
set #end_date = '2019-12-01';
set #months = -1;
insert into date_table(d)
select DATE_FORMAT(date_range,'%Y-%c-%d') AS result_date from (
select (date_add(#start_date, INTERVAL (#months := #months +1 ) month)) as date_range
from mysql.help_topic a limit 0,1000) a
where a.date_range between #start_date and last_day(#end_date);
Then this should do it:
select month(dt.d) as month, year(dt.d) as year, ifnull(sum(c.price), 0) as price_sum
from date_table dt left join contract c on
dt.d >= date(from_unixtime(c.start_date)) and dt.d < date(from_unixtime(c.end_date))
group by dt.d
order by dt.d
;
Resulting in:
+-------+------+-----------+
| month | year | price_sum |
+-------+------+-----------+
| 1 | 2019 | 299 |
| 2 | 2019 | 299 |
| 3 | 2019 | 1098 |
| 4 | 2019 | 799 |
| 5 | 2019 | 799 |
| 6 | 2019 | 0 |
| 7 | 2019 | 0 |
| 8 | 2019 | 0 |
| 9 | 2019 | 0 |
| 10 | 2019 | 0 |
| 11 | 2019 | 0 |
| 12 | 2019 | 0 |
+-------+------+-----------+
See demo
I am not sure about the semantics of the column end_date. Right now I am comparing the first a follows: start_date <= first_of_month < end_date. Perhaps the test should be start_date <= first_of_month <= end_date, in which case:
dt.d >= date(from_unixtime(c.start_date)) and dt.d < date(from_unixtime(c.end_date))
becomes:
dt.d between date(from_unixtime(c.start_date)) and date(from_unixtime(c.end_date))
With end_date being the last day of the month, it would not matter either way.

sql query to count twitter comments by month in 2016

I want to list the number of Tweets made according to month in 2016. I am new to SQL but have tried different ways to do this. Below is my latest attempt. I keep getting a message that I am not using datelogged properly. Lastly, I am not sure of how to format Total and Tweet_Cnt.
The format of the date in Twitter is as follows: MESSAGE_POSTED_TIME: 2015-08-06 21:48:34. FYI---- Column Name=MESSAGE_POSTED_TIME; Table Name=DTrumpCampaign_Tweets
Select
Year(DATELOGGED),
Sum(Case When Month(DATELOGGED) = 1 Then 1 Else 0 End) Jan,
Sum(Case When Month(DATELOGGED) = 2 Then 1 Else 0 End) Feb,
Sum(Case When Month(DATELOGGED) = 3 Then 1 Else 0 End) Mar
From
DTrumpCampaign_Tweets
Group By
Year(DATELOGGED);
I would like for my table's format to look like this
Month(2016) Tweet_Cnt
Jan 25
Feb 100
Mar 200
total 325
I greatly appreciate your help.
Thanks.
Assuming that you only want data of year 2016 only.
SELECT
(CASE WHEN t.`month` IS NULL THEN 'total'ELSE t.monthName END) AS 'Month(2016)',
t.Tweet_Cnt
FROM
(
SELECT
MONTHNAME(DATELOGGED) AS monthName,
YEAR (DATELOGGED) `year`,
MONTH (DATELOGGED) `month`,
COUNT(*) Tweet_Cnt
FROM DTrumpCampaign_Tweets
WHERE YEAR (DATELOGGED) = '2016'
GROUP BY `year`,`month` WITH ROLLUP
LIMIT 13
) t;
Demo with some sample data
You will get an output structure like below:
| Month(2016) | Tweet_Cnt |
|-------------|-----------|
| January | 1 |
| February | 2 |
| March | 1 |
| April | 1 |
| May | 1 |
| June | 1 |
| July | 1 |
| August | 1 |
| September | 1 |
| October | 1 |
| November | 1 |
| December | 1 |
| total | 13 |
More:
If you want the month names having only first three letters only then just change the corresponding line in the above query as below:
Change this line : SELECT MONTHNAME(DATELOGGED) AS monthName,
To this: SELECT DATE_FORMAT(DATELOGGED,"%b") AS monthName,
Demo of this modified query
Here is a solution, with a sqlfiddle demo:
http://sqlfiddle.com/#!9/786acb/5
SELECT
COALESCE(`Month(2016)`, 'Total') AS `Month(2016)`, Tweet_cnt
FROM
(
Select
DATE_FORMAT(datelogged, '%b') AS `Month(2016)`,
COUNT(*) AS Tweet_cnt
From dtrumpcampaign_tweets
WHERE YEAR(datelogged) = '2016'
Group BY `Month(2016)` WITH ROLLUP
) t;
The output is like:
+-------------+-----------+
| Month(2016) | Tweet_cnt |
+-------------+-----------+
| Feb | 1 |
| Jan | 2 |
| Mar | 3 |
| Total | 6 |
+-------------+-----------+
4 rows in set (0.00 sec)

Format grouped date as readable date

So I have a query that correctly displays the number of registrations for the last 12 months. Here is display:
Registrations per month for last 2 years
1--17
2--12
3--17
4--8
5--9
6--8
7--15
8--20
9--12
10--14
11--13
12--14
But since im running this in say June, the last mont I need to say the readable date May and not '1'. I want instead:
May--17
Apr--12
March--17
.
.
.
Here is my current MYSQL:
SELECT MONTH(create_date) as month , COUNT(create_date) as count
FROM `users`
WHERE create_date >= NOW() - INTERVAL 1 YEAR
GROUP BY MONTH(create_date)
I assumed I just have to use FORMAT_DATE() on the GROUP By as:
GROUP BY FORMAT_DATE(MONTH(create_date, '%M'))
And that would give me my readable month, but the sql statement reports it is not correct. Anyone know how to accomplish this?
Try this:
SELECT DATE_FORMAT(create_date, '%M') AS month, COUNT(create_date) AS count
FROM users
WHERE create_date >= NOW() - INTERVAL 1 YEAR
GROUP BY MONTH(create_date);
The result will be:
+-----------+-------+
| month | count |
+-----------+-------+
| January | 1 |
| February | 1 |
| March | 1 |
| April | 1 |
| May | 2 |
| June | 2 |
| July | 1 |
| August | 1 |
| September | 1 |
| November | 1 |
| December | 1 |
+-----------+-------+
You can use STR_TO_DATE() to convert the number to a date, and then back with MONTHNAME()
SELECT MONTHNAME(create_date(6, '%m')) as month , COUNT(create_date) as count
FROM `users`
WHERE create_date >= NOW() - INTERVAL 1 YEAR
GROUP BY MONTH(create_date)