Mysql , SUM column and GROUP BY HOUR - mysql

Situation (simplified):
tableA
id | date | val
------------------------------
0 2018-02-19 00:01:00 | 10
1 2018-02-19 00:02:00 | 10
2 2018-02-19 00:03:00 | 10
.. 2018-02-19 23:59:00 | 10
I need to do a query that return for each hour the SUM of the column val.
This is the query
SELECT `AllHours`.`hour` , COALESCE(SUM(`A`.`val`),0) AS `A`.`total`
FROM `tableA` AS `A`
RIGHT JOIN (
SELECT 0 AS `hour`
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12
UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18
UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21
UNION ALL SELECT 22 UNION ALL SELECT 23
) AS `AllHours` ON HOUR(`A`.`date`) = `AllHours`.`hour`
WHERE `A`.`date` BETWEEN '2018-02-19 00:00:00' AND '2018-02-19 23:59:59' OR `A`.`date` IS NULL
GROUP BY `AllHours`.`hour`
ORDER BY `AllHours`.`hour`
Result
The query works, but hour 11 is missing.
Important note
I need all the hour, also if there aren't data. Otherwise i wouldn't have added the RIGHT JOIN and it would have been enough a GROUP BY HOUR(date).

Consider using a numbers table - see links below. This will give you the desired output for now:
Testdata:
CREATE TABLE T (`id` int, `dt` datetime, `val` int);
INSERT INTO T (`id`, `dt`, `val`)
VALUES
(0, '2018-02-19 00:01:00', 10),
(1, '2018-02-19 00:02:00', 10),
(2, '2018-02-19 00:03:00', 10),
(4, '2018-02-19 01:01:00', 10),
(5, '2018-02-19 01:02:00', 10),
(6, '2018-02-19 02:03:00', 10)
;
Sql:
select
lpad(cast(HH as char(2)),2,'0') as hour,
sum(val) as sumVal from
(
select
EXTRACT(HOUR from dt) AS HH,
val
from T
WHERE dt >= '2018-02-19 0:0:0' and dt < '2018-02-20 0:0:0'
UNION ALL SELECT 0,0
UNION ALL SELECT 1,0 UNION ALL SELECT 2,0 UNION ALL SELECT 3,0
UNION ALL SELECT 4,0 UNION ALL SELECT 5,0 UNION ALL SELECT 6,0
UNION ALL SELECT 7,0 UNION ALL SELECT 8,0 UNION ALL SELECT 9,0
UNION ALL SELECT 10,0 UNION ALL SELECT 11,0 UNION ALL SELECT 12,0
UNION ALL SELECT 13,0 UNION ALL SELECT 14,0 UNION ALL SELECT 15,0
UNION ALL SELECT 16,0 UNION ALL SELECT 17,0 UNION ALL SELECT 18,0
UNION ALL SELECT 19,0 UNION ALL SELECT 20,0 UNION ALL SELECT 21,0
UNION ALL SELECT 22,0 UNION ALL SELECT 23,0
) as m
group by lpad(cast(HH as char(2)),2,'0')
gets you an output of:
hour sumVal
00 30
01 20
02 10
03 0
04 0
05 0
06 0
07 0
08 0
09 0
10 0
11 0
12 0
13 0
14 0
15 0
16 0
17 0
18 0
19 0
20 0
21 0
22 0
23 0
If you need this more often, create a numbers table and use the join syntax you already have. Without a numbers table you can as well union all without join.
SO-Readups:
MYSQL: Sequential Number Table
Creating a "Numbers Table" in mysql
With a numbers table the big union all above could be rewritten to
UNION ALL SELECT num, 0 from numbers where num between 0 and 23
or you could use a join on it and your coalesce syntax.

try it:
SELECT
`AllHours`.`tmp_hour`,
COALESCE(SUM(`val`),0) AS `total`
FROM
tableA
RIGHT JOIN (
SELECT 0 AS `tmp_hour`
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12
UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18
UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21
UNION ALL SELECT 22 UNION ALL SELECT 23
) AS `AllHours` ON DATE_FORMAT(dt, '%H') = `AllHours`.`tmp_hour`
WHERE `dt` BETWEEN '2018-02-19 00:00:00' AND '2018-02-19 23:59:59' OR `dt` IS NULL
GROUP BY `AllHours`.`tmp_hour`
ORDER BY `AllHours`.`tmp_hour`

Related

Get dates lies between two date ranges, as per the Day of week in my sql

I have fromDate, toDate and dayOfWeek in mysql. I want all the dates that lies on the particular dayofWeek and lies between fromDate and toDate.
eg
fromDate- '2021-09-01 00:00:00'
toDate- '2021-09-30 00:00:00'
dayOfWeek(4,5) i.e thurday and Friday
output
output
2021-09-02
2021-09-03
2021-09-01
2021-09-10
2021-09-16
2021-09-17
2021-09-23
2021-09-24
2021-09-30
I am using MYSQLWORKBENCH 8.0
here for you
select * from
(select adddate('1970-01-01',t4.i*10000 + t3.i*1000 + t2.i*100 + t1.i*10 + t0.i) as selected_date from
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t0,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t1,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t2,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t3,
(select 0 i union select 1 union select 2 union select 3 union select 4 union select 5 union select 6 union select 7 union select 8 union select 9) t4) v
where selected_date between '2021-09-01' and '2021-09-30'
having WEEKDAY(selected_date) in (3,4)
ORDER BY selected_date ASC;
0 is monday , 3 is thurday ...
from this post
Another way to get all Thursday and Friday between two dates. I use LAST_DAY() function (last month day) to set the end of the interval, but you can hard code a specific date:
WITH RECURSIVE days AS (SELECT '2021-09-01' as day
UNION ALL
SELECT DATE_ADD(day, INTERVAL 1 DAY)
FROM days
WHERE day < LAST_DAY(day))
SELECT *
FROM days
WHERE weekday(day) IN (3,4);
Output:
day
2021-09-02
2021-09-03
2021-09-09
2021-09-10
2021-09-16
2021-09-17
2021-09-23
2021-09-24
2021-09-30

laravel union query to get data per hour of the day and show 0 if there is no data

I have this mysql query that I am trying to translate into laravel query builder can anyone please help?
SELECT CONCAT(HOUR, ':00-', HOUR+1, ':00') AS Hours,
COUNT(o.id) AS id_count
FROM ( SELECT 0 AS HOUR
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12
UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18
UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21
UNION ALL SELECT 22 UNION ALL SELECT 23) AS AllHours
LEFT JOIN orders AS o ON HOUR(created_at) = HOUR AND o.created_at = '2019-10-10'
GROUP BY HOUR
ORDER BY HOUR;
Cheers
Cam

Get hourly data with gaps from midnight till now

I'm using the code below in order to generate data from midnight till now.
SELECT CONCAT(Hour, ':00-', Hour+1, ':00') AS Hours, IFNULL(COUNT(product_id), 0) AS `total_count`
FROM clicks
RIGHT JOIN (
SELECT 0 AS Hour
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12
UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18
UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21
UNION ALL SELECT 22 UNION ALL SELECT 23
) AS AllHours ON HOUR(clicked_at) = Hour
WHERE ( clicked_at BETWEEN CURRENT_DATE() AND NOW() OR clicked_at IS NULL ) AND clicks.site='awesome-site.com'
GROUP BY Hour
ORDER BY Hour
I need the code to return something like
Hours total_count
----------------------
0:00-1:00 19
1:00-2:00 2
2:00-3:00 0
3:00-4:00 0
4:00-5:00 0
5:00-6:00 1
6:00-7:00 0
7:00-8:00 0
8:00-9:00 0
9:00-10:00 4
10:00-11:00 2
11:00-12:00 0
12:00-13:00 17
13:00-14:00 1
The issue is that the query above is return is returning data with gap in the Hours column; something like:
Hours total_count
----------------------
0:00-1:00 19
1:00-2:00 2
5:00-6:00 1
9:00-10:00 4
10:00-11:00 2
12:00-13:00 17
13:00-14:00 1
Thanks for the help.
right join is the correct approach, but you are using columns from clicks table in the where statement. Instead put the filter in on:
SELECT CONCAT(Hour, ':00-', Hour+1, ':00') AS Hours, IFNULL(COUNT(product_id), 0) AS `total_count`
FROM clicks
RIGHT JOIN (
SELECT 0 AS Hour
UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12
UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15
UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18
UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21
UNION ALL SELECT 22 UNION ALL SELECT 23
) AS AllHours ON HOUR(clicked_at) = Hour
and ( clicked_at BETWEEN CURRENT_DATE() AND NOW() OR clicked_at IS NULL ) AND clicks.site='awesome-site.com'
GROUP BY Hour
ORDER BY Hour
An easy potential solution would be to just have a separate table with all the hours in a day (since the only problem seems to be having 0 entries that fall within certain hours):
Hours
-------------
0:00-1:00
1:00-2:00
2:00-3:00
...
22:00-23:00
23:00-24:00
Then JOIN that to the other table you have, I think basically all the other stuff you do should work with this such as the IFNULL and WHERE ( clicked_at BETWEEN CURRENT_DATE() AND NOW() ...

Return Everyday from MYSQL

I want to fill in the dates between today and 30 days ago in mysql.
eg:
Date Value
2015-08-05 1
2015-08-04 2
2015-08-03 0
......
2015-07-05 1
Below is mysql:
SELECT IFNULL(SUM(units.price), 0) as price, DATE(units.solddate) as date, DAY(units.solddate) as day,
(
select a.Date
from (
select curdate() - INTERVAL (a.a + (10 * b.a) + (100 * c.a)) DAY as Date
from (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as a union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
) a
where a.Date >= DATE_SUB(NOW(), INTERVAL 30 day)
) as onemonth
FROM (tables)
GROUP BY date
However, I got this error: #1242 - Subquery returns more than 1 row I understood this error.
Please advice. Thank you.
This is a solution by correcting your approach
SELECT
`dates`.`date` ,SUM( COALESCE( `units`.`price` ,0 ) ) AS `price`
FROM
(
-- Start of query for making dates
SELECT
`dates`.`date`
,CONCAT( `dates`.`date` ,' 00:00:00' ) AS `day_start_datetime`
,CONCAT( `dates`.`date` ,' 23:59:59' ) AS `day_end_datetime`
FROM
(
SELECT
DATE_SUB( CURDATE( ) ,INTERVAL `intervals`.`days` DAY ) AS `date`
FROM
(
SELECT
( `hundreds_place`.`num` * 100 )
+( `tens_place`.`num` * 10 )
+( `ones_place`.`num` * 1 ) AS `days`
FROM ( SELECT 0 AS `num` UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9 ) AS `ones_place`
JOIN ( SELECT 0 AS `num` UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9 ) AS `tens_place`
JOIN ( SELECT 0 AS `num` UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9 ) AS `hundreds_place`
) `intervals`
HAVING
`date` >= DATE_SUB( CURDATE() ,INTERVAL 30 DAY )
) `dates`
-- End of query for making dates
) `dates`
LEFT OUTER JOIN `units`
ON `units`.`sold_date` BETWEEN `day_start_datetime` AND `day_end_datetime`
WHERE
1
GROUP BY
`dates`.`date`
ORDER BY
`dates`.`date`
After getting better understanding of what you are trying to achieve , I have put together this MySQL query .
I have tested it and it worked for me .
SELECT
`dates_table`.`date`
-- ,`dates_table`.`day`
,SUM( COALESCE( `units`.`price` ,0 ) ) AS `price`
FROM
-- Start of query for making dates_table
(
SELECT
DATE( DATE_SUB( CURDATE( ) ,INTERVAL `intervals_table`.`days` DAY ) ) AS `date`
,DAY( DATE_SUB( CURDATE( ) ,INTERVAL `intervals_table`.`days` DAY ) ) AS `day`
FROM
(
SELECT 0 AS `days`
UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4
UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8
UNION SELECT 9 UNION SELECT 10 UNION SELECT 11 UNION SELECT 12
UNION SELECT 13 UNION SELECT 14 UNION SELECT 15 UNION SELECT 16
UNION SELECT 17 UNION SELECT 18 UNION SELECT 19 UNION SELECT 20
UNION SELECT 21 UNION SELECT 22 UNION SELECT 23 UNION SELECT 24
UNION SELECT 25 UNION SELECT 26 UNION SELECT 27 UNION SELECT 28
UNION SELECT 29 UNION SELECT 30 UNION SELECT 31
) `intervals_table`
HAVING
`date` >= DATE_SUB( CURDATE( ) ,INTERVAL 30 DAY )
)
-- End of query for making dates_table
`dates_table`
LEFT OUTER JOIN `units` ON
`units`.`sold_date` >= CONCAT( `dates_table`.`date` ,' 00:00:00' )
AND `units`.`sold_date` <= CONCAT( `dates_table`.`date` ,' 23:59:59' )
WHERE
1
GROUP BY
`dates_table`.`date`
ORDER BY
`dates_table`.`date`
The units table I have used for testing is :
CREATE TABLE IF NOT EXISTS `units` (
`id` int(11) NOT NULL AUTO_INCREMENT
,`price` int(11) NOT NULL
,`sold_date` datetime NOT NULL
,PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `units` (`id`, `price`, `sold_date`) VALUES
(1, 10, '2015-08-02 00:00:00'),
(2, 11, '2015-08-03 00:00:00'),
(3, 14, '2015-08-03 00:00:00'),
(4, 3, '2015-08-04 00:00:00');
Result
date price
---------- -----
2015-07-07 0
2015-07-08 0
2015-07-09 0
2015-07-10 0
2015-07-11 0
2015-07-12 0
2015-07-13 0
2015-07-14 0
2015-07-15 0
2015-07-16 0
2015-07-17 0
2015-07-18 0
2015-07-19 0
2015-07-20 0
2015-07-21 0
2015-07-22 0
2015-07-23 0
2015-07-24 0
2015-07-25 0
2015-07-26 0
2015-07-27 0
2015-07-28 0
2015-07-29 0
2015-07-30 0
2015-07-31 0
2015-08-01 0
2015-08-02 10
2015-08-03 25
2015-08-04 3
2015-08-05 0

Mysql hourly report query

I want to get an hourly report of conversions for all 24 hours.
I have this query but it returns only 19rows instead of 24
can anyone plz tell me wats wrong in this?
Thanks in advance.
SELECT HOUR( `date_time` ) AS Hours, COUNT(conversion_id) AS `conversion` FROM conversions
RIGHT JOIN (SELECT 0 AS Hour UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT
5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11
UNION ALL SELECT 12 UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL
SELECT 18 UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21 UNION ALL SELECT 22 UNION ALL SELECT 23) AS AllHours
ON HOUR(date_time) = Hour
WHERE DATE(date_time) = CURDATE() OR date_time IS NULL
GROUP BY Hour
ORDER BY Hour
If there are not entries for this hour, it is never selected. You have to query the other way round.
I think it should be something like this (hard to test without your database):
select * from (SELECT 0 AS Hour UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL SELECT 18 UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21 UNION ALL SELECT 22 UNION ALL SELECT 23) as AllHours
left outer join
(select COUNT(conversion_id) as cnt, HOUR(date_time) as h
FROM conversions
WHERE DATE(date_time) = CURDATE() OR date_time IS NULL
group by h) as c
on Hour = e.h
The right join is almost correct. I prefer that the where condition be in the on clause (rather than checking for NULL values. The key, though, is using the AllHours table in the select and group by:
SELECT AllHours.Hour AS Hours, COUNT(conversion_id) AS `conversion`
FROM conversions RIGHT JOIN
(SELECT 0 AS Hour UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT
5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11
UNION ALL SELECT 12 UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 UNION ALL SELECT 17 UNION ALL
SELECT 18 UNION ALL SELECT 19 UNION ALL SELECT 20 UNION ALL SELECT 21 UNION ALL SELECT 22 UNION ALL SELECT 23
) AS AllHours
ON HOUR(conversions.date_time) = AllHours.Hour and DATE(conversions.date_time) = CURDATE()
GROUP BY AllHOurs.Hour
ORDER BY AllHours.Hour