How to group by time slot between two dates - mysql

I have a question that make me feel silly !
I have to do some stats on the use of my apps.
I have a table call : customer_point
id int(11) auto_increment
id_customer int(11)
type_point int(11)
date timestamp CURRENT_TIMESTAMP
I want to make this request for the entire month (with a row for each night ;) ) :
SELECT COUNT( id_customer ) , type_point, date(date)
FROM customer_point
WHERE date BETWEEN "2014-06-01 20:00:00" AND "2014-06-02 10:00:00"
GROUP BY type_point, date;
I nearly sure that i miss a crusial point but i can't find witch one.
Thank you very much for reading me !
Bye,
edit :
Sample :
INSERT INTO `customer_point` ( `id` , `id_customer` , `type_point`, `date` )
VALUES ( '', '15', '1', '2014-06-01 22:50:00'), ( '', '15', '1', '2014-06-01 23:52:00'), ( '', '15', '1', '2014-06-02 9:50:00'), ( '', '15', '1', '2014-06-30 22:50:00'), ( '', '15', '1', '2014-06-30 23:52:00'), ( '', '15', '1', '2014-07-01 02:50:00', ( '', '15', '1', '2014-07-01 09:50:00');
result :
1, 3, 2014-06-01
1, 4, 2014-06-30
I hope this will help everbody to understand my probleme :/

If you just want coutns of the actual data, check the date is within the range you are interested in and that the time is at night (ie, greater than 8pm or less than 10am, if would seem from your SQL):-
SELECT type_point, date(customer_point.date) AS aDate, COUNT( id_customer )
FROM customer_point
WHERE DATE(customer_point.date) BETWEEN "2014-06-01" AND "2014-06-30"
AND TIME(customer_point.date) >= '20:00:00' OR TIME(customer_point.date) <= '10:00:00'
GROUP BY type_point, aDate;
To get a row per day, irrespective of whether there is any data that day(ie, a count of zero it no data) then you need to generate a list of dates and then LEFT JOIN your data to it.
Something like this:-
SELECT sub0.aDate, type_point, COUNT( id_customer )
FROM
(
SELECT DATE_ADD('2014-06-01', INTERVAL units.i + tens.i * 10 DAY) AS aDate
FROM
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) units
CROSS JOIN
(SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3) tens
) sub0
LEFT OUTER JOIN customer_point
ON sub0.aDate = date(customer_point.date)
WHERE sub0.aDate BETWEEN "2014-06-01" AND "2014-06-30"
GROUP BY sub0.aDate, type_point;
You would also probably need to generate a list of type_point values.
EDIT - to go with the updated question, can you just subtract 10 hours from the date / time. So 10am on the 1st July becomes midnight on the 30th June?
SELECT type_point, date(DATE_ADD(customer_point.date, INTERVAL -10 HOUR)) AS aDate, COUNT( id_customer )
FROM customer_point
WHERE DATE(DATE_ADD(customer_point.date, INTERVAL -10 HOUR)) BETWEEN "2014-06-01" AND "2014-06-30"
AND TIME(customer_point.date) >= '20:00:00' OR TIME(customer_point.date) <= '10:00:00'
GROUP BY type_point, aDate;
SQL fiddle:-
http://www.sqlfiddle.com/#!2/ddc95/2
The issue with this is whether items from before 10am on the 1st of June count as dates for May or for June?

Using mysql you even could do
WHERE date LIKE "2014-06-%"
Edit: You need exactly from 20:00 and then you have to take in account the first day of the next mounth until the 22:00...
Ok, then just substract those 20 hours to the date:
SELECT DATE_SUB(column, INTERVAL 20 HOUR)....
Finally:
SELECT COUNT( id_customer ) , type_point, DATE_SUB(date, INTERVAL 20 HOUR) as mydate
FROM customer_point
WHERE mydate LIKE "2014-06-%"
GROUP BY type_point, date;

Related

Get active users by month

Using MySQL, I'm trying to get the number of active users I have in any given month. I have a table with ActivationDate and TerminationDate columns, and if the month being counted is after the ActivationDate and TerminationDate is null, then the user is active and should be counted. I would like to summarize these amounts by month. I'm thinking I could just sum each side and calculate the total but breaking that down won't give me a running total. I've tried with window functions, but I don't have enough experience with them to know exactly what I'm doing wrong and I'm not certain how to ask the right question.
So for instance, if I have the following data...
UserId ActivationDate TerminationDate
1 2020-01-01 null
2 2020-01-15 null
3 2020-01-20 2020-01-30
4 2020-02-01 null
5 2020-02-14 2020-02-27
6 2020-02-15 2020-02-28
7 2020-03-02 null
8 2020-03-05 null
9 2020-03-20 2020-03-21
I would like my results to be similar to:
2020-01 2 (there are 2 active users, since one signed up but cancelled before the end of the month)
2020-02 3 (2 from the previous month, plus 1 that signed up this month and is still active)
2020-03 5 (3 from previous, 2 new, 1 cancellation)
You can unpivot, then aggregate and sum. In MySQL 8.0.14 or higher, you can use a lateral join:
select date_format(x.dt, '%Y-%m-01') as dt_month,
sum(sum(cnt)) over(order by date_format(x.dt, '%Y-%m-01')) as cnt_active_users
from mytable t
cross join lateral (
select t.activationdate as dt, 1 as cnt
union all select t.terminationdate, -1
) x
where x.dt is not null
group by dt_month
order by dt_month
In earlier 8.x versions:
select date_format(x.dt, '%Y-%m-01') as dt_month,
sum(sum(cnt)) over(order by date_format(x.dt, '%Y-%m-01')) as cnt_active_users
from (
select activationdate as dt, 1 as cnt from from mytable
union all select terminationdate, -1 from mytable
) x
where x.dt is not null
group by dt_month
order by dt_month
You don't say what version of MySQL. If you're using 8.0, this should work:
create table userdates (
UserId int not null,
ActivationDate date not null,
TerminationDate date null
);
insert into userdates (UserId, ActivationDate, TerminationDate)
values
(1, cast("2020-01-01" as date), null )
, (2, cast("2020-01-15" as date), null )
, (3, cast("2020-01-20" as date), cast("2020-01-30" as date))
, (4, cast("2020-02-01" as date), null )
, (5, cast("2020-02-14" as date), cast("2020-02-27" as date))
, (6, cast("2020-02-15" as date), cast("2020-02-28" as date))
, (7, cast("2020-03-02" as date), null )
, (8, cast("2020-03-05" as date), null )
, (9, cast("2020-03-20" as date), cast("2020-03-21" as date))
, (10, cast("2020-07-20" as date), null)
, (11, cast("2019-09-12" as date), cast("2019-09-14" as date));
WITH RECURSIVE d (dt)
AS (
SELECT cast("2019-01-01" as date)
UNION ALL
SELECT date_add(dt, interval 1 month)
FROM d
WHERE dt < cast("2020-12-01" as date)
)
select d.dt
, count(distinct ud.UserId) as UserCount
from userdates ud
right outer join d on d.dt >= date_format(ud.ActivationDate, '%Y-%m-01')
and (d.dt <= ud.TerminationDate or ud.TerminationDate is null)
group by d.dt;

Show all data in a date range using MYSQL recursive function

I'm trying to get a list of sales for the past 6 months and get 0 values if I have no data for a specific month. So I'm using recursive_all_dates to generate a date range for the past 6 months which works great:
with recursive all_dates(dt) as (
-- anchor
select DATE_SUB(now(), INTERVAL 6 MONTH) dt
union all
-- recursion with stop condition
select dt + interval 1 month from all_dates where dt + interval 1 month <= DATE(now())
)
select DATE_FORMAT(dt, '%Y-%m') as ym from all_dates
This will return:
ym
------
2019-10
2019-11
2019-12
2020-01
2020-02
2020-03
2020-04
Now I want to left join this with my real data:
with recursive all_dates(dt) as (
-- anchor
select DATE_SUB(now(), INTERVAL 6 MONTH) dt
union all
-- recursion with stop condition
select dt + interval 1 month from all_dates where dt + interval 1 month <= now()
)
SELECT
DATE_FORMAT(ad.dt, '%Y-%m') as ym,
sum(profit) as profit
FROM
all_dates as ad
LEFT JOIN organisation_invoices as i
ON
DATE_FORMAT(ad.dt, '%Y-%m') = DATE_FORMAT(i.issue_date, '%Y-%m')
JOIN (
SELECT
invoice_id,
SUM(value) as profit
FROM organisation_invoice_services isrv
GROUP BY invoice_id
) isrv
ON i.id = isrv.invoice_id
WHERE
i.organisation_id = '4b166dbe-d99d-5091-abdd-95b83330ed3a' AND
i.issue_date >= DATE_SUB(NOW(), INTERVAL 6 MONTH)
GROUP BY `ym`
ORDER BY `ym` ASC
But I still only get the populated months:
ym profit
------------------
2019-12 8791
2020-02 302
2020-04 10452
The desired result:
ym profit
------------------
2019-10 0
2019-11 0
2019-12 8791
2020-01 0
2020-02 302
2020-03 0
2020-04 10452
What am I missing?
Edit: Sample data set and fiddle:
CREATE TABLE `organisation_invoices` (
`id` varchar(255) NOT NULL,
`organisation_id` varchar(255) NOT NULL,
`issue_date` date NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `organisation_invoice_services` (
`id` varchar(255) NOT NULL,
`organisation_id` varchar(255) NOT NULL,
`invoice_id` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`qty` float NOT NULL,
`value` float NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `organisation_invoices` (id, organisation_id, issue_date)
VALUES ('e11cec69-138f-4e20-88e5-5430b6c8d0a1', '4b166dbe-d99d-5091-abdd-95b83330ed3a', '2020-01-20');
INSERT INTO `organisation_invoice_services` (id, organisation_id, invoice_id, qty, `value`)
VALUES ('fe45dfd67-138f-4e20-88e5-5430b6c8d0a1', '4b166dbe-d99d-5091-abdd-95b83330ed3a', 'e11cec69-138f-4e20-88e5-5430b6c8d0a1', 1, 1000);
https://www.db-fiddle.com/f/dibyQi31CBtr2Cr8vjJA8i/0
You can use the following:
with recursive all_dates(dt) as (
-- anchor
select DATE_SUB(now(), INTERVAL 6 MONTH) dt
union all
-- recursion with stop condition
select dt + interval 1 month from all_dates where dt + interval 1 month <= now()
)
SELECT DATE_FORMAT(ad.dt, '%Y-%m') as ym, IFNULL(sum(profit),0) as profit
FROM all_dates as ad
LEFT JOIN organisation_invoices as i
ON DATE_FORMAT(ad.dt, '%Y-%m') = DATE_FORMAT(i.issue_date, '%Y-%m')
LEFT JOIN (
SELECT
invoice_id,
SUM(value) as profit
FROM organisation_invoice_services isrv
GROUP BY invoice_id
) isrv
ON i.id = isrv.invoice_id
WHERE
(i.organisation_id = '4b166dbe-d99d-5091-abdd-95b83330ed3a' AND
i.issue_date >= DATE_SUB(NOW(), INTERVAL 6 MONTH)) OR i.organisation_id IS NULL
GROUP BY `ym`
ORDER BY `ym` ASC
demo on dbfiddle.uk
Changes:
The conditions on the WHERE clause change the behaviour of your LEFT JOIN. Since you check for a specific organization_id, you only get matches between your month table and data (the LEFT JOIN behaves like a INNER JOIN). You need the following WHERE clause instead:
WHERE (i.organisation_id = '4b166dbe-d99d-5091-abdd-95b83330ed3a' AND
i.issue_date >= DATE_SUB(NOW(), INTERVAL 6 MONTH)) OR i.organisation_id IS NULL
You also have to change the second JOIN to a LEFT JOIN.

mysql: need records only if the date is there at least once each day, the date should be repeated for 4 days from the current datetime

I have a table with 6 columns: failure date, ipaddress, assettag, sid(primary key), rdl and error type.
I need a table with columns as First failure, Recent(Last) failure, ipaddress, assettag, rdl
But the records are to be there only if the date is repeated for 4 days from the current datetime. Not even one single day to be missed.
Ex: If today is 30th May, I need all the records whose failure date is there every single day--30th, 29th, 28th, 27th. If a record date is there only for two/three/one day(s)--it has to be ignored.
I can get First and Last failures using "min(date) and max(date)-group by ipaddress" but not able to get the records as per the condition--"failure (date) to be repeated for 4 days from current datetime"
select min(date), max(date), ipaddress, assettag, rdl
from flashinglist.response
where ((DATE_FORMAT((date_sub(NOW(), interval 24 hour)), '%y-%m-%d')) in
(select group_concat((DATE_FORMAT(date,'%y-%m-%d')) separator ', ')
from flashinglist.response group by ipaddress)
and (DATE_FORMAT((date_sub(NOW(), interval 48 hour)), '%y-%m-%d')) in
(select group_concat((DATE_FORMAT(date,'%y-%m-%d')) separator ', ')
from flashinglist.response group by ipaddress)
and (DATE_FORMAT((date_sub(NOW(), interval 72 hour)), '%y-%m-%d')) in
(select group_concat((DATE_FORMAT(date,'%y-%m-%d')) separator ', ')
from flashinglist.response group by ipaddress)
and (DATE_FORMAT((date_sub(NOW(), interval 96 hour)), '%y-%m-%d')) in
(select group_concat((DATE_FORMAT(date,'%y-%m-%d')) separator ', ')
from flashinglist.response group by ipaddress) )
order by max(date) desc
The above query should work as I am concatenating all dates group by IP and checking through 'IN' condition, but it doesn't work, not able to figure out why. (used 'date_format' to find only date instead of timestamp)
Below is the schema and sample data:
CREATE TABLE `response` (
`date` varchar(50) NOT NULL,
`ipaddress` varchar(16) NOT NULL,
`assettag` varchar(200) NOT NULL,
`sid` int(4) NOT NULL AUTO_INCREMENT PRIMARY KEY,
`rdl` varchar(30) NOT NULL,
`errortype` int(2) NOT NULL)
ENGINE=InnoDB DEFAULT CHARSET=latin1;
Sample data:
INSERT INTO `response` (`date`, `ipaddress`, `assettag`, `sid`, `rdl`, `errortype`) VALUES
('2019-05-31 09:46:10.878', '123.34.45.67', 'fresh', 483, '13234', 1),
('2019-05-30 19:46:11.578', '123.34.45.67', 'fresh', 490, '13234', 1),
('2019-05-29 14:30:11.577', '123.34.45.67', 'fresh', 496, '13234', 1),
('2019-05-28 17:23:11.573', '123.34.45.67', 'fresh', 499, '13234', 1),
('2019-05-27 22:32:11.550', '123.34.45.67', 'fresh', 503, '13234', 1),
('2019-05-29 12:54:11.571', '457.673.768.24', 'store', 560, '9297', 1),
('2019-05-31 08:46:11.569', '457.673.768.24', 'store', 565, '9297', 1),
('2019-05-28 10:45:11.566', '457.673.768.24', 'store', 567, '9297', 1),
('2019-05-30 20:16:11.566', '457.673.768.24', 'store', 569, '9297', 1),
('2019-05-29 23:46:11.234', '140.232.546.74', 'sample', 580, '6076', 1),
('2019-05-31 09:26:11.562', '140.232.546.74', 'sample', 581, '6076', 1),
('2019-05-30 19:34:16.533', '140.232.546.74', 'sample', 583, '6076', 1);
COMMIT;
Please change values according to today's date and the last 4 days.
My output should return First failure, Recent(Last) failure, ipaddress, assettag, rdl-- with the above sample data, it has to show IP records: 123.34.45.67 and 457.673.768.24 with corresponding max and min dates with in the range of 1 to 96 hours (4 days) only.
IP- 140.232.546.74 should not appear as it is the failure is not repeated for 4 days (28th date is missing). Hope this clears my question.
Count the number of different dates in the result, and test if this is the required number.
SELECT min(date) AS mindate, max(date) AS maxdate, date, ipaddress, assettag, rdl
FROM flashinglist.response
WHERE date < DATE_SUB(NOW(), interval 1 hour)
AND date > date_sub(NOW(), interval 96 hour)
GROUP BY ipaddress
ORDER BY mindate DESC
HAVING COUNT(DISTINCT DATE(date)) = DATE_SUB(maxdate, mindate) + 1
You also shouldn't have these lines:
AND (date > date_sub(NOW(), interval 24 hour) )
AND (date > date_sub(NOW(), interval 48 hour))
AND (date > date_sub(NOW(), interval 72 hour))
since they will exclude rows that are more than 1 day old.

Summarize database values of the last 12 hours for each hour

I've got a table with the columns id, type, value, created_at and I want to sum up the last 12 hours with one query. The result should be in an array for each hour ago.
At the moment I use this query to fetch the data for an hour:
select sum(`value`) as aggregate from `clan_values` where `type` = '2' and `created_at` between '2016-04-12 10:00:00' and '2016-04-12 10:59:59'
I dont have any clue how to solve this with just one query.
Thank you.
you can try using this query, it'll return sum(value) for last 12 hours including the hour of current_timestamp(). If there are no rows for a certain hour you'll get a zero for the aggregate of that hour.
SELECT sum(IFNULL(`value`,0)) as aggregate,
MyTime.MyHour,
DATE_ADD(DATE(created_at),
INTERVAL HOUR(created_at) hour) as ActualHour
FROM
(SELECT DATE_ADD(DATE(CURRENT_TIMESTAMP()),
INTERVAL HOUR(CURRENT_TIMESTAMP())-hour.count hour) as MyHour
FROM
(SELECT 0 as count
UNION SELECT 1
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
UNION SELECT 7
UNION SELECT 8
UNION SELECT 9
UNION SELECT 10
UNION SELECT 11)AS Hour
)as MyTime LEFT JOIN
`clan_values` ON DATE_ADD(DATE(created_at),INTERVAL HOUR(created_at) hour) = MyTime.MyHour
AND `type` = '2'
GROUP BY MyTime.MyHour,ActualHour
ORDER BY MyTime.MyHour ASC
sqlfiddle
You can try group by DATE_FORMAT %H
select sum(`value`) as aggregate
from `clan_values`
where `type` = '2' and `created_at` between '2016-04-12 10:00:00' and '2016-04-12 10:59:59'
group by DATE_FORMAT( `created_at`,'%H');

How I can count the number of times a value appears in a column grouped by day?

My table structure is:
CREATE TABLE `survey` (
`id` int(11) NOT NULL auto_increment,
`submitdate` datetime default NULL,
`answer` varchar(5) collate utf8_unicode_ci default NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=499 ;
Here answer contains values such as a1, a2, a3.
I want to calculate the last 10 days records depending upon answer. Is there is no record on a particular day, it should be zero.
The output I want is
date count answer
19-11-2012 10 a1
19-11-2012 8 a2
19-11-2012 0 a3
18-11-2012 30 a1
18-11-2012 30 a2
18-11-2012 30 a3
I used a query like
SELECT days.day, count(survey.id)
FROM
(select curdate() as day
union select curdate() - interval 1 day
union select curdate() - interval 2 day
union select curdate() - interval 3 day
union select curdate() - interval 4 day
union select curdate() - interval 5 day
union select curdate() - interval 6 day
union select curdate() - interval 7 day
union select curdate() - interval 8 day
union select curdate() - interval 9 day) days
left join survey
on days.day = date(survey.submitdate)
group by
days.day
You can look back 10 days by using the SUBDATE function and compare dates using '>'. You'll also want to GROUP BY the answer as well as the day, since you're attempting to calculate the count of each individual answer per day.
SELECT DATE(submitdate) AS day
answer,
COUNT(*) AS answer_count
FROM survey
WHERE DATE(submitdate) > DATE(SUBDATE(CURRENT_DATE, 10))
GROUP BY day, answer;