MySQL function IFNULL not working with GROUP BY - mysql

I've got a standard table with the list of users and I've got a column lastactivity with UNIX Timestamp (which shows when they have logged in) and column timestamp with UNIX Timestamp that shows when they have registered.
I've build a SQL query that shows how many users were active within 24 hours (86400 seconds) from now and grouped results by weeks so the counter counts how many users have registered each week:
SELECT
IFNULL(COUNT(*),0) as `counter`,
(WEEK(`timestamp`)) as `week`
FROM
`clients`
WHERE
(CAST(UNIX_TIMESTAMP() as signed) - CAST(`lastactivity` as signed)) <= 86400
GROUP BY
WEEK(`timestamp`);
The issue is that function IFNULL(COUNT(*),0) is not working as I intended. This SQL query won't display the week if there is NULL / 0 on the counter even with IFNULL() MySQL function. That is probably because of how GROUP BY works. So for example I will get this kind of result:
counter | week
2 | 11
1 | 13
9 | 14
6 | 17
But I would like to show each week like this:
counter | week
2 | 11
0 | 12
1 | 13
9 | 14
0 | 15
0 | 16
6 | 17
Anyone have idea how can I fix this issue?
Gordon is trying to help me by getting LEFT JOIN query but I still got the same results, maybe I am doing something wrong here:
SELECT
COUNT(a.id) as `counter`,
(WEEK(b.timestamp)) as `week`
FROM
`users` a
LEFT JOIN
`users` b
ON
a.id = b.id
WHERE
(CAST(UNIX_TIMESTAMP() as signed) - CAST(a.lastactivity as signed)) <= 86400
GROUP BY
WEEK(b.timestamp);

The problem is that you don't understand how the query works. IFNULL() (or the standard version COALESCE() converts a column value that is NULL to some other value. However, COUNT() never returns NULL. So, leave it out:
SELECT COUNT(*) as `counter`, WEEK(`timestamp`) as `week`
FROM `clients`
WHERE (CAST(UNIX_TIMESTAMP() as signed) - CAST(`lastactivity` as signed)) <= 86400
GROUP BY WEEK(`timestamp`);
Your problem is missing rows, not NULL values. You would have to solve this with a LEFT JOIN.
EDIT:
You need a left join to include all the weeks:
SELECT COUNT(c.timestamp) as `counter`, wk as `week`
FROM (SELECT 11 as wk UNION ALL
SELECT 12 UNION ALL
SELECT 13 UNION ALL
SELECT 14 UNION ALL
SELECT 15 UNION ALL
SELECT 16 UNION ALL
SELECT 17
) w LEFT JOIN
`clients` c
ON WEEK(c.`timestamp`) = w.wk
WHERE (CAST(UNIX_TIMESTAMP() as signed) - CAST(`lastactivity` as signed)) <= 86400
GROUP BY WEEK(`timestamp`);

Related

MySQL query for records that existed at any point each week

I have a table with created_at and deleted_at timestamps. I need to know, for each week, how many records existed at any point that week:
week
records
2022-01
4
2022-02
5
...
...
Essentially, records that were created before the end of the week and deleted after the beginning of the week.
I've tried various variations of the following but it's under-reporting and I can't work out why:
SELECT
DATE_FORMAT(created_at, '%Y-%U') AS week,
COUNT(*)
FROM records
WHERE
deleted_at > DATE_SUB(deleted_at, INTERVAL (WEEKDAY(deleted_at)+1) DAY)
AND created_at < DATE_ADD(created_at, INTERVAL 7 - WEEKDAY(created_at) DAY)
GROUP BY week
ORDER BY week
Any help would be massively appreciated!
I would create a table wktable that looks like so (for the last 5 weeks of last year):
yrweek | wkstart | wkstart
-------+------------+------------
202249 | 2022-11-27 | 2022-12-03
202250 | 2022-12-04 | 2022-12-10
202251 | 2022-12-11 | 2022-12-17
202252 | 2022-12-18 | 2022-12-24
202253 | 2022-12-25 | 2022-12-31
To get there, find a way to create 365 consecutive integers, make all the dates of 2022 out of that, and group them by year-week.
This is an example:
CREATE TABLE wk AS
WITH units(units) AS (
SELECT 0 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION
SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9
)
,tens AS(SELECT units * 10 AS tens FROM units )
,hundreds AS(SELECT tens * 10 AS hundreds FROM tens )
,
i(i) AS (
SELECT hundreds +tens +units
FROM units
CROSS JOIN tens
CROSS JOIN hundreds
)
,
dt(dt) AS (
SELECT
DATE_ADD(DATE '2022-01-01', INTERVAL i DAY)
FROM i
WHERE i < 365
)
SELECT
YEAR(dt)*100 + WEEK(dt) AS yrweek
, MIN(dt) AS wkstart
, MAX(dt) AS wkend
FROM dt
GROUP BY yrweek
ORDER BY yrweek;
With that table, go:
SELECT
yrweek
, COUNT(*) AS records
FROM wk
JOIN input_table ON wk.wkstart < input_table.deleted_at
AND wk.wkend > input_table.created_at
GROUP BY
yrweek
;
I first build a list with the records, their open count, and the closed count
SELECT
created_at,
deleted_at,
(SELECT COUNT(*)
from records r2
where r2.created_at <= r1.created_at ) as new,
(SELECT COUNT(*)
from records r2
where r2.deleted_at <= r1.created_at) as closed
FROM records r1
ORDER BY r1.created_at;
After that it's just adding a GROUP BY:
SELECT
date_format(created_at,'%Y-%U') as week,
MAX((SELECT COUNT(*)
from records r2
where r2.created_at <= r1.created_at )) as new,
MAX((SELECT COUNT(*)
from records r2
where r2.deleted_at <= r1.created_at)) as closed
FROM records r1
GROUP BY week
ORDER BY week;
see: DBFIDDLE
NOTE: Because I use random times, the results will change when re-run. A sample output is:
week
new
closed
2022-00
31
0
2022-01
298
64
2022-02
570
212
2022-03
800
421

Mysql: Subtraction between rows and sum with other table

I have two tables, both with a Time column as timestamp type which is filled by default when the row is created: Table1 is updated approximately every 10 seconds:
Time | Val_1a | Val_2a | Val_3a
2021-11-06 13:59:53 | 15 | 10 | 35
2021-11-06 14:00:02 | 12 | 15 | 34
.................
2021-11-06 14:05:25 | 11 | 13 | 35
2021-11-06 14:05:35 | 11 | 17 | 36
Table2 is updated every hour after mathematical operations on table1:
Time | Var_1b | Var_2b | Var_3b
2021-11-06 11:00:00 | 2 | 15 | 30
2021-11-06 12:00:00 | 8 | 12 | 32
2021-11-06 13:00:00 | 12 | 11 | 35
What I would like to get but I'm not able to do in any way, is:
Check that the last table1.Val_2a value is greater than the first table1.Val_2a value written at the beginning of the current hour (with the tables above, check if 17 > 15). If this condition is not met, the entire query must return 0 otherwise:
2a) If the last row in table2 refers to the previous day, then the query result is simply the difference of the two table1.Val_2a values (17 - 15 = 2)
2b) Otherwise their difference is calculated as at point 2a (17-15 = 2) and it is added to the table2.Var_1b value (2 + 12 = 14)
I hope I was able to explain it in a clearly way, and that it all is possible with a single query. Thanks everyone for the support
Sorry, if I add an Answer but I couldn't add the image into the comment.
This is the qwery I used to test the CASE clause
SELECT t1.dtm, t1.Val_2a2, t1.Val_2a1,
CASE WHEN Val_2a2 > Val_2a1
THEN Val_2a2-Val_2a1 ELSE 0 END AS ValF FROM (SELECT DATE_FORMAT(time, '%Y-%m-%d %H:00:00') dtm,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time),',',1) Val_2a1,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time DESC),',',1) Val_2a2 FROM table1 GROUP BY dtm) t1
and this is the unexpected result
Qwery result
It is possible in a single query but different people will have different method of doing it. Whatever the method is, I personally think that the most important part is to keep the logic intact. The details you've provided in your question got me assuming that this might be a kind of query you're looking for:
SELECT t1.dtm, t1.Val_2a2, t1.Val_2a1, t2.Val_1b2,
CASE WHEN Val_2a2 > Val_2a1
THEN Val_2a2-Val_2a1+Val_1b2 ELSE 0 END AS ValF
FROM
(SELECT DATE_FORMAT(time, '%Y-%m-%d %H:00:00') dtm,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time),',',1) Val_2a1 ,
SUBSTRING_INDEX(GROUP_CONCAT(Val_2a ORDER BY time DESC),',',1) Val_2a2
FROM table1
GROUP BY dtm) t1
LEFT JOIN
(SELECT DATE(time) dtm,
SUBSTRING_INDEX(GROUP_CONCAT(Val_1b ORDER BY time DESC),',',1) Val_1b2
FROM table2
GROUP BY dtm) t2
ON DATE(t1.dtm)=t2.dtm;
Demo fiddle
hoping it can help someone else, after some more test this is the final qwery I got, considering I just need a value on the fly without needing of storing it.
Of course every consideration by the experts is more than appreciate.
Thanks to all
SELECT
CASE WHEN
(ABS(t1.Val_2a2) - ABS(t1.Val_2a1)) BETWEEN 0 AND 30
THEN t1.Val_2a2-t1.Val_2a1+t2.Val_1b2
ELSE t2.Val_1b2
END AS My_result
FROM
(SELECT DATE_FORMAT(Time, '%Y-%m-%d %H:00:00') dtm,
(SELECT Val_2a FROM table1 WHERE Time >= DATE_FORMAT(NOW(),"%Y-%m-%d %H:00:00") ORDER BY Time LIMIT 1) Val_2a1,
(SELECT Val_2a FROM table1 WHERE Time >= DATE_FORMAT(NOW(),"%Y-%m-%d %H:00:00") ORDER BY Time DESC LIMIT 1) Val_2a2
FROM table1
GROUP BY dtm
ORDER BY Time DESC LIMIT 1) t1
LEFT JOIN
(SELECT (Time) dtm,
(Val_1b) Val_1b2
FROM table2
GROUP BY dtm ORDER BY dtm DESC LIMIT 1) t2
ON DATE(t1.dtm)= DATE(t2.dtm)

Count of rows grouped by date and older

I have a table that contains records for registration for an event.
I am able to get a count of registrations, grouped by the date -
SELECT DATE(`date_registered`) as `date`, COUNT(*) as `total`
FROM `registration`
GROUP BY DATE(`date_registered`)
But I would like to get the running count, based on the date -
So I am looking at doing something like -
SELECT DATE(`date_registered`) as `date`, COUNT(*) as `total`
FROM `registration`
GROUP BY DATE(`date_registered`) <= DATE(`date_registered`)
I attempted to do a LEFT JOIN with a count of all the dates prior, but that does not work -
SELECT DATE(r1.`date_registered`) as `date`, COUNT(*)+r3.preTotal as `total`
FROM `registration` r1
LEFT JOIN (
SELECT COUNT(*) as preTotal
FROM `registration` r2
WHERE DATE(r2.`date_registered`) < DATE(r1.`date_registered`)
) r3
GROUP BY DATE(r1.`date_registered`)
I have created a SQLFiddle
with the basic count - http://sqlfiddle.com/#!9/420d1d/5
and with my failed attempt - http://sqlfiddle.com/#!9/420d1d/6
I assume there is a simple way that I am missing.
Edit
Here are the same at rextester.com
basic count - http://rextester.com/GISU91151
and failed attempt - http://rextester.com/RDTFK34261
How about this:
SELECT
DATE(`date_registered`) AS date,
(
SELECT COUNT(*)
FROM `registration` t2
WHERE DATE(t2.`date_registered`) <= DATE(t1.`date_registered`)
) AS total
FROM `registration` t1
GROUP BY DATE(`date_registered`);
One way to make this work is creating a corelated subquery.
Query
SELECT
registration_counted.date
, (registration_counted.total + registration_counted.preCount) AS total
FROM (
SELECT
DATE(date_registered) AS DATE
, COUNT(*) AS total
, (
SELECT
COUNT(*)
FROM
registration AS registration2
WHERE
DATE(registration2.date_registered) < DATE(registration1.date_registered)
)
AS
preCount
FROM
registration AS registration1
GROUP BY
DATE(date_registered)
ORDER BY
DATE(date_registered) ASC
)
AS
registration_counted
Result
date total
---------- --------
2014-01-16 1
2014-01-20 2
2014-01-22 3
2014-01-31 18
2014-02-01 19
2014-02-04 22
2014-02-12 23
2014-02-19 24
2014-02-20 28
2014-02-26 30
2014-02-27 34
2014-02-28 37

MySQL get count of periods where date in row

I have an MySQL table, similar to this example:
c_id date value
66 2015-07-01 1
66 2015-07-02 777
66 2015-08-01 33
66 2015-08-20 200
66 2015-08-21 11
66 2015-09-14 202
66 2015-09-15 204
66 2015-09-16 23
66 2015-09-17 0
66 2015-09-18 231
What I need to get is count of periods where dates are in row. I don't have fixed start or end date, there can be any.
For example: 2015-07-01 - 2015-07-02 is one priod, 2015-08-01 is second period, 2015-08-20 - 2015-08-21 is third period and 2015-09-14 - 2015-09-18 as fourth period. So in this example there is four periods.
SELECT
SUM(value) as value_sum,
... as period_count
FROM my_table
WHERE cid = 66
Cant figure this out all day long.. Thx.
I don't have enough reputation to comment to the above answer.
If all you need is the NUMBER of splits, then you can simply reword your question: "How many entries have a date D, such that the date D - 1 DAY does not have an entry?"
In which case, this is all you need:
SELECT
COUNT(*) as PeriodCount
FROM
`periods`
WHERE
DATE_ADD(`date`, INTERVAL - 1 DAY) NOT IN (SELECT `date` from `periods`);
In your PHP, just select the "PeriodCount" column from the first row.
You had me working on some crazy stored procedure approach until that clarification :P
I should get deservedly flamed for this, but anyway, consider the following...
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(date DATE NOT NULL PRIMARY KEY
,value INT NOT NULL
);
INSERT INTO my_table VALUES
('2015-07-01',1),
('2015-07-02',777),
('2015-08-01',33),
('2015-08-20',200),
('2015-08-21',11),
('2015-09-14',202),
('2015-09-15',204),
('2015-09-16',23),
('2015-09-17',0),
('2015-09-18',231);
SELECT x.*
, SUM(y.value) total
FROM
( SELECT a.date start
, MIN(c.date) end
FROM my_table a
LEFT
JOIN my_table b
ON b.date = a.date - INTERVAL 1 DAY
LEFT
JOIN my_table c
ON c.date >= a.date
LEFT
JOIN my_table d
ON d.date = c.date + INTERVAL 1 DAY
WHERE b.date IS NULL
AND c.date IS NOT NULL
AND d.date IS NULL
GROUP
BY a.date
) x
JOIN my_table y
ON y.date BETWEEN x.start AND x.end
GROUP
BY x.start;
+------------+------------+-------+
| start | end | total |
+------------+------------+-------+
| 2015-07-01 | 2015-07-02 | 778 |
| 2015-08-01 | 2015-08-01 | 33 |
| 2015-08-20 | 2015-08-21 | 211 |
| 2015-09-14 | 2015-09-18 | 660 |
+------------+------------+-------+
4 rows in set (0.00 sec) -- <-- This is the number of periods
there is a simpler way of doing this, see here SQLfiddle:
SELECT min(date) start,max(date) end,sum(value) total FROM
(SELECT #i:=#i+1 i,
ROUND(Unix_timestamp(date)/(24*60*60))-#i diff,
date,value
FROM tbl, (SELECT #i:=0)n WHERE c_id=66 ORDER BY date) t
GROUP BY diff
This select groups over the same difference between sequential number and date value.
Edit
As Strawberry remarked quite rightly, there was a flaw in my apporach, when a period spans a month change or indeed a change into the next year. The unix_timestamp() function can cure this though: It returns the seconds since 1970-1-1, so by dividing this number by 24*60*60 you get the days since that particular date. The rest is simple ...
If you only need the count, as your last comment stated, you can do it even simpler:
SELECT count(distinct diff) period_count FROM
(SELECT #i:=#i+1 i,
ROUND(Unix_timestamp(date)/(24*60*60))-#i diff,
date,value
FROM tbl,(SELECT #i:=0)n WHERE c_id=66 ORDER BY date) t
Tnx. #cars10 solution worked in MySQL, but could not manage to get period count to echo in PHP. It returned 0. Got it working tnx to #jarkinstall. So my final select looks something like this:
SELECT
sum(coalesce(count_tmp,coalesce(count_reserved,0))) as sum
,(SELECT COUNT(*) FROM my_table WHERE cid='.$cid.' AND DATE_ADD(date, INTERVAL - 1 DAY) NOT IN (SELECT date from my_table WHERE cid='.$cid.' AND coalesce(count_tmp,coalesce(count_reserved,0))>0)) as periods
,count(*) as count
,(min(date)) as min_date
,(max(date)) as max_date
FROM my_table WHERE cid=66
AND coalesce(count_tmp,coalesce(count_reserved,0))>0
ORDER BY date;

MySQL - Full outer join on same table using COUNT

I am trying to generate a table in the following format.
Proday | 2014-04-01 | 2014-03-01
--------------------------------
1 | 12 | 17
2 | 6 | 0
7 | 0 | 24
13 | 3 | 7
Prodays (duration between two timestamps) is a calculated value and the data for months is a COUNT. I can output the data for a single month, but am having troubles joining queries to additional months. The index (prodays) may not match for each month. e.g.. 2014-04-01 may not have any data for Prodays 7, whereas 2014-03-01 may not have Proday 2. Should indicate with 0 or null.
I suspect FULL OUTER JOIN is what should do the trick. But have read that's not possible in Mysql?
This is the query to get data for a single month:
SELECT round((protime - createtime) / 86400) AS prodays, COUNT(id) AS '2014-04-01'
FROM `tbl_users` as t1
WHERE status = 1 AND DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') >= '2014-04-01'
AND DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') <= LAST_DAY('2014-04-01')
GROUP BY prodays
ORDER BY `prodays` ASC
How can I join/union an additional query to create a column for 2014-03-01?
You want to use conditional aggregation -- that is, move the filtering logic from the where clause to the select clause:
SELECT round((protime - createtime) / 86400) AS prodays,
sum(DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') >= '2014-04-01' AND
DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') <= LAST_DAY('2014-04-01')
) as `2014-04-01`,
sum(DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') >= '2014-03-01' AND
DATE_FORMAT(FROM_UNIXTIME(createtime),'%Y-%m-%d') <= LAST_DAY('2014-03-01')
) as `2014-03-01`
FROM `tbl_users` as t1
WHERE status = 1
GROUP BY prodays
ORDER BY `prodays` ASC;