MySQL: filling empty fields with zeroes when using GROUP BY - mysql

I've got MySQL table
CREATE TABLE cms_webstat (
ID int NOT NULL auto_increment PRIMARY KEY,
TIMESTAMP_X timestamp DEFAULT CURRENT_TIMESTAMP,
# ... some other fields ...
)
which contains statistics about site visitors.
For getting visits per hour I use
SELECT
hour(TIMESTAMP_X) as HOUR
, count(*) AS HOUR_STAT
FROM cms_webstat
GROUP BY HOUR
ORDER BY HOUR DESC
which gives me
| HOUR | HOUR_STAT |
| 24 | 15 |
| 23 | 12 |
| 22 | 9 |
| 20 | 3 |
| 18 | 2 |
| 15 | 1 |
| 12 | 3 |
| 9 | 1 |
| 3 | 5 |
| 2 | 7 |
| 1 | 9 |
| 0 | 12 |
And I'd like to get following:
| HOUR | HOUR_STAT |
| 24 | 15 |
| 23 | 12 |
| 22 | 9 |
| 21 | 0 |
| 20 | 3 |
| 19 | 0 |
| 18 | 2 |
| 17 | 0 |
| 16 | 0 |
| 15 | 1 |
| 14 | 0 |
| 13 | 0 |
| 12 | 3 |
| 11 | 0 |
| 10 | 0 |
| 9 | 1 |
| 8 | 0 |
| 7 | 0 |
| 6 | 0 |
| 5 | 0 |
| 4 | 0 |
| 3 | 5 |
| 2 | 7 |
| 1 | 9 |
| 0 | 12 |
How should I modify the query to get such result (with one mysql query, without creating temporary tables)?
Is it possible to get such result with one MySQL query?

Create another table with a single column,
CREATE TABLE hours_list (
hour int NOT NULL PRIMARY KEY
)
Fill it with all 24 hours.
Then do a join on that table to fill in the zeroes.
SELECT
hs.hour as HOUR, COUNT(ws.ID) AS HOUR_STAT
FROM hours_list hs
LEFT JOIN cms_webstat ws ON hs.hour = hour(ws.TIMESTAMP_X)
GROUP BY hs.hour
ORDER BY hs.hour DESC

This is just the 'why it is not returning` part. Marcus' answer covers the 'how to' part.
The SQL
SELECT
hour(TIMESTAMP_X) as HOUR
, count(*) AS HOUR_STAT
FROM cms_webstat
GROUP BY HOUR
ORDER BY HOUR DESC
gets the count of the records per hour, for the timestamps present in the table
It does not give the details of what is not present in the table. Since there is no recors for the timestamp corresponding to the hour 8 (from your example) the SQL does not return any records.

I've finaly found the answer.
Maybe I'm insane, but this works.
SELECT HOUR, max(HOUR_STAT) as HOUR_STAT FROM (
(
SELECT HOUR(TIMESTAMP_X) as HOUR, count(*) as HOUR_STAT
FROM cms_webstat
WHERE date(TIMESTAMP_X) = date(now())
)
UNION (SELECT 0 as HOUR, 0)
UNION (SELECT 1 as HOUR, 0)
UNION (SELECT 2 as HOUR, 0)
UNION (SELECT 3 as HOUR, 0)
UNION (SELECT 4 as HOUR, 0)
UNION (SELECT 5 as HOUR, 0)
UNION (SELECT 6 as HOUR, 0)
UNION (SELECT 7 as HOUR, 0)
UNION (SELECT 8 as HOUR, 0)
UNION (SELECT 9 as HOUR, 0)
UNION (SELECT 10 as HOUR, 0)
UNION (SELECT 11 as HOUR, 0)
UNION (SELECT 12 as HOUR, 0)
UNION (SELECT 13 as HOUR, 0)
UNION (SELECT 14 as HOUR, 0)
UNION (SELECT 15 as HOUR, 0)
UNION (SELECT 16 as HOUR, 0)
UNION (SELECT 17 as HOUR, 0)
UNION (SELECT 18 as HOUR, 0)
UNION (SELECT 19 as HOUR, 0)
UNION (SELECT 20 as HOUR, 0)
UNION (SELECT 21 as HOUR, 0)
UNION (SELECT 22 as HOUR, 0)
UNION (SELECT 23 as HOUR, 0)
)
AS `combined_table`
GROUP BY HOUR
ORDER BY HOUR DESC
One MySQL query as desired.

$sql = 'SELECT g, MAX(v) AS v, MAX(c) AS c FROM (';
$sql .= '(SELECT DATE_FORMAT(viewed, \'%d.%m.%Y\') AS g, COUNT(1) AS v, 0 AS c FROM '.$this->prefix.'view WHERE campaignid IN ('.join(', ',$ids).') GROUP BY g)';
$sql .= ' UNION (SELECT DATE_FORMAT(clicked, \'%d.%m.%Y\') AS g, 0 AS v, COUNT(1) AS c FROM '.$this->prefix.'clicks WHERE campaignid IN ('.join(', ',$ids).') GROUP BY g)';
$today = strtotime("00:00:00");
for ($i=$today; $i>=time()-30*86400; $i-=86400) {
$sql .= ' UNION (SELECT \''.date('d.m.Y',$i).'\' AS g, 0 AS v, 0 AS c)';
}
$sql .= ') AS tmp GROUP BY g ORDER BY g DESC';
$chart = DB::getAll($sql);
p($chart);
Thanks! Made it! From 2 tables, clicks and views, joined.. works. ajaxel.com

Related

How to write this query MYSQL

I have this database:
| id | name | email | control_number | created | | | | | |
|:--:|-------|-----------------|----------------|------------|---|---|---|---|---|
| 1 | john | john#gmail.com | 1 | 14/09/2016 | | | | | |
| 2 | carl | carl#gmail.com | 1 | 13/08/2016 | | | | | |
| 3 | frank | frank#gmail.com | 2 | 12/08/2016 | | | | | |
And i want to get the COUNT in the last 12 months by the control_number.
basicly is a COUNT where control_number = 1 but by month.
So if the query is done today, its september, it should start from september to October 2015 and display the count of records for each month.
Result should be:
09/2016 = 50
08/2016 = 35
07/2016 = 20
06/2016 = 50
05/2016 = 21
04/2016 = 33
03/2016 = 60
02/2016 = 36
01/2016 = 11
12/2015 = 0
11/2015 = 0
10/2015 = 0
Hmmm. Getting the 0 values can be tricky. Assuming that you have some data each month (even if not for "1"), th en you can do:
select extract(year_month from created) as yyyymm,
sum(control_number = 1)
from t
where created >= date_sub(curdate(), interval 12 month)
group by extract(year_month from created)
order by yyyymm;
If you don't have at least one record for each month, then you'll need a left join and a table with one row per month.
Try this:
select CONCAT(SUBSTRING(ym, 5, 2), '/', SUBSTRING(ym, 1, 4)) Month, Count from (
select EXTRACT(YEAR_MONTH FROM created) ym, count(*) Count
from mytable
where EXTRACT(YEAR_MONTH FROM created) > (EXTRACT(YEAR_MONTH FROM SUBDATE(NOW(), INTERVAL 1 YEAR))
group by 1
order by 1 desc) x
Try:
select concat(month(created),'/',year(created)) as period, count(*) as cnt
from mytable
where control_number=1 and TIMESTAMPDIFF(year, created, now())=0
group by (month(created));

MySQL user retention and day to day

I'm trying to figure out how to write my SQL query to get users day to day and retention.
consider having the following row table round_statistics
on each play round i have date of the round,
now i would like to:
1. know how many users play two days in a row meaning played on Sunday and Monday, Monday and Tuesday, but Sunday and Tuesday doesn't count as two days in a row.
2. users retention 1-7
retention 7 is : % of users that have the chance to play the last 7 days (meaning they are registered at least 7 days) and had some activity (record) after 7 days.
retention 6-1 are the same only for 6-1 days.
Please help me to find out my game retention :) you will get a free coins to play it....
Thanks.
The table structure is:
user_id,round_time
for example if i played 3 times today:
user id | round_time
1000, | '2013-08-10 14:02:53'
1000, | '2013-08-10 14:03:25'
1000, | '2013-08-10 14:04:47'
the result structure is:
date | 2013-08-10 | 2013-07-10
day to day | 10 | 100
retention 7 | 15 | 125
retention 6 | 20 | 210
retention 5 | 30 | 320
retention 4 | 40 | 430
retention 3 | 50 | 540
retention 2 | 60 | 650
retention 1 | 120 | 1620
My sql don't has analytic functions, neither CTE and pivot table features, for this reasons it is not direct to do your required query (and nobody answer your question).
For this data:
create table t ( uid int, rt date);
insert into t values
(99, '2013-08-7 14:02:53' ), <- gap
(99, '2013-08-9 14:02:53' ), <-
(99, '2013-08-10 14:03:25' ),
(1000, '2013-08-7 14:02:53' ),
(1000, '2013-08-8 14:03:25' ),
(1000, '2013-08-9 14:03:25' ),
(1000, '2013-08-10 14:04:47');
This is an approach before pivot retentions, for a given date ( '2013-08-10 00:00:00' , '%Y-%m-%d') :
select count( distinct uid ) as n, d, dt from
(
select uid,
'2013-08-10 00:00:00' as d,
G.dt
from
t
inner join
( select 7 as dt union all
select 6 union all select 5 union all
select 4 union all select 3 union all
select 2 union all select 1 union all select 0) G
on DATE_FORMAT( t.rt, '%Y-%m-%d') between
DATE_FORMAT( date_add( '2013-08-10 00:00:00', Interval -1 * G.dt DAY) ,
'%Y-%m-%d')
and
DATE_FORMAT( '2013-08-10 00:00:00' , '%Y-%m-%d')
where DATE_FORMAT(rt , '%Y-%m-%d') <= DATE_FORMAT( '2013-08-10 00:00:00' ,
'%Y-%m-%d')
group by uid, G.dt
having count( distinct DATE_FORMAT( T.rt, '%Y-%m-%d') ) = G.dt + 1
) TT
group by dt
Your pre-cooked data ( DT = 0 means today visits, DT = 1 means 2 consecutive days, ...):
| N | D | DT |
--------------------------------
| 2 | 2013-08-10 00:00:00 | 0 |
| 2 | 2013-08-10 00:00:00 | 1 |
| 1 | 2013-08-10 00:00:00 | 2 |
| 1 | 2013-08-10 00:00:00 | 3 |
Here it is ( for same data ):
select count( distinct uid ) as n, d, dt from
(
select uid,
z.zt as d,
G.dt
from
t
cross join
( select distinct DATE_FORMAT( t.rt, '%Y-%m-%d') as zt from t) z
inner join
( select 7 as dt union all
select 6 union all select 5 union all
select 4 union all select 3 union all
select 2 union all select 1 union all select 0) G
on DATE_FORMAT( t.rt, '%Y-%m-%d') between
DATE_FORMAT( date_add( z.zt, Interval -1 * G.dt DAY) ,
'%Y-%m-%d')
and
z.zt
where z.zt <= z.zt
group by uid, G.dt, z.zt
having count( distinct DATE_FORMAT( T.rt, '%Y-%m-%d') ) = G.dt + 1
) TT
group by d,dt
order by d,dt
Results at sqlfiddle: http://sqlfiddle.com/#!2/c26ec/10/0
| N | D | DT | GROUP_CONCAT( UID) |
--------------------------------------------
| 2 | 2013-08-07 | 0 | 1000,99 |
| 1 | 2013-08-08 | 0 | 1000 |
| 1 | 2013-08-08 | 1 | 1000 |
| 2 | 2013-08-09 | 0 | 1000,99 |
| 1 | 2013-08-09 | 1 | 1000 |
| 1 | 2013-08-09 | 2 | 1000 |
| 2 | 2013-08-10 | 0 | 1000,99 |
| 2 | 2013-08-10 | 1 | 99,1000 |
| 1 | 2013-08-10 | 2 | 1000 |
| 1 | 2013-08-10 | 3 | 1000 |

Getting UNIX TIME with number

MySQL Query:
SELECT c.day,
COUNT(site_id)
FROM calendar c
LEFT JOIN
(
SELECT *
FROM visitors
WHERE site_id = 16
) d ON DAYOFMONTH(d.created) = c.day
WHERE c.day BETWEEN DAYOFMONTH('2012-10-01') AND DAYOFMONTH('2012-10-31')
GROUP BY c.day
ORDER BY c.day
My Tables
Calendar
id | day
---------
1 | 1
2 | 2
3 | 3
...
31 | 31
Visitors
id | site_id | created
-----------------------------------
1 | 16 | 2012-10-18 11:14:39
2 | 16 | 2012-10-18 11:15:17
3 | 11 | 2012-10-18 11:49:14
4 | 11 | 2012-10-18 11:49:43
5 | 16 | 2012-10-19 11:54:37
6 | 1 | 2012-10-19 05:56:31
7 | 2 | 2012-10-19 05:57:56
I used the above query to retrieve a daily result of visits to a site. The query solved my question here.
Results:
day | COUNT(*)
-------------
1 | 0
2 | 0
3 | 0
....
18 | 2
19 | 1
...
31 | 0
Although, now, I am having problems retrieving UNIX_TIMESTAMP from the day which I need for graphing purposes.
How do I retrieve it from the c.day in the query?
Edited:
SELECT
UNIX_TIMESTAMP('2012-10-01' + INTERVAL c.day - 1 DAY) unix_ts_day,
COUNT(v.site_id)
FROM
calendar c
LEFT JOIN (
SELECT * FROM visitors
WHERE site_id = 16 AND DATE(created) BETWEEN '2012-10-01' AND '2012-10-31'
) v
ON DAYOFMONTH(v.created) = c.day
GROUP BY
unix_ts_day

Grouping MYSQL results by multiple columns and time span

How can I select one, most recent NID, per every 7 days, per UID, starting count back from today.
If today is July 11, the following table
+-----+------------+-----+
| NID | timestamp | UID |
+-----+------------+-----+
| 1 | 1341719851 | 8 | //July 7
| 2 | 1341115051 | 8 | //July 1
| 3 | 1341547051 | 8 | //July 6
| 4 | 1341719851 | 8 | //July 8
| 5 | 1341979051 | 8 | //July 11
| 6 | 1341806251 | 9 | //July 9
| 7 | 1341460651 | 9 | //July 5
| 8 | 1341892651 | 9 | //July 10
+-----+------------+-----+
Will output this:
+-----+------------+-----+
| NID | timestamp | UID |
+-----+------------+-----+
| 2 | 1341115051 | 8 | //July 1
| 5 | 1341979051 | 8 | //July 11
| 8 | 1341892651 | 9 | //July 10
+-----+------------+-----+
In the last 7 days, most recent NID for each user is '5' and '8', in the prior 7 days, most recent NID is '2', and so on...
I'm assuming, Group By will do the trick; but I don't have a clue where to start.
UPDATE
This is the query that worked, based on the top answer:
SELECT nid, timestamp, uid, weeks_ago
FROM (
SELECT nid, timestamp, uid, FLOOR(
(UNIX_TIMESTAMP()- timestamp)/604800
) weeks_ago
FROM `table`
ORDER BY timestamp DESC
) x
GROUP BY uid, weeks_ago
select nid, max(timestamp), uid, weeks_ago
from (select nid, timestamp, uid, floor(datediff(now(), from_unixtime(timestamp))/7) weeks_ago
from mytable) x
group by nid, uid, weeks_ago
select * from (
(
select nid, uid,timestamp,floor(datediff(now(), from_unixtime(timestamp))/7) as weeks,from_unixtime(timestamp) as Dt from test t1 order by dt desc limit 0,2)
Union
(
select nid, uid,timestamp,
floor(datediff(now(), from_unixtime(timestamp))/7) as weeks,from_unixtime(timestamp) as Dt
from test t2 group by weeks having weeks>0 order by dt asc limit 0,1
)
) t4
May be this resolve your issue. Try this

MySQL left join with a single table

Using the following query to get a hourly breakdown of transactions
SELECT hour(Stamp) AS Hour, count(1) AS Count FROM Transactions GROUP by 1 WITH ROLLUP;
Results in the following output:
+------+-------+
| Hour | Count |
+------+-------+
| 0 | 269 |
| 1 | 342 |
| 2 | 319 |
| 3 | 284 |
| 4 | 235 |
| 5 | 174 |
| 6 | 91 |
| 7 | 54 |
| 8 | 31 |
| 9 | 21 |
| 10 | 21 |
| 11 | 1 |
| NULL | 1842 |
+------+-------+
I would like to display the hours with 0 transactions (e.g. in this example, every hour between 12 and 23 would show '0'). What would be the simplest way to do this?
try something like this (the -1 hour_id is for the rollup total):
drop table if exists hours;
create table hours(hour_id tinyint primary key) engine=innodb;
insert into hours (hour_id) values
(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),
(13),(14),(15),(16),(17),(18),(19),(20),(21),(22),(23),(0),(-1);
select
h.hour_id,
if(t.counter is null, 0, t.counter) as counter
from
hours h
left outer join
(
select
if(hour(stamp) is null, -1, hour(stamp)) as hour_id,
count(stamp) as counter
from
transactions group by stamp with rollup
) t
on h.hour_id = t.hour_id;
if you want it by months create a months table 1..12 + -1 etc...
SELECT Temp.Hour, COUNT(1)
FROM (
SELECT 0 AS Hour UNION
SELECT 1 UNION
SELECT 2 UNION
SELECT 3 UNION
SELECT 4 UNION
SELECT 5 UNION
SELECT 6 UNION
SELECT 7 UNION
SELECT 8 UNION
SELECT 9 UNION
SELECT 10 UNION
SELECT 11 UNION
SELECT 12 UNION
SELECT 13 UNION
SELECT 14 UNION
SELECT 15 UNION
SELECT 16 UNION
SELECT 17 UNION
SELECT 18 UNION
SELECT 19 UNION
SELECT 20 UNION
SELECT 21 UNION
SELECT 22 UNION
SELECT 23
) AS Temp
LEFT JOIN Transactions
ON Temp.Hour = HOUR(Transactions.Stamp)
GROUP BY Temp.Hour
ORDER BY Temp.Hour