Dynamically count values in column, grouped by months - mysql

I'm trying to get a single mySQL query that returns the count of unique values, grouped by months.
I have a table created based on data similar to this:
CREATE TABLE `animals` (
`timestamp` datetime NOT NULL,
`animal` tinytext NOT NULL,
`comment` text NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `animals` (`timestamp`, `animal`, `comment`) VALUES
('2019-06-03 09:09:00', 'dog', 'good'),
('2019-06-03 12:00:00', 'cat', ''),
('2019-06-03 19:00:00', 'cat', ''),
('2019-07-04 09:00:00', 'cat', ''),
('2019-07-04 12:00:00', 'cat', 'feisty'),
('2019-07-04 18:51:00', 'dog', ''),
('2019-08-05 09:00:00', 'cat', ''),
('2019-08-05 12:00:00', 'cat', ''),
('2019-08-05 19:00:00', 'cat', ''),
('2019-09-06 09:00:00', 'cat', ' evil'),
('2019-09-06 12:00:00', 'cat', ''),
('2019-09-06 19:00:00', 'cat', '')
I've managed to write a query that at least gives me the count per month (as long as it is more than zero), but the query just returns the count for "cat", "dog" or anything I explicitly ask for.
My goal is to get a response similar to the following:
month | dog | cat
-------------------
2019-06 | 1 | 2
2019-07 | 1 | 2
2019-08 | 0 | 3
2019-09 | 0 | 3
How do I writhe such a query?
Is it possible to write a query that automatically counts any new values in the animal column too?
Thanks

You can use the following code, to get flexible columns from the animal column
, that does the counting for you.
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'Sum(`animal` = ''',
col,
''') as `',
col, '`')
) INTO #sql
FROM
(
select animal col
from animals
)d;
SET #sql = CONCAT('SELECT date_format(`timestamp`, "%Y-%m") `month`, ', #sql, '
from `animals`
group by `month`
order by `month`');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
Se dbfiddle example https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=09d0f26087d66452fde1a22e91de7b3a

You can have
SELECT date_format(`timestamp`, '%Y-%m') AS month, animal, COUNT(*) as count
FROM animals
GROUP BY 1, 2
but this won't give you dynamically more columns. For more columns, I guess you need to build a dynamic SQL command looping over the distinct animals you have. If you really need this, you should consider to build that gives you the SQL string or directly the result.

You want conditional aggregation:
select
date_format(`timestamp`, '%Y-%m') `month`,
sum(`animal` = 'dog') dog,
sum(`animal` = 'cat') cat
from `animals`
group by `month`
order by `month`
Demo on DB Fiddle:
month | dog | cat
:------ | --: | --:
2019-06 | 1 | 2
2019-07 | 1 | 2
2019-08 | 0 | 3
2019-09 | 0 | 3
If you want to handle dynamically the column list, then you have to go for dynamic sql:
set #sql = null;
select
concat(
'select ',
'date_format(`timestamp`, ''%Y-%m'') `month`, ',
group_concat(
concat(
'sum(`animal` = ''',
`animal`,
''') ',
`animal`
)
order by `animal`
separator ', '
),
' from `animals` ',
'group by `month` '
'order by `month`'
)
into #sql
from (
select distinct `animal` from `animals`
) t;
select #sql;
prepare stmt from #sql;
execute stmt;
deallocate prepare stmt;

Related

using pivot function in mysql to make a table apriori

this was almost same cases in this case MySQL pivot row into dynamic number of columns but the results was different and it make me confuse.
let's say i have 3 tables
create table order_match
(
id int(10) PRIMARY KEY not null,
order_status_id int(10) not null
);
create table order_match_detail
(
id int(10) PRIMARY KEY not null,
order_match_id int(10) not null,
product_id int(10) NOT NULL
);
create table product
(
id int(10) PRIMARY KEY not null,
name varchar(255) not null
);
Insert into order_match (id, order_status_id)
select 1, 6 union all
select 2, 7 union all
select 3, 6 union all
select 4, 6;
Insert into order_match_detail (id, order_match_id, product_id)
select 1, 1, 147 union all
select 2, 2, 148 union all
select 3, 3, 147 union all
select 4, 4, 149 union all
select 5, 4, 147;
Insert into product (id, name)
select 147, 'orange' union all
select 148, 'carrot' union all
select 149, 'Apple';
with order_match.id = order_match_detail.order_match_id
and order_match_detail.product_id = product.id
so like the previous case in MySQL pivot row into dynamic number of columns i want to input the product name with the transaction in order_status_id not in 7 (because 7 is expired transaction and denied)
the expected results was like this :
id (in order_match) | Orange | Carrot | Apple
1 1 0 0
3 1 0 0
4 1 0 1
based on solution in previous cases, i used this
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'count(case when product.name = ''',
product.name,
''' then 1 end) AS ',
replace(product.name, ' ', '')
)
) INTO #sql
from product;
SET #sql = CONCAT('SELECT omd.order_match_id, ', #sql, ' from order_match_detail omd
left join order_match om
on omd.order_match_id = om.id
left join product p
on omd.product_id = p.id
where om.order_status_id in (4, 5, 6, 8)
group by omd.order_match_id');
PREPARE stmt FROM #sql;
EXECUTE stmt;
but idk why return 0 and it's no way
this is the fiddle https://www.db-fiddle.com/f/nDe3oQ3VdtfS5QDokieHN4/6
For your GROUP_CONCAT query; in your case stmt, you are referring to your product table as product itself. But in your join query, you are referring to product table as alias p. Since the first group_concat query is a part of the join query, you need to keep the table aliases same.(made changes at line 5)
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'count(case when p.name = ''',
product.name,
''' then 1 end) AS ',
replace(product.name, ' ', '')
)
) INTO #pivotsql
from product;
SET #sql = CONCAT('SELECT omd.order_match_id, ', #pivotsql, ' from order_match_detail omd
left join order_match om
on omd.order_match_id = om.id
left join product p
on omd.product_id = p.id
where om.order_status_id in (4, 5, 6, 8)
group by omd.order_match_id');
PREPARE stmt FROM #sql;
EXECUTE stmt;

MySQL select non null columns on pivoted table

I'm still having some troubles to understand pivoted tables, but I managed to do this working query:
select region,
sum(case when rescheduleCause = 'CLOSED' then round(reschedulePercentage,2) end) as 'CLOSED',
sum(case when rescheduleCause = 'RESHUFFLE' then round(reschedulePercentage,2) end) as 'RESHUFFLE',
sum(case when rescheduleCause = 'NEW PRIORITY' then round(reschedulePercentage,2) end) as 'NEW PRIORITY'
from fancy_table
group by region
I'm getting the following result which calculations are correct:
region | CLOSED | RESHUFFLE | NEW PRIORITY
___________________________________________
RegionA | 23.08 | NULL | 38.46
RegiobB | 23.08 | NULL | 7.69
My problem in that RESHUFFLE column full of NULL values. My desired result set would look like this:
region | CLOSED | NEW PRIORITY
___________________________________________
RegionA | 23.08 | 38.46
RegiobB | 23.08 | 7.69
I know this can be achieved if I modify this query into a stored procedure using a dynamic sql, but the columns I need are as limitated as these ones, so I think it's not really that necessary. I've tried:
ifnull(sum(case when rescheduleCause = 'RESHUFFLE' then round(reschedulePercentage,2) end),0) as 'RESHUFFLE',
where `RESHUFFLE` is not null
where 'RESHUFFLE' is not null
having `RESHUFFLE` is not null
I'm out of ideas.
When you are querying in SQL you need to know the number of columns ahead of time, you can't just exclude columns on the fly. If you are going to have an unknown number of columns, then you'll need to use dynamic SQL to first create a sql string that will be executed.
In MySQL this can be done using prepared statements. This will first query your table to only return the rescheduleCause values that you actually have. These values are concatenated together into a sql string, then you'll use this string to create your final query to be executed:
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'sum(case when rescheduleCause = ''',
rescheduleCause,
''' then round(reschedulePercentage,2) end) AS ',
replace(rescheduleCause, ' ', '')
)
) INTO #sql
from fancy_table;
SET #sql = CONCAT('SELECT region, ', #sql, '
from fancy_table
group by region');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
See SQL Fiddle with Demo

Dynamic pivot query using MySQL

I have the following table with the details:
Example:
CREATE TABLE Table1
(`PK` int, `Name` varchar(3), `Subject` varchar(9), `Grade` varchar(1));
INSERT INTO Table1
(`PK`, `Name`, `Subject`, `Grade`)
VALUES
(1, 'Bob', 'Math', 'A'),
(2, 'Bob', 'History', 'B'),
(3, 'Bob', 'Language', 'C'),
(4, 'Bob', 'Biology', 'D'),
(5, 'Sue', 'History', 'C'),
(6, 'Sue', 'Math', 'A'),
(7, 'Sue', 'Music', 'A'),
(8, 'Sue', 'Geography', 'C');
Now I want to write a store procedure through which I want to pivot the table.
Attempt
DELIMITER $$
create PROCEDURE sptest1(IN nm varchar(50),IN sub varchar(50))
begin
SET #sql = NULL;
SELECT GROUP_CONCAT(DISTINCT
CONCAT('MAX(CASE WHEN ', nm ,' = ''', nm,
''' THEN grade END) `', nm, '`'))
INTO #sql
FROM table1;
SET #sql = CONCAT('SELECT ', sub ,',', #sql, '
FROM table1
GROUP BY ', sub ,'');
select #sql;
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END$$
DELIMITER ;
Calling Function:
call sptest1('name','subject')
But getting wrong result:
subject name
----------------
Biology null
Geography null
History null
Language null
Math null
Music null
Expected Result:
subject Bob Sue
-------------------------
Biology D null
Geography null C
History B C
Language C null
Math A A
Music null A
You can rewrite you dynamic sql query as
SET #sql = NULL;
SELECT GROUP_CONCAT( DISTINCT
CONCAT('MAX(CASE WHEN `Name` = ''',
`Name`,
''' THEN grade END) ',
`Name`)
)
INTO #sql
FROM table1;
SET #sql = CONCAT('SELECT `Subject`, ', #sql, ' FROM table1 GROUP BY `Subject`');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
Demo
In your stored procedure you have to hard code your name column for you dynamic sql if you pass it as parameter then you can use case to check for column, for example
MAX(CASE WHEN ', nm ,' = ''', nm, ..
Above part of procedure will evaluated as MAX(CASE WHEN name = 'name' .. because variable nm contains 'name' it will not recognize it as column
DROP PROCEDURE `sptest1`;
DELIMITER $$
CREATE
PROCEDURE `sib`.`sptest1`(IN nm VARCHAR(50),IN sub VARCHAR(50))
BEGIN
SET #sql = NULL;
SELECT GROUP_CONCAT( DISTINCT
CONCAT('MAX(CASE WHEN ',nm,' = ''',CASE WHEN nm = 'name' THEN `Name` END,''' THEN grade END) ',CASE WHEN nm = 'name' THEN `Name` END)
)
INTO #sql
FROM table1;
SET #sql = CONCAT('SELECT ', sub ,',', #sql, ' FROM table1 GROUP BY ', sub ,'');
/*select #sql; */
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END$$
DELIMITER ;
Demo

count row values as column name mysql

I have table user_completed
CREATE TABLE IF NOT EXISTS `user_completed` (
`rowId` int(10) unsigned NOT NULL AUTO_INCREMENT,
`designer_id` int(10) unsigned NOT NULL,
`status` varchar(54) DEFAULT NULL,
PRIMARY KEY (`rowId`),
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=7 ;
INSERT INTO `user_completed` (`rowId`, `designer_id`, `status`) VALUES
(1, 1, accept),
(2, 1, reject),
(3, 1, accept),
(4, 1, reject),
(5, 1, overtime),
(6, 2, accept)
(7, 2, accept)
(8, 3, accept)
(9, 2, reject);
Which look like:
rowId designer_id status
1 1 accept
2 1 reject
3 1 accept
4 1 reject
5 1 overtime
6 2 accept
7 2 accept
8 3 accept
9 2 reject
I want to get result below:
designer_id accept overtime reject
1 2 1 2
2 2 0 1
3 1 0 0
But I have no idea how to group designer_id then count distinct status and each into columns like above.
Try this
SELECT designer_id,
SUM(IF(status = 'accept',1,0)) as 'Accept',
SUM(IF(status = 'reject',1,0)) as 'Reject',
SUM(IF(status = 'overtime',1,0)) as 'Overtime'
FROM
user_completed
Group By designer_id
Fiddle Demo
As Jack said It's simply workig with this
SELECT designer_id,
SUM(status = 'accept') as 'Accept',
SUM(status = 'reject') as 'Reject',
SUM(status = 'overtime') as 'Overtime'
FROM
user_completed
Group By designer_id
Fiddle Demo
Try this one it wil work
select designer_id,
count(case status when 'accept'then 1 else null end)as accept,
count(case status when 'reject'then 1 else null end)as reject,
count(case status when 'overtime'then 1 else null end)as overtime
from user_completed group by designer_id
If you don't know how many distinct status you have then you can check this link for solution
Mysql query to dynamically convert rows to columns
e.g you can use something like below
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'SUM(IF(`status` = "', `status`, '",1,0)) AS ', `status`)
) INTO #sql
FROM user_completed;
SET #sql = CONCAT('SELECT designer_id, ', #sql, '
FROM user_completed
GROUP BY designer_id');
SELECT #sql;
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;

Mysql query to pull data in the form of matrix

I have data in database with three columns id, topic, and subtopic. Something like this,
CREATE TABLE Table2 (`id` int, `topic` varchar(5), `subtopic` varchar(6));
INSERT INTO Table2 (`id`, `topic`, `subtopic`) VALUES
(1, 'place', 'paris'),
(1, 'group', 'A'),
(1, 'group', 'B'),
(2, 'place', 'us'),
(2, 'group', 'C'),
(3, 'group', 'A'),
(3, 'water', 'salt'),
(4, 'water', 'sweet'),
(4, 'world', 'ep'),
(5, 'place', 'venus'),
(5, 'place', 'paris'),
(5, 'group', 'A');
I want to output the result matrix place vs group from topic. Something like this.
Paris|US|Venus
A 2 |0 | 1
B 1 |0 | 0
C 0 |1 | 0
Idea is to pick up all the values of 'Group' (A,B,C) and 'Places' (paris,us,venus) in subtopic column. And then find number of co-occurrences with such condition.
Any idea how to solve in MySql?
You will need to join on your table twice to get the group and places, then you can use an aggregate function with a CASE expression to convert the rows into columns:
select g.subtopic as `group`,
sum(case when p.subtopic = 'Paris' then 1 else 0 end) Paris,
sum(case when p.subtopic = 'US' then 1 else 0 end) US,
sum(case when p.subtopic = 'Venus' then 1 else 0 end) Venus
from table2 g
left join table2 p
on g.id = p.id
and p.topic = 'place'
where g.topic = 'group'
group by g.subtopic;
See SQL Fiddle with Demo.
If you are going to have unknown values for the subtopic, then you can use a prepared statement and dynamic SQL:
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT(
'sum(case when p.subtopic = ''',
subtopic,
''' then 1 else 0 end) as `',
subtopic, '`')
) INTO #sql
FROM table2
where topic = 'place';
SET #sql = CONCAT('SELECT g.subtopic as `group`, ', #sql, '
from table2 g
left join table2 p
on g.id = p.id
and p.topic = ''place''
where g.topic = ''group''
group by g.subtopic');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
See SQL Fiddle with Demo