Pivot Table Omitting Rows that Have Null values - mysql

I am solving a problem very similar to this only in my case, I am not summing any values.
I have been able to write a select that works using solution from this page
SELECT
id,
GROUP_CONCAT(if(colID = 1, value, NULL)) AS 'First Name',
GROUP_CONCAT(if(colID = 2, value, NULL)) AS 'Last Name',
GROUP_CONCAT(if(colID = 3, value, NULL)) AS 'Job Title'
FROM tbl
GROUP BY id;
However, I want to omit rows that have the value to be null

I assume you want to drop the result row if any of the source rows has value IS NULL.
You should be able to achieve that with bit_and() in the HAVING clause:
SELECT id
, max(CASE WHEN colID = 1 THEN value END) AS fn
, max(CASE WHEN colID = 2 THEN value END) AS ln
, max(CASE WHEN colID = 3 THEN value END) AS jt
FROM tbl
GROUP BY id
HAVING bit_and(value IS NOT NULL);
Alternative:
...
HAVING count(*) = count(value);
I didn't spell out ELSE NULL in the CASE statements because (per documentation):
If there was no matching result value, the result after ELSE is returned, or NULL if there is no ELSE part.

Just add this constraint to the where statement of your query, like this:
SELECT
id,
GROUP_CONCAT(if(colID = 1, value, NULL)) AS 'First Name',
GROUP_CONCAT(if(colID = 2, value, NULL)) AS 'Last Name',
GROUP_CONCAT(if(colID = 3, value, NULL)) AS 'Job Title'
FROM tbl
WHERE value IS NOT NULL
GROUP BY id;
EDIT
After some tests I could make a solution to work, but it seems interesting why value is not null won't work.
Solution link: http://sqlfiddle.com/#!2/b7a445/3
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
where not exists (select * from tbl b where tbl.id=b.id and value is null)
group by id

Related

group by picking columns with null value - mysql

I am working on a query and want to group the rows and return groupped data but my query is not working as expected.
my query-
select item, branch, packunit,packlevel,dealqty,PromotionFlag,PromotionID, PromotionEndDate, cnc,delivery, volumedeal, standard_price_scheme,
deliv_price_scheme
from
(
SELECT
`item` AS `item`,
`branch` AS `branch`,
`PackUnit` AS `PackUnit`,
`PackLevel` AS `PackLevel`,
`DealQty` AS `DealQty`,
`PromotionFlag` AS `PromotionFlag`,
`PromotionID` AS `PromotionID`,
`PromotionEndDate` AS `PromotionEndDate`,
SUM(`cnc`) AS `cnc`,
SUM(`delivery`) AS `delivery`,
SUM(`volumedeal`) AS `volumedeal`,
`standard_price_scheme` AS `standard_price_scheme`,
`deliv_price_scheme` AS `deliv_price_scheme`
FROM
(
SELECT DISTINCT
`Pricing_Today`.`item` AS `item`,
`Pricing_Today`.`branch` AS `branch`,
`Pricing_Today`.`price_scheme` AS `price_scheme`,
`Pricing_Today`.`PackUnit` AS `PackUnit`,
`Pricing_Today`.`PackLevel` AS `PackLevel`,
`Pricing_Today`.`DealQty` AS `DealQty`,
`Pricing_Today`.`PromotionFlag` AS `PromotionFlag`,
`Pricing_Today`.`PromotionID` AS `PromotionID`,
`Pricing_Today`.`PromotionEndDate` AS `PromotionEndDate`,
(CASE
WHEN (`Pricing_Today`.`PriceType` = 'C&C') THEN `Pricing_Today`.`Sell`
END) AS `cnc`,
(CASE
WHEN (`Pricing_Today`.`PriceType` = 'Delivery') THEN `Pricing_Today`.`Sell`
END) AS `delivery`,
(CASE
WHEN (`Pricing_Today`.`PriceType` = 'Volume Deal') THEN `Pricing_Today`.`Sell`
END) AS `volumedeal`,
(CASE
WHEN (`Pricing_Today`.`PriceType` = 'C&C') THEN `Pricing_Today`.`price_scheme`
END) AS `standard_price_scheme`,
(CASE
WHEN
((`Pricing_Today`.`PriceType` = 'Delivery')
OR (`Pricing_Today`.`PriceType` = 'Volume Deal'))
THEN
`Pricing_Today`.`price_scheme`
END) AS `deliv_price_scheme`
FROM
`Pricing_Today`
where item = 78867
and branch = 0
GROUP BY `Pricing_Today`.`item` , `Pricing_Today`.`PackUnit` , `Pricing_Today`.`PriceType`,`standard_price_scheme`,`deliv_price_scheme`
) as a
GROUP BY branch,`item` , `PackUnit`,`standard_price_scheme`,`deliv_price_scheme`
) as a
-- group by item, packunit
which returns -
BUT, when I group by item, pack I get this -
for cnc its showing null values. How do I eliminate null values and get the numbers?
Thanks in advance
You need to take aggregates of the CASE expressions:
SELECT
p.item,
p.branch,
p.price_scheme,
p.PackUnit,
p.PackLevel,
p.DealQty,
p.PromotionFlag,
p.PromotionID,
p.PromotionEndDate,
MAX(CASE WHEN p.PriceType = 'C&C' THEN p.Sell END) AS cnc,
MAX(CASE WHEN p.PriceType = 'Delivery' THEN p.Sell END) AS delivery,
MAX(CASE WHEN p.PriceType = 'Volume Deal' THEN p.Sell END) AS volumedeal,
MAX(CASE WHEN p.PriceType = 'C&C' THEN p.price_scheme END) AS standard_price_scheme,
MAX(CASE WHEN p.PriceType = 'Delivery' OR p.PriceType = 'Volume Deal'
THEN p.price_scheme END) AS deliv_price_scheme
FROM
Pricing_Today p
WHERE
item = 78867 AND branch = 0
GROUP BY
p.item,
p.branch,
p.price_scheme,
p.PackUnit,
p.PackLevel,
p.DealQty,
p.PromotionFlag,
p.PromotionID,
p.PromotionEndDate;
This is just a standard pivot query. The idea behind taking the MAX of a CASE expression is that if a given group of records has a single non NULL value, then MAX would correctly extract it. This works because MAX ignores NULL values.
Note that I removed the backticks from your query, none of which were necessary. I try to avoid using backticks unless they are really needed, because it makes the query harder to read.

One query to fetch from A, join field values from B as row fields in A and then filter and sort?

I have created the following tables in my database for the purpose of creating an easy-to-reuse-and-configure CMS. I can create a PHP array of objects with any number of any named properties and with this structure it can be saved without having to make tables for every object type. A bit like Wordpress uses the 'posts' table for every custom item type.
struct (id, type)
1, 'article'
2, 'article'
3, 'employee'
4, 'article'
...
struct_fields (id, struct_id, sub_id, type, val)
1, 1, 'title', 'string', 'Great article'
2, 1, 'publish_date', 'datetime', '2019-03-02 21:00:00'
3, 1, 'active', 'boolean', 1
4, 1, 'body', 'wysiwyg', '<p>Contents</p>'
5, 2, 'title', 'string', 'Another fantastic article'
6, 2, 'publish_date', 'datetime', '2018-03-01 14:32:17'
7, 2, 'active', 'boolean', 1
8, 2, 'body', 'wysiwyg', '<p>Cool contents</p>'
9, 3, 'firstname', 'string', 'Jos'
10, 3, 'lastname', 'string', 'Fabre'
11, 4, 'title', 'string', 'This is amazing'
12, 4, 'publish_date', 'datetime', '2018-03-01 19:21:34'
13, 4, 'active', 'boolean', 0
14, 4, 'body', 'wysiwyg', '<p>Bad content!</p>'
...
I'm seriously struggling to create the one query that can take out a list of structs with their related fields as properties.
I'm not sure it's even possible.
I would like to sort on properties (thus fields in the related table) as well
e.g. I would like to create a query that can grab the latest 3 active articles of which publish_date is in the past.
I've struggled with JOIN and GROUP BY for days now.
The query for a different type of struct could be ordered by a specific field, or filtered on another field, or both..
So, e.g. for articles I would like the output to be this:
[id, active, publish_date, title, body]
And then filter on active and sort on publish_date. The id would come from the struct table, the rest of the fields from the joined fields in the struct_fields table.
And for employees it should be this:
[id, firstname, lastname]
And then sorted on lastname ASC, firstname ASC
In one case I was able to get the struct_field properties into the result row fields with MAX, but I could not sort on them without sql errors
SELECT struct.id, struct.type, struct.published,
MAX(CASE WHEN struct_fields.sub_id = 'active' THEN struct_fields.vc1 ELSE NULL END) AS active,
MAX(CASE WHEN struct_fields.sub_id = 'title' THEN struct_fields.vc1 ELSE NULL END) AS title,
MAX(CASE WHEN struct_fields.sub_id = 'body' THEN struct_fields.tx1 ELSE NULL END) AS body,
MAX(CASE WHEN struct_fields.sub_id = 'allday' THEN struct_fields.vc1 ELSE NULL END) AS allday,
MAX(CASE WHEN struct_fields.sub_id = 'start_datetime' THEN struct_fields.vc1 ELSE NULL END) AS start_datetime,
MAX(CASE WHEN struct_fields.sub_id = 'end_datetime' THEN struct_fields.vc1 ELSE NULL END) AS end_datetime
FROM struct
INNER JOIN struct_fields ON (struct_fields.struct_id = struct.id)
WHERE struct.type = 'events'
GROUP BY struct.id
Help! Can this even be done in one query??
UPDATE: #Nic3500 suggested posting create sql and some sample inserts, so here they are: https://pastebin.com/raw/0hPYByDq
This could happen if you use alias name for order by
but you could use also the column position eg:
SELECT struct.id, struct.type, struct.published,
MAX(CASE WHEN struct_fields.sub_id = 'active' THEN struct_fields.vc1 ELSE NULL END) AS active,
MAX(CASE WHEN struct_fields.sub_id = 'title' THEN struct_fields.vc1 ELSE NULL END) AS title,
MAX(CASE WHEN struct_fields.sub_id = 'body' THEN struct_fields.tx1 ELSE NULL END) AS body,
MAX(CASE WHEN struct_fields.sub_id = 'allday' THEN struct_fields.vc1 ELSE NULL END) AS allday,
MAX(CASE WHEN struct_fields.sub_id = 'start_datetime' THEN struct_fields.vc1 ELSE NULL END) AS start_datetime,
MAX(CASE WHEN struct_fields.sub_id = 'end_datetime' THEN struct_fields.vc1 ELSE NULL END) AS end_datetime
FROM struct
INNER JOIN struct_fields ON (struct_fields.struct_id = struct.id)
WHERE struct.type = 'events'
GROUP BY struct.id
order 1,2,3
in this way you should be a bit more indendent and you could build a dynamic range or set for you order by
I think -probably due to actively diving into this again after getting your answers- I managed to come up with a solution in this query (in the case of struct type = articles)
SELECT struct.id, struct.type, struct.published,
MAX(CASE WHEN struct_fields.sub_id = 'active' THEN struct_fields.vc1 ELSE NULL END) AS active,
MAX(CASE WHEN struct_fields.sub_id = 'title' THEN struct_fields.vc1 ELSE NULL END) AS title,
MAX(CASE WHEN struct_fields.sub_id = 'body' THEN struct_fields.tx1 ELSE NULL END) AS body,
MAX(CASE WHEN struct_fields.sub_id = 'publish_date' THEN struct_fields.vc1 ELSE NULL END) AS publish_date
FROM struct
INNER JOIN struct_fields ON (struct_fields.struct_id = struct.id)
WHERE struct.type = 'articles'
GROUP BY struct.id
HAVING (active = 1 AND CAST(publish_date as datetime)<NOW() )
ORDER BY publish_date DESC
LIMIT 7

MySQL - Rows to Columns and keeps NULL

We have the following table (TEST2) in the MySQL database (MySQL 5.6):
TEAM_ID,MEMBER_ID,TYPE,SCORE
1,2,A,150
1,3,B,200
1,1,B,50
1,1,A,100
1,2,B,NULL
We try to transform/pivot the above table based on the TYPE column:
If the TYPE column has value == A, move the value in the SCORE column into a new column called A_SCORE. If the value in the SCORE column is NULL, it should show NULL in the new A_SCORE column.
If the TYPE column has value == B, move the value in the SCORE column into a new column called B_SCORE. If the value in the SCORE column is NULL, it should show NULL in the new B_SCORE column.
The following table is the one we are looking for (the wanted table):
TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE,A_SCORE_MINUS_B_SCORE
1,1,100,50,50
1,2,150,NULL,NULL
1,3,0,200,-200
We tried the following query
SELECT TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE,SUM(A_SCORE-B_SCORE) AS ACTUAL_MINUS_B_SCORE FROM
(SELECT TEAM_ID,MEMBER_ID,
CASE
WHEN SCORE IS NULL
THEN NULL
ELSE SUM(if(TYPE = 'A', SCORE,0) )
END A_SCORE,
CASE
WHEN SCORE IS NULL
THEN NULL
ELSE SUM(if(TYPE = 'B', SCORE,0) )
END B_SCORE
FROM TEST2
GROUP BY TEAM_ID,MEMBER_ID,SCORE) AS A
GROUP BY TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE);
It returns something we don’t want:
TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE,A_SCORE_MINUS_B_SCORE
1,1,0,50,-50
1,1,100,0,100
1,2,0,0,0
1,2,150,0,150
1,3,0,200,-200
If we tried the following, it generates a table close to what we want, but it doesn’t return any NULL value.
SELECT TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE,SUM(A_SCORE-B_SCORE) AS A_SCORE _MINUS_B_SCORE FROM
(SELECT TEAM_ID,MEMBER_ID,
SUM(if(TYPE = 'A', SCORE,0) ) AS A_SCORE,
SUM(if(TYPE = 'B', SCORE,0) )AS B_SCORE
FROM TEST2
GROUP BY TEAM_ID,MEMBER_ID) AS A
GROUP BY TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE;
The result of the above query:
TEAM_ID,MEMBER_ID,A_SCORE,B_SCORE,A_SCORE_MINUS_B_SCORE
1,1,100,50,50
1,2,150,0,0
1,3,0,200,-200
Could any guru enlighten how to generate the wanted table in this case using MySQL? The SQL fiddle is here for your convenience.
http://sqlfiddle.com/#!9/cfe7a1/1
Thanks!
Try this;)
SELECT TEAM_ID, MEMBER_ID, A_SCORE, B_SCORE, A_SCORE - B_SCORE AS A_SCORE_MINUS_B_SCORE
FROM (
SELECT
TEAM_ID, MEMBER_ID,
CASE
WHEN A_SCORE IS NULL AND NOT EXISTS (
SELECT 1 FROM TEST2
WHERE TEAM_ID = T1.TEAM_ID
AND MEMBER_ID = T1.MEMBER_ID
AND TYPE = 'A'
) THEN 0 ELSE A_SCORE END AS A_SCORE,
CASE
WHEN B_SCORE IS NULL AND NOT EXISTS (
SELECT 1 FROM TEST2
WHERE TEAM_ID = T1.TEAM_ID
AND MEMBER_ID = T1.MEMBER_ID
AND TYPE = 'A'
) THEN 0 ELSE B_SCORE END AS B_SCORE
FROM (
SELECT
TEAM_ID, MEMBER_ID,
MAX(CASE WHEN TYPE = 'A' THEN SCORE END) AS A_SCORE,
MAX(CASE WHEN TYPE = 'B' THEN SCORE END) AS B_SCORE
FROM TEST2
GROUP BY TEAM_ID, MEMBER_ID
) T1
)T
SQLFiddle demo here
I don't quite understand the calculation criteria, but something like this should work...
SELECT team_id
, member_id
, COALESCE(MAX(CASE WHEN type = 'A' THEN score END),0) a_score
, COALESCE(MAX(CASE WHEN type = 'B' THEN score END),0) b_score
, COALESCE(MAX(CASE WHEN type = 'A' THEN score END),0)
- COALESCE(MAX(CASE WHEN type = 'B' THEN score END),0) diff
FROM test2
GROUP
BY team_id
, member_id;

MySql, Postgres, Oracle and SQLServer ignoring IS NOT NULL filter

While I was preparing an answer to one of our fellows here on SO I've encounter an odd situation, at least to me. The original question is here: Pivot Table Omitting Rows that Have Null values
I've modified the query to use max instead of group_concat in order to show the "problem" in all databases.
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
The result of this query is this:
ID FN LN JT
1 Sampo Kallinen Office Manager
2 Jakko Salovaara Vice President
3 (null) Foo No First Name
The user asks to filter the row with id 3 because the field value is null.
When it seems pretty obvious that only it needs to do was to add a WHERE value IS NOT NULL constraint on that query to achieve what the user expect. It won't work.
So I start to test it on the other databases to see what happens (Queries with the WHERE CLAUSE)
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
WHERE value is not null
GROUP BY id
Mysql: http://sqlfiddle.com/#!2/78395/1
Postgres: http://sqlfiddle.com/#!15/78395/1
SQLServer: http://sqlfiddle.com/#!6/78395/1
Oracle: http://sqlfiddle.com/#!4/78395/1
For my surprise the result was the same, none worked.
Then I tried a different version of the same query:
SELECT * FROM (
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
) T
WHERE fn IS NOT NULL
AND ln IS NOT NULL
AND jt IS NOT NULL
Oracle: http://sqlfiddle.com/#!4/78395/2 WORKED
MySql: http://sqlfiddle.com/#!2/78395/2
Postgres: http://sqlfiddle.com/#!15/78395/2
SQLServer: http://sqlfiddle.com/#!6/78395/2
The only way I could make it work on all databases was with this query:
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
WHERE NOT EXISTS (SELECT * FROM tbl b WHERE tbl.id=b.id AND value IS NULL)
GROUP BY id
So I ask:
What is happening here that except for that specific case on Oracle all other DBs seem to ignore the IS NOT NULL filter?
To omit the row from the result if any of the source rows for the same id has value IS NULL, a solution in Postgres would be to use the aggregate function every() or (synonym for historical reasons) bool_and() in the HAVING clause:
SELECT id
, max(case when colID = 1 then value else '' end) AS fn
, max(case when colID = 2 then value else '' end) AS ln
, max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
HAVING every(value IS NOT NULL);
SQL Fiddle.
Explain
Your attempt with a WHERE clause would just eliminate one source row for id = 3 in your example (the one with colID = 1), leaving two more for the same id. So we still get a row for id = 3 in the result after aggregating.
But since we have no row with colID = 1, we get an empty string (note: not a NULL value!) for fn in the result for id = 3.
A faster solution in Postgres would be to use crosstab(). Details:
PostgreSQL Crosstab Query
Other RDBMS
While EVERY is defined in the SQL:2008 standard, many RDBMS do not support it, presumably because some of them have shady implementations of the boolean type. (Not dropping any names like "MySQL" or "Oracle" ...). You can probably substitute everywhere (including Postgres) with:
SELECT id
, max(case when colID = 1 then value else '' end) AS fn
, max(case when colID = 2 then value else '' end) AS ln
, max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
HAVING count(*) = count(value);
Because count() doesn't count NULL values. In MySQL there is also bit_and().
More under this related question:
Is there any equivalent to Postgresql EVERY aggregate function on other RDBMS?
It works in Oracle because Oracle handles NULL incorrectly in that NULL and '' are the same. The other databases don't do this because it is wrong. NULL is unknown, versus '' which is just a blank, empty string.
So if your where clause said something like WHERE (fn IS NOT NULL or fn <> '') you would probably get further.
I think this is a case where a HAVING clause will do what you need.
SELECT id, max ... (same stuff as before)
FROM tbl
GROUP by id
HAVING fn IS NOT NULL
AND ln IS NOT NULL
AND jt IS NOT NULL

Multiple count on same column and insert into another table

I have a student table that has three columns
1. Student Name
2. Class Name
3. Test result
A student takes more than one tests with different results. I am trying to get the data into another table that has
1. Stundent Name+CLass Name ( Concatenated )
2. Pass (No of tests Passed)
3. Fail (No of tests failed)
4. Absent (No of tests Absent)
I use
select count(*)
from Student
where Result in ('Passed')
group by StuName, ClassName;
to get the count of passed subject for each stu+class combination. Similarly for failed and absent tests.
How should I modify the code to make an insert into the Table two??
MySQL supports inline IF statement,
SELECT CONCAT(StudentName, ' ', ClassName) Student_Class,
SUM(test = 'PASSED') totalPass,
SUM(test = 'Failed') totalFailed,
SUM(test = 'Absent') totalAbsent
FROM student
GROUP BY CONCAT(StudentName, ' ', ClassName)
and when you want to insert the result from the query above, use INSERT INTO..SELECT statement,
INSERT INTO tableNAME(col1, totalPass, totalFailed, totalAbsent)
SELECT CONCAT(StudentName, ' ', ClassName) Student_Class,
SUM(test = 'PASSED') totalPass,
SUM(test = 'Failed') totalFailed,
SUM(test = 'Absent') totalAbsent
FROM student
GROUP BY CONCAT(StudentName, ' ', ClassName)
You can easily pivot the data using an aggregate function with a CASE:
select concat(StuName, ',', ClassName) StuNameClass,
sum(case when result = 'Passed' then 1 else 0 end) Passed,
sum(case when result = 'Fail' then 1 else 0 end) Fail,
sum(case when result = 'Absent' then 1 else 0 end) Absent
from Student
group by concat(StuName, ',', ClassName);
Then if you want to insert the data into your other table:
insert into Table2 (StudentClassName, Passed, Fail, Absent)
select concat(StuName, ',', ClassName) StuNameClass,
sum(case when result = 'Passed' then 1 else 0 end) Passed,
sum(case when result = 'Fail' then 1 else 0 end) Fail,
sum(case when result = 'Absent' then 1 else 0 end) Absent
from Student
group by concat(StuName, ',', ClassName);
INSERT INTO t (name_class, passed_count, failed_count, absent_count)
SELECT CONCAT(StuName, ' ', ClassName) AS name_class,
SUM(IF(Result='Passed', 1, 0)) AS passed_count,
SUM(IF(Result='Failed', 1, 0)) AS failed_count,
SUM(IF(Result='Absent', 1, 0)) AS absent_count
FROM Student
GROUP BY StuName, ClassName;