MySQL join two tables with group by in each - mysql

I have two mysql tables with part numbers and qty's. I want to sum each tables qty sum(qty) ... group by partNumber Then join the two tables on the part number.
Sometimes table A will have part numbers that table b does not and vice versa. Below is an image of what I am expecting.
I've tried something like this, but this returns a row for each table and I want it to return 1 combined row
SELECT *, null as macroQty, sum(qty) as cardinalQty
FROM parts.cardinal where fileinfoid IN
(select cardinalFiles from parts.reports where fileinfoid = 418)
GROUP BY partNumber UNION ALL
SELECT *, sum(qty) as macroQty, null as cardinalQty
FROM parts.macro where fileinfoid IN
(select macroFiles from parts.reports where fileinfoid = 418 )
GROUP BY partNumber
I also tried wrapping it in an outer select and grouping by the part number from the outer select like this, but this results in the second inner select being null always
SELECT * FROM (
SELECT *, null as macroQty, sum(qty) as cardinalQty
FROM parts.cardinal where fileinfoid IN
(select cardinalFiles from parts.reports where fileinfoid = 418)
GROUP BY partNumber UNION ALL
SELECT *, sum(qty) as macroQty, null as cardinalQty
FROM parts.macro where fileinfoid IN
(select macroFiles from parts.reports where fileinfoid = 418 )
GROUP BY partNumber
) combined GROUP BY combined.partNumber

One approach would be to identify unique part numbers across the 2 tables (using a UNION with it's applied distinct) and then use correlated sub queries to get the sums. For example
drop table if exists a,b;
create table a(id int,val int);
create table b(id int,val int);
insert into a values(1,10),(1,10),(3,10),(4,10);
insert into b values (2,10),(4,10),(4,10);
select (select sum(a.val) from a where a.id = s.id) aval,
(select sum(b.val) from b where b.id = s.id) bval,
s.id partno
from
(
select id from a
union select id from b
) s
order by s.id;
+------+------+--------+
| aval | bval | partno |
+------+------+--------+
| 20 | NULL | 1 |
| NULL | 10 | 2 |
| 10 | NULL | 3 |
| 10 | 20 | 4 |
+------+------+--------+
4 rows in set (0.00 sec)

I would phrase this as a join between two subqueries which each find the sum in their respective tables. However, since each table does not necessarily contain all part numbers, and in fact there may be part numbers unique to each table, we will have to use a full outer join approach.
SELECT
t1.partNumber,
t1.cardinalQty,
COALECSE(t2.macroQty, 0) AS macroQty
FROM
(
SELECT partNumber, SUM(qty) AS cardinalQty
FROM cardinal
GROUP BY partNumber
) t1
LEFT JOIN
(
SELECT partNumber, SUM(qty) AS macroQty
FROM macro
GROUP BY partNumber
) t2
ON t1.partNumber = t2.partNumber
UNION ALL
SELECT
t2.partNumber,
0 AS cardinalQty,
t2.macroQty
FROM
(
SELECT partNumber, SUM(qty) AS cardinalQty
FROM cardinal
GROUP BY partNumber
) t1
RIGHT JOIN
(
SELECT partNumber, SUM(qty) AS macroQty
FROM macro
GROUP BY partNumber
) t2
ON t1.partNumber = t2.partNumber
WHERE t1.partNumber IS NULL;
Keep in mind that under normal conditions, in a well designed database, you should rarely encounter a situation which requires using a full outer join. Actually, a full outer join screams out that there is a design problem. In this case, you don't have a single parts table containing all part numbers. That table should exist, so unless you enjoy big ugly queries, you should create a parts table where the partNumber is a primary key.

Related

SQL/MySQL DELETE all rows EXCEPT 2 of them

I have a database table setup like this:
id | code | group_id | status ---
---|-------|---------|------------
1 | abcd1 | group_1 | available
2 | abcd2 | group_1 | available
3 | adsd3 | group_1 | available
4 | dfgd4 | group_1 | available
5 | vfcd5 | group_1 | available
6 | bgcd6 | group_2 | available
7 | abcd7 | group_2 | available
8 | ahgf8 | group_2 | available
9 | dfgd9 | group_2 | available
10 | qwer6 | group_2 | available
In the example above, each group_id has 5 total rows (arbitrary for example, total rows will be dynamic and vary), I need to remove every row that matches available in status except for 2 of them (which 2 does not matter, as long as there are 2 of them remaining)
Basically every unique group_id should only have 2 total rows with status of available. I am able to do a simple SQL query to remove all of them, but struggling to come up with a SQL query to remove all except for 2 ... please helppppp :)
If code is unique, you can use subqueries to keep the "min" and "max"
DELETE FROM t
WHERE t.status = 'available'
AND (t.group_id, t.code) NOT IN (
SELECT group_id, MAX(code)
FROM t
WHERE status = 'available'
GROUP BY group_id
)
AND (t.group_id, t.code) NOT IN (
SELECT group_id, MIN(code)
FROM t
WHERE status = 'available'
GROUP BY group_id
)
Similarly, with an auto increment id:
DELETE FROM t
WHERE t.status = 'available'
AND t.id NOT IN (
SELECT MAX(id) FROM t WHERE status = 'available' GROUP BY group_id
UNION
SELECT MIN(id) FROM t WHERE status = 'available' GROUP BY group_id
)
I reworked the subquery into a UNION instead in this version, but the "AND" format would work just as well too. Also, if "code" was unique across the whole table, the NOT IN could be simplified down to excluding the group_id as well (though it would still be needed in the subqueries' GROUP BY clauses).
Edit: MySQL doesn't like subqueries referencing tables being UPDATEd/DELETEd in the WHERE of the query doing the UPDATE/DELETE; in those cases, you can usually double-wrap the subquery to give it an alias, causing MySQL to treat it as a temporary table (behind the scenes).
DELETE FROM t
WHERE t.status = 'available'
AND t.id NOT IN (
SELECT * FROM (
SELECT MAX(id) FROM t WHERE status = 'available' GROUP BY group_id
UNION
SELECT MIN(id) FROM t WHERE status = 'available' GROUP BY group_id
) AS a
)
Another alternative, I don't recall if MySQL complains as much about joins in DELETE/UPDATE....
DELETE t
FROM t
LEFT JOIN (
SELECT MIN(id) AS minId, MAX(id) AS maxId, 1 AS keep_flag
FROM t
WHERE status = 'available'
GROUP BY group_id
) AS tKeep ON t.id IN (tKeep.minId, tKeep.maxId)
WHERE t.status = 'available'
AND tKeep.keep_flag IS NULL
To keep the min and max ids, I think a join is the simplest solution:
DELETE t
FROM t LEFT JOIN
(SELECT group_id, MIN(id) as min_id, MAX(id) as max_id
FROM t
WHERE t.status = 'available'
GROUP BY group_id
) tt
ON t.id IN (tt.min_id, tt.max_id)
WHERE t.status = 'available' AND
tt.group_id IS NULL;
If the column "id" is the PRIMARY KEY or a UNIQUE KEY, then we could use a correlated subquery to get the second lowest value for a particular group_id.
We could then use that to identify rows for group_id that have higher values of the "id" column.
A query something like this:
SELECT t.`id`
, t.`group_id`
FROM `setup_like_this` t
WHERE t.`status` = 'available'
AND t.`id`
> ( SELECT s.`id`
FROM `setup_like_this` s
WHERE s.`status` = 'available'
AND s.`group_id` = t.`group_id`
ORDER
BY s.`id`
LIMIT 1,1
)
We test that as a SELECT first, to examine the rows that are returned. When we are satisfied this query is returning the set of rows we want to delete, we can replace SELECT ... FROM with DELETE t.* FROM to convert it to a DELETE statement to remove the rows.
Error 1093 encountered converting to DELETE statement.
One workaround is to make the query above into a inline view, and then join to the target table
DELETE q.*
FROM `setup_like_this` q
JOIN ( -- inline view, query from above returns `id` of rows we want to delete
SELECT t.`id`
, t.`group_id`
FROM `setup_like_this` t
WHERE t.`status` = 'available'
AND t.`id`
> ( SELECT s.`id`
FROM `setup_like_this` s
WHERE s.`status` = 'available'
AND s.`group_id` = t.`group_id`
ORDER
BY s.`id`
LIMIT 1,1
)
) r
ON r.id = q.id
select id, code, group_id, status
from (
select id, code, group_id, status
, ROW_NUMBER() OVER (
PARTITION BY group_id
ORDER BY id DESC) row_num
) rownum
from a
) q
where rownum < 3

Count group by enum including possible enum values that have 0 count

I have a table of items. One of the fields is a category (represented by an enum). Some categories have zero items.
So I did this:
select category, count(*) as total from items group by category;
+------------+-------+
| category | total |
+------------+-------+
| one | 6675 |
+------------+-------+
I want to generate a table like this (where two is the other possible enum value):
+------------+-------+
| category | total |
+------------+-------+
| one | 6675 |
+------------+-------+
| two | 0 |
+------------+-------+
How do I do this with an mysql SQL query?
Enum datatype is generally preferred for those cases where possible options (values) are not too many (prefer <= 10), and you are not going to add new options in future (atleast not very frequently). So, a good use-case for Enum is gender: (m, f, n). In your case, it would be generally better to have a Master table of all possible Categories, instead of using Enum for them. Then it is easier to do a LEFT JOIN from the Master table.
However, as asked by you:
A solution uses the enum type to generate the table, and includes 0
entries
Works for all MySQL/MariaDB versions:
We will need to get the list of all possible Enum values from INFORMATION_SCHEMA.COLUMNS:
SELECT
SUBSTRING(COLUMN_TYPE, 6, CHAR_LENGTH(COLUMN_TYPE) - 6) AS enum_values
FROM
information_schema.COLUMNS
WHERE
TABLE_NAME = 'items' -- your table name
AND
COLUMN_NAME = 'category' -- name of the column
AND
TABLE_SCHEMA = 'your_db' -- name of the database (schema)
But then, this query will give you all the enum values in comma-separated string, like below:
'one','two','three','four'
Now, we will need to convert this string into multiple rows. To achieve that, we can use a Sequence (Number series) table. You can define a permanent table in your database storing integers ranging from 1 to 100 (you may find this table helpful in many other cases as well) (OR, another approach is to use a Derived Table - check this to get an idea: https://stackoverflow.com/a/58052199/2469308).
CREATE TABLE seq (n tinyint(3) UNSIGNED NOT NULL, PRIMARY KEY(n));
INSERT INTO seq (n) VALUES (1), (2), ...... , (99), (100);
Now, we will do a JOIN between "enum values string" and seq table, based on the position of comma, to extract enum values into different rows. Note that instead of just using , (comma) to extract enum values, we would use ',' (to avoid cases when there might be a comma inside the value string). String operations utilizing Substring_Index(), Trim(), Char_Length() etc functions can be used to extract enum values. You can check this answer to get a general idea about this technique:
Schema (View on DB Fiddle)
CREATE TABLE items
(id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
category ENUM('one','two','three','four'),
item_id INT UNSIGNED) ENGINE=InnoDB;
INSERT INTO items (category, item_id)
VALUES ('one', 1),
('two', 2),
('one', 2),
('one', 3);
CREATE TABLE seq (n tinyint(3) UNSIGNED NOT NULL,
PRIMARY KEY(n));
INSERT INTO seq (n) VALUES (1),(2),(3),(4),(5);
Query #1
SELECT Trim(BOTH '\'' FROM Substring_index(Substring_index(e.enum_values,
'\',\'',
seq.n),
'\',\'', -1)) AS cat
FROM (SELECT Substring(column_type, 6, Char_length(column_type) - 6) AS
enum_values
FROM information_schema.columns
WHERE table_name = 'items'
AND column_name = 'category'
AND table_schema = 'test') AS e
JOIN seq
ON ( Char_length(e.enum_values) - Char_length(REPLACE(e.enum_values,
'\',\'',
''))
) / 3 >= seq.n - 1
| cat |
| ----- |
| one |
| two |
| three |
| four |
Now, the hard part is done. All we need to do is do a LEFT JOIN from this subquery (having all category enum values) to your items table, to get Count per category.
The final query follows (View on DB Fiddle):
SELECT all_cat.cat AS category,
Count(i.item_id) AS total
FROM (SELECT Trim(BOTH '\'' FROM Substring_index(
Substring_index(e.enum_values,
'\',\'',
seq.n),
'\',\'', -1)) AS cat
FROM (SELECT Substring(column_type, 6, Char_length(column_type) - 6)
AS
enum_values
FROM information_schema.columns
WHERE table_name = 'items'
AND column_name = 'category'
AND table_schema = 'test') AS e
JOIN seq
ON ( Char_length(e.enum_values) - Char_length(
REPLACE(e.enum_values,
'\',\'',
''))
) / 3 >= seq.n - 1) AS all_cat
LEFT JOIN items AS i
ON i.category = all_cat.cat
GROUP BY all_cat.cat
ORDER BY total DESC;
Result
| category | total |
| -------- | ----- |
| one | 3 |
| two | 1 |
| three | 0 |
| four | 0 |
Here is some fun with MySQL 8.0 and JSON_TABLE():
select c.category, count(i.category) as total
from information_schema.COLUMNS s
join json_table(
replace(replace(replace(trim('enum' from s.COLUMN_TYPE),'(','['),')',']'),'''','"'),
'$[*]' columns (category varchar(50) path '$')
) c
left join items i on i.category = c.category
where s.TABLE_SCHEMA = 'test' -- replace with your db/schema name
and s.TABLE_NAME = 'items'
and s.COLUMN_NAME = 'category'
group by c.category
It converts the ENUM type definition from information_schema to a JSON array, which is then converted by JSON_TABLE() to a table, which you then can use for a LEFT JOIN.
See demo on db-fiddle
Note: The categories should not contain any characters from ()[]'".
But seriously – Just create the categories table. There are more reasons to do that. For example you might want to render a drop-down menu with all possible categories. That would be simple with
select category from categories
I would say that it's basically bad practice to encode your enumerations into the script. Therefore, create a table with the enumerations present (and their relative keys), then it's a simple case of grouping a left joined query...
SELECT
cat.enum_name,
COUNT(data.id) AS total
FROM
category_table cat
LEFT JOIN
data_table data
ON cat.cate_id = data.cat_id
GROUP BY
cat.enum_name
using in-select subquery
select cat.categoryname
(
select count(*) -- count total
from items as i
where i.category = cat.category -- connect
) as totalcount
from cat
order by cat.categoryname
You can make a fictive dataset of the different categories and do a left join with your original table as shown below.
SELECT A.category, count(*) total FROM
(SELECT 'one' as Category
UNION ALL
SELECT 'two' as Category) A
LEFT JOIN items B
ON A.Category=B.Category
GROUP BY B.Category;
If you would prefer to get a list of all the category dynamically, then save them in another table (say All_category_table) then do a join as shown below:
SELECT A.category, count(*) total FROM
(SELECT Category FROM All_category_table) A
LEFT JOIN items B
ON A.Category=B.Category
GROUP BY B.Category;
This answer is applicable for when you do not have another table holding the possible category values.
Let's say you have a table called real_table with a not null & value constrained column category. In this column you know you can theoretically encounter 5 different values: 'CATEGORY_0', 'CATEGORY_1', 'CATEGORY_2', 'CATEGORY_3', 'CATEGORY_4':
CREATE TABLE real_table
(
id VARCHAR(255) NOT NULL
PRIMARY KEY,
category VARCHAR(255) NOT NULL
CONSTRAINT category_in CHECK (
category in ('CATEGORY_0',
'CATEGORY_1',
'CATEGORY_2',
'CATEGORY_3',
'CATEGORY_4')
)
);
But your actual data set in the table does not include any row with value 'CATEGORY_0'. So when you run a query such as:
SELECT real_table.category AS category, COUNT(*) AS cnt
FROM real_table
GROUP BY real_table.category;
you will see, that you get result like this:
category
cnt
CATEGORY_1
150
CATEGORY_2
20
CATEGORY_3
12
CATEGORY_4
1
Hmm, the 'CATEGORY_0' is omitted. Not good.
Since your categories are not backed by another table, then you must create an artificial dataset of the possible categories that looks as below:
SELECT 'CATEGORY_0' AS category_entry
UNION ALL
SELECT 'CATEGORY_1' AS category_entry
UNION ALL
SELECT 'CATEGORY_2' AS category_entry
UNION ALL
SELECT 'CATEGORY_3' AS category_entry
UNION ALL
SELECT 'CATEGORY_4' AS category_entry;
You can use this in your original query as a table to do a right join on:
SELECT all_categories.category_entry AS category,
COUNT(real_table.id) AS cnt -- important to count some non-null value, such as PK of the real_table
FROM real_table
RIGHT JOIN
(SELECT 'CATEGORY_0' AS category_entry -- not present in any row in table 'all_categories'
UNION ALL
SELECT 'CATEGORY_1' AS category_entry
UNION ALL
SELECT 'CATEGORY_2' AS category_entry
UNION ALL
SELECT 'CATEGORY_3' AS category_entry
UNION ALL
SELECT 'CATEGORY_4' AS category_entry) all_categories
ON real_table.category = all_categories.category_entry
GROUP BY all_categories.category_entry;
Now when you run the query, you should get the desired output:
category
cnt
CATEGORY_0
0
CATEGORY_1
150
CATEGORY_2
20
CATEGORY_3
12
CATEGORY_4
1
The 'CATEGORY_0' is now included with zero cnt. Nice.
Now let's say that the category column is not not null constrained and can also possibly include some other unexpected category values (e.g. 'CATEGORY_66'):
CREATE TABLE real_table
(
id VARCHAR(255) NOT NULL
PRIMARY KEY,
category VARCHAR(255) -- nullable and no constraint for valid values
);
We would like to include these null and unexpected category counts in the result set as well.
Then we must prepare the artificial dataset of the possible categories differently:
SELECT DISTINCT all_categories.category_entry
FROM (SELECT 'CATEGORY_0' AS category_entry -- not present in any row in table 'all_categories'
UNION ALL
SELECT 'CATEGORY_1' AS category_entry
UNION ALL
SELECT 'CATEGORY_2' AS category_entry
UNION ALL
SELECT 'CATEGORY_3' AS category_entry
UNION ALL
SELECT 'CATEGORY_4' AS category_entry
UNION ALL
SELECT DISTINCT category
FROM real_table AS category_entry) all_categories;
and use it as before:
SELECT distinct_categories.category_entry AS category,
COUNT(real_table.id) AS cnt -- important to count some non-null value, such as PK of the real_table
FROM real_table
RIGHT JOIN
(SELECT DISTINCT all_categories.category_entry
FROM (SELECT 'CATEGORY_0' AS category_entry -- not present in any row in table 'all_categories'
UNION ALL
SELECT 'CATEGORY_1' AS category_entry
UNION ALL
SELECT 'CATEGORY_2' AS category_entry
UNION ALL
SELECT 'CATEGORY_3' AS category_entry
UNION ALL
SELECT 'CATEGORY_4' AS category_entry
UNION ALL
SELECT DISTINCT category
FROM real_table AS category_entry) all_categories) distinct_categories
ON real_table.category = distinct_categories.category_entry
GROUP BY distinct_categories.category_entry;
Now when you run the query, the output should also include counts for additional categories and null categories
category
cnt
CATEGORY_0
0
CATEGORY_1
150
CATEGORY_2
20
CATEGORY_3
12
CATEGORY_4
1
CATEGORY_66
13
10
Both unexpected 'CATEGORY_66' (with 13 entries) as well as null category (with 10 entries) are now included in the result set
I cannot vouch for the performance of the provided queries - somebody more experienced might weigh in on that?

SQL Find date range gaps in Table

Good day.
I seem to be struggling with what seems like a simple problem.
I have a table that has a value connected to a date (Monthly) for a finite number of ID's
ie. Table1
ID | Date ---| Value
01 | 2015-01 | val1
01 | 2015-02 | val2
02 | 2015-01 | val1
02 | 2015-03 | val2
So ID: 02 does not have a value for date 2015-02.
I would like to return all ID's and Dates that do not have a value.
Date range is: select distinct date from Table1
I can't seem to think outside the realms of selecting and joining on the same table.
I need to include the ID in my select to I can somehow select the ID and Date range that exists for that ID and compare to the entire date range, to get all the dates for each ID that isn't in the "entire" date range.
Please advise.
Thank you
Not very clear about your last two sentences. But you can play with the following query with different #max_days and #min_date:
-- DROP TABLE table1;
CREATE TABLE table1(ID int not null, `date` date not null, value varchar(64) not null);
INSERT table1(ID,`date`,value)
VALUES (1,'2015-01-01','v1'),(1,'2015-01-02','v2'),(2,'2015-01-01','v1'),(2,'2015-01-03','v2'),(4,'2015-01-01','v1'),(4,'2015-01-04','v2');
SELECT * FROM table1;
SET #day=0;
SET #max_days=5;
SET #min_date='2015-01-01';
SELECT i.ID,d.`date`
FROM (SELECT DISTINCT ID FROM table1) i
CROSS JOIN (
SELECT TIMESTAMPADD(DAY,#day,#min_date) AS `date`,#day:=#day+1 AS day_num
FROM table1 WHERE #day<#max_days) d
LEFT JOIN table1 t
ON t.ID=i.ID
AND t.`date`=d.`date`
WHERE t.`date` IS NULL
ORDER BY i.ID,d.`date`;
I now understand your requirement of dates being taken from the table; you want to find any gaps in the date ranges for each id.
This does what you need, but can probably be improved. Explanation below and you can view a working example.
DROP TABLE IF EXISTS Table1;
DROP TABLE IF EXISTS Year_Month_Calendar;
CREATE TABLE Table1 (
id INTEGER
,date CHAR(7)
,value CHAR(4)
);
INSERT INTO Table1
VALUES
(1,'2015-01','val1')
,(1,'2015-02','val2')
,(2,'2015-01','val1')
,(2,'2015-03','val1');
CREATE TABLE Year_Month_Calendar (
date CHAR(10)
);
INSERT INTO Year_Month_Calendar
VALUES
('2015-01')
,('2015-02')
,('2015-03');
SELECT ID_Year_Month.id, ID_Year_Month.date, Table1.id, Table1.date
FROM (
SELECT Distinct_ID.id, Year_Month_Calendar.date
FROM Year_Month_Calendar
CROSS JOIN
( SELECT DISTINCT id FROM Table1 ) AS Distinct_ID
WHERE Year_Month_Calendar.date >= (SELECT MIN(date) FROM Table1 WHERE id=Distinct_ID.ID)
AND Year_Month_Calendar.date <= (SELECT MAX(date) FROM Table1 WHERE id=Distinct_ID.ID)
) AS ID_Year_Month
LEFT JOIN Table1
ON ID_Year_Month.id = Table1.id AND ID_Year_Month.date = Table1.date
-- WHERE Table1.id IS NULL
ORDER BY ID_Year_Month.id, ID_Year_Month.date
Explanation
You need a calendar table which contains all dates (year/months) to cover the data you are querying.
CREATE TABLE Year_Month_Calendar (
date CHAR(10)
);
INSERT INTO Year_Month_Calendar
VALUES
('2015-01')
,('2015-02')
,('2015-03');
The inner select creates a table with all dates between the min and max date for each id.
SELECT Distinct_ID.id, Year_Month_Calendar.date
FROM Year_Month_Calendar
CROSS JOIN
( SELECT DISTINCT id FROM Table1 ) AS Distinct_ID
WHERE Year_Month_Calendar.date >= (SELECT MIN(date) FROM Table1 WHERE id=Distinct_ID.ID)
AND Year_Month_Calendar.date <= (SELECT MAX(date) FROM Table1 WHERE id=Distinct_ID.ID)
This is then LEFT JOINED to the original table to find the missing rows.
If you only want to return the missing row (my query displays the whole table to show how it works), add a WHERE clause to restrict the output to those rows where an id and date is not returned from Table1
Original answer before comments
You can do this without a tally table, since you say
Date range is: select distinct date from Table1
I've slightly changed the field names to avoid reserved words in SQL.
SELECT id_table.ID, date_table.`year_month`, table1.val
FROM (SELECT DISTINCT ID FROM table1) AS id_table
CROSS JOIN
(SELECT DISTINCT `year_month` FROM table1) AS date_table
LEFT JOIN table1
ON table1.ID=id_table.ID AND table1.`year_month` = date_table.`year_month`
ORDER BY id_table.ID
I've not filtered the results, in order to show how the query is working. To return the rows where only where a date is missing, add WHERE table1.year_month IS NULL to the outer query.
SQL Fiddle
You will need a tally table(s) or month/year tables. So you can then generate all of the potential combinations you want to test with. As far as exactly how to use it your example could use some expanding on such as last 12 months, last3 months, etc. but here is an example that might help you understand what you are looking for:
http://rextester.com/ZDQS5259
CREATE TABLE IF NOT EXISTS Tbl (
ID INTEGER
,Date VARCHAR(10)
,Value VARCHAR(10)
);
INSERT INTO Tbl VALUES
(1,'2015-01','val1')
,(1,'2015-02','val2')
,(2,'2015-01','val1')
,(2,'2015-03','val1');
SELECT yr.YearNumber, mn.MonthNumber, i.Id
FROM
(
SELECT 2016 as YearNumber
UNION SELECT 2015
) yr
CROSS JOIN (
SELECT 1 MonthNumber
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
UNION SELECT 7
UNION SELECT 8
UNION SELECT 9
UNION SELECT 10
UNION SELECT 11
UNION SELECT 12
) mn
CROSS JOIN (
SELECT DISTINCT ID
FROM
Tbl
) i
LEFT JOIN Tbl t
ON yr.YearNumber = CAST(LEFT(t.Date,4) as UNSIGNED)
AND mn.MonthNumber = CAST(RIGHT(t.Date,2) AS UNSIGNED)
AND i.ID = t.ID
WHERE
t.ID IS NULL
The basic idea to determine what you don't know is to generate all possible combinations of something could be. E.g. Year X Month X DISTINCT Id and then join back to figure out what is missing.
Probably not the prettiest but this should work.
select distinct c.ID, c.Date, d.Value
from (select a.ID, b.Date
from (select distinct ID from Table1) as a, (select distinct Date from Table1) as b) as c
left outer join Table1 d on (c.ID = d.ID and c.Date = d.Date)
where d.Value is NULL

Fetch 2nd Higest value from MySql DB with GROUP BY

I have a table tbl_patient and I want to fetch last 2 visit of each patient in order to compare whether patient condition is improving or degrading.
tbl_patient
id | patient_ID | visit_ID | patient_result
1 | 1 | 1 | 5
2 | 2 | 1 | 6
3 | 2 | 3 | 7
4 | 1 | 2 | 3
5 | 2 | 3 | 2
6 | 1 | 3 | 9
I tried the query below to fetch the last visit of each patient as,
SELECT MAX(id), patient_result FROM `tbl_patient` GROUP BY `patient_ID`
Now i want to fetch the 2nd last visit of each patient with query but it give me error
(#1242 - Subquery returns more than 1 row)
SELECT id, patient_result FROM `tbl_patient` WHERE id <(SELECT MAX(id) FROM `tbl_patient` GROUP BY `patient_ID`) GROUP BY `patient_ID`
Where I'm wrong
select p1.patient_id, p2.maxid id1, max(p1.id) id2
from tbl_patient p1
join (select patient_id, max(id) maxid
from tbl_patient
group by patient_id) p2
on p1.patient_id = p2.patient_id and p1.id < p2.maxid
group by p1.patient_id
id11 is the ID of the last visit, id2 is the ID of the 2nd to last visit.
Your first query doesn't get the last visits, since it gives results 5 and 6 instead of 2 and 9.
You can try this query:
SELECT patient_ID,visit_ID,patient_result
FROM tbl_patient
where id in (
select max(id)
from tbl_patient
GROUP BY patient_ID)
union
SELECT patient_ID,visit_ID,patient_result
FROM tbl_patient
where id in (
select max(id)
from tbl_patient
where id not in (
select max(id)
from tbl_patient
GROUP BY patient_ID)
GROUP BY patient_ID)
order by 1,2
SELECT id, patient_result FROM `tbl_patient` t1
JOIN (SELECT MAX(id) as max, patient_ID FROM `tbl_patient` GROUP BY `patient_ID`) t2
ON t1.patient_ID = t2.patient_ID
WHERE id <max GROUP BY t1.`patient_ID`
There are a couple of approaches to getting the specified resultset returned in a single SQL statement.
Unfortunately, most of those approaches yield rather unwieldy statements.
The more elegant looking statements tend to come with poor (or unbearable) performance when dealing with large sets. And the statements that tend to have better performance are more un-elegant looking.
Three of the most common approaches make use of:
correlated subquery
inequality join (nearly a Cartesian product)
two passes over the data
Here's an approach that uses two passes over the data, using MySQL user variables, which basically emulates the analytic RANK() OVER(PARTITION ...) function available in other DBMS:
SELECT t.id
, t.patient_id
, t.visit_id
, t.patient_result
FROM (
SELECT p.id
, p.patient_id
, p.visit_id
, p.patient_result
, #rn := if(#prev_patient_id = patient_id, #rn + 1, 1) AS rn
, #prev_patient_id := patient_id AS prev_patient_id
FROM tbl_patients p
JOIN (SELECT #rn := 0, #prev_patient_id := NULL) i
ORDER BY p.patient_id DESC, p.id DESC
) t
WHERE t.rn <= 2
Note that this involves an inline view, which means there's going to be a pass over all the data in the table to create a "derived tabled". Then, the outer query will run against the derived table. So, this is essentially two passes over the data.
This query can be tweaked a bit to improve performance, by eliminating the duplicated value of the patient_id column returned by the inline view. But I show it as above, so we can better understand what is happening.
This approach can be rather expensive on large sets, but is generally MUCH more efficient than some of the other approaches.
Note also that this query will return a row for a patient_id if there is only one id value exists for that patient; it does not restrict the return to just those patients that have at least two rows.
It's also possible to get an equivalent resultset with a correlated subquery:
SELECT t.id
, t.patient_id
, t.visit_id
, t.patient_result
FROM tbl_patients t
WHERE ( SELECT COUNT(1) AS cnt
FROM tbl_patients p
WHERE p.patient_id = t.patient_id
AND p.id >= t.id
) <= 2
ORDER BY t.patient_id ASC, t.id ASC
Note that this is making use of a "dependent subquery", which basically means that for each row returned from t, MySQL is effectively running another query against the database. So, this will tend to be very expensive (in terms of elapsed time) on large sets.
As another approach, if there are relatively few id values for each patient, you might be able to get by with an inequality join:
SELECT t.id
, t.patient_id
, t.visit_id
, t.patient_result
FROM tbl_patients t
LEFT
JOIN tbl_patients p
ON p.patient_id = t.patient_id
AND t.id < p.id
GROUP
BY t.id
, t.patient_id
, t.visit_id
, t.patient_result
HAVING COUNT(1) <= 2
Note that this will create a nearly Cartesian product for each patient. For a limited number of id values for each patient, this won't be too bad. But if a patient has hundreds of id values, the intermediate result can be huge, on the order of (O)n**2.
Try this..
SELECT id, patient_result FROM tbl_patient AS tp WHERE id < ((SELECT MAX(id) FROM tbl_patient AS tp_max WHERE tp_max.patient_ID = tp.patient_ID) - 1) GROUP BY patient_ID
Why not use simply...
GROUP BY `patient_ID` DESC LIMIT 2
... and do the rest in the next step?

MySQL - count values and merge results from multiple tables

I have two tables with one column each, containing names.
Names can have duplicates. One name can be found on every table or only in one.
I want to make an query that count duplicates, for each name in every table an list these values like this:
| name | table1 | table2 |
| john | 12 | 23 |
| mark | 2 | 5 |
| mary | | 10 |
| luke | 4 | |
I tried different strategies using UNION but no luck.
Thanks in advance!!!
SELECT DISTINCT t1.name, t1.cnt1, t2.cnt2
FROM
(SELECT name,count(name) as cnt1 FROM table1 GROUP BY name) t1
LEFT JOIN
(SELECT name,count(name) as cnt2 FROM table2 GROUP BY name) t2
ON t1.name = t2.name
UNION
SELECT DISTINCT t2.name, t1.cnt1, t2.cnt2
FROM
(SELECT name,count(name) as cnt1 FROM table1 GROUP BY name) t1
RIGHT JOIN
(SELECT name,count(name) as cnt2 FROM table2 GROUP BY name) t2
ON t1.name = t2.name
Here's a simpler solution:
You can UNION the names from the two tables together, manually differentiating their origin tables with a tbl column.
Then it's just a simple GROUP BY with conditional aggregation using the differentiating column:
SELECT a.name,
NULLIF(COUNT(CASE a.tbl WHEN 1 THEN 1 END), 0) AS table1,
NULLIF(COUNT(CASE a.tbl WHEN 2 THEN 1 END), 0) AS table2
FROM
(
SELECT name, 1 AS tbl FROM table1 UNION ALL
SELECT name, 2 FROM table2
) a
GROUP BY a.name
In accordance with your desired result-set, we NULL the count value if it turns out to be 0.
SQLFiddle Demo
SELECT SUM(res.cn), name
FROM
(
SELECT name, count(name) as cn from table1 GROUP BY name HAVING count(name) > 1
UNION ALL
SELECT name, count(name) as cn from table2 GROUP BY name HAVING count(name)>1
) as res
GROUP BY nam
e
Try the above :) I made a fiddle for you to test it:
http://sqlfiddle.com/#!3/796b2/3
It has a few double names in each table and will show you which names have doubles and then print them. The names that only appear once are not shown (acheived by the HAVING clause)
After some reading i don't think that it's posibil what i want to do. This situation ca be solved with pivot table in excel or libreoffice.
In fact this is method that i used, combined with some sql stataments to count occurence of names and export as CSV.
UNION definitetly not work. Some chance are with join, but not shure.
I found a post that discusses the same problem as mine.
MySQL - Rows to Columns