Using MySQL 5.7.17-log
We have a number of report queries using a dependent subquery in the select statement that is returning null when there should be a value. I know this style can be refactored into a left join but I wanted to understand the reason we are having this problem and possibly file a bug with MySQL. Here is the simplest way to reproduce this:
CREATE TABLE tableA (
id varchar(36) NOT NULL,
col2 int(10) unsigned zerofill NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
;
CREATE TABLE tableB (
id varchar(36) NOT NULL,
col2 int(10) unsigned zerofill NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE TABLE tableC (
refA varchar(36) NOT NULL,
refB varchar(36) NOT NULL,
PRIMARY KEY (refA, refB),
CONSTRAINT fkCA FOREIGN KEY (refA) REFERENCES tableA (id),
CONSTRAINT fkCB FOREIGN KEY (refB) REFERENCES tableB (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE TABLE tableD (
refA varchar(36) NOT NULL,
refB varchar(36) NOT NULL,
col3 int(10) unsigned zerofill NOT NULL,
PRIMARY KEY (refA, refB),
CONSTRAINT fkDA FOREIGN KEY (refA) REFERENCES tableA (id),
CONSTRAINT fkDB FOREIGN KEY (refB) REFERENCES tableB (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
insert into tableA values ('123',5)
;
insert into tableB values ('234',6)
;
insert into tableC values ('123','234')
;
insert into tableD values ('123','234',7)
;
select a.id as 'aid',
b.id as 'bib',
( select d.col3 from tableD d where d.refA=a.id and d.refB=b.id ) as 'shouldBe7'
from tableA a
inner join tableC c on c.refA = a.id
inner join tableB b on c.refB = b.id
where a.id='123'
group by a.id, b.id
;
The select statement produces ['123','234',null] although I would expect ['123','234',7]. If you remove the where clause in the statement (where a.id='123'), the expected result - ['123','234',7] is what you get.
I did an explain on the whole select statement and saw that the "ref" uses a func which I did a SHOW WARNINGS and the dependent subquery is comparing the id against an empty string.
(/* select#2 */ select dev.d.col3 from dev.tableD d where ((dev.d.refA = '') and (dev.d.refB = dev.b.id))) AS shouldBe7
Seems like a 5.7 bug...I've filed http://bugs.mysql.com/87915 . the db-fiddle tool was helpful - thanks James!
This ran out of the box for me on 10.1.25-MariaDB locally:
Tried this 5.6 fiddle and it worked.
Tried this 5.7 fiddle and it also worked
I had a look at your code but couldn't see any obvious causes of this error, have you tried on other 5.7 instances to rule out any possible local causes?
Regards,
James
Try this instead...there's always more than one way to skin a cat.
select a.id as 'aid',
b.id as 'bib',
d.col3 as 'shouldBe7'
from tableA a
inner join tableC c on c.refA = a.id
inner join tableB b on c.refB = b.id
inner join ( select col3, refA, refB from tableD ) as d
on d.refA=a.id and d.refB=b.id
where a.id='123'
group by a.id, b.id
;
Related
I am trying to execute the following query
SELECT `id`,
`name`,
`ownerid`,
`creationdata`,
`motd`,
(SELECT Count(*)
FROM guild_membership a,
players_online b
WHERE a.player_id = b.player_id
AND a.guild_id = id) AS `online`,
(SELECT Max(b.level)
FROM guild_membership a,
players b
WHERE a.player_id = b.id
AND a.guild_id = id) AS `toplevel`,
(SELECT Min(a.level)
FROM players a,
guild_membership b
WHERE a.id = b.player_id
AND b.guild_id = id) AS `lowlevel`
FROM `guilds`
WHERE `name` = 'Wideswing Poleaxe'
LIMIT 1;
The tables used in here are the followin
CREATE TABLE IF NOT EXISTS `players` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
`group_id` int(11) NOT NULL DEFAULT '1',
`account_id` int(11) NOT NULL DEFAULT '0',
`level` int(11) NOT NULL DEFAULT '1',
...
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`),
FOREIGN KEY (`account_id`) REFERENCES `accounts` (`id`) ON DELETE CASCADE,
KEY `vocation` (`vocation`)
) ENGINE=InnoDB;
CREATE TABLE IF NOT EXISTS `guilds` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
`ownerid` int(11) NOT NULL,
`creationdata` int(11) NOT NULL,
`motd` varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
UNIQUE KEY (`name`),
UNIQUE KEY (`ownerid`),
FOREIGN KEY (`ownerid`) REFERENCES `players`(`id`) ON DELETE CASCADE
) ENGINE=InnoDB;
CREATE TABLE IF NOT EXISTS `guild_membership` (
`player_id` int(11) NOT NULL,
`guild_id` int(11) NOT NULL,
`rank_id` int(11) NOT NULL,
`nick` varchar(15) NOT NULL DEFAULT '',
PRIMARY KEY (`player_id`),
FOREIGN KEY (`player_id`) REFERENCES `players` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
FOREIGN KEY (`guild_id`) REFERENCES `guilds` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
FOREIGN KEY (`rank_id`) REFERENCES `guild_ranks` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB;
I am trying to get the MAX level and MIN level on the players table inside one guild
However I am always getting toplevel and lowlevel the same value and tis always the lowest level
I am not sure what is wrong
First thing I notice is: you are using LIMIT without ORDER BY. So from the guilds table you expect to find more than one entry for name = 'Wideswing Poleaxe', but only look at the first the DBMS happens to find. Is this desired?
Next thing I see is the out-dated join syntax. Where did you get this from? A twenty year old book? No, stop, twenty years ago this syntax was already made redundant, so it must be even older ;-) Use explicit joins instead (JOIN ... ON ...)
As to your subqueries: You are comparing with id without any qualifier, so the DBMS will take this to be guild_membership.id or players_online resp. players.id, where you really want it to be guild.id. This should explain that you get unexpected values.
As to how the query is built: You could join to the aggregated player data instead. And use alias names that match the tables.
select
guilds.id,
guilds.name,
guilds.ownerid,
guilds.creationdata,
guilds.motd,
players.online,
players.toplevel,
players.lowlevel
from guilds
left join
(
select
gms.guild_id,
max(p.level) as toplevel,
min(p.level) as lowlevel,
sum((select count(*) from players_online po where po.player_id = p.id)) as online
from guild_membership gms
join players p on p.id = gms.player_id
group by gms.guild_id
) players on players.guild_id = guilds.id
where guilds.name = 'Wideswing Poleaxe';
You can change the left outer join (left join) to an inner join (join), if you don't need to see guilds without any player.
I think the problem is here: a.guild_id = id
The id being used is from players, not guilds, as it is still part of the sub-query.
You shouldn't need all those subqueries, JOINs are almost always faster and should usually be first technique tried.
Try this...
SELECT `id`, `name`, `ownerid`, `creationdata`, `motd`
, COUNT(po.player_id) AS online
, MAX(p.level) AS toplevel
, MIN(p.level) AS lowlevel
FROM `guilds` AS g
LEFT JOIN guild_membership AS gm ON g.id = gm.guild_id
LEFT JOIN players AS p ON gm.player_id = p.player_id
LEFT JOIN players_online AS po ON gm.player_id = po.player_id
WHERE g.`name` = 'Wideswing Poleaxe'
;
COUNT only counts non-null values; similarly MAX, MIN, and most other aggregate functions ignore null values (only returning null if only null values were processed).
You should consider modifying your query like
SELECT g.`id`,
g.`name`,
g.`ownerid`,
g.`creationdata`,
g.`motd`,
(SELECT Count(*)
FROM guild_membership a,
players_online b
WHERE a.player_id = b.player_id
AND a.guild_id = id) AS `online`,
(SELECT Max(b.level)
FROM players b join guild_membership a on a.player_id = b.id
AND a.guild_id = g.id) AS `toplevel`,
(SELECT Min(a.level)
FROM players a join
guild_membership b on a.id = b.player_id
AND b.guild_id = g.id) AS `lowlevel`
FROM `guilds` g
WHERE g.`name` = 'Wideswing Poleaxe'
LIMIT 1;
I have 3 Tables CompanyMaster(Which has 3 Million rows), Token1, Token2 and the table structure is,
CompanyMaster
CREATE TABLE `CompanyMaster` (
`CompanyUID` int(11) NOT NULL AUTO_INCREMENT,
`WebDomain` varchar(150) DEFAULT NULL,
`CompanyPrimaryName` varchar(200) DEFAULT NULL,
PRIMARY KEY (`CompanyUID`)
) ENGINE=InnoDB AUTO_INCREMENT=3941244 DEFAULT CHARSET=latin1
Token1
CREATE TABLE `Token1`(
`CompanyUID` int(11) NOT NULL,
`Token` varchar(50) NOT NULL,
KEY `Token` (`Token`),
KEY `CompanyUID` (`CompanyUID`),
CONSTRAINT `CompanyAlias4_ibfk_1` FOREIGN KEY (`CompanyUID`) REFERENCES `CompanyMaster` (`CompanyUID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Token2
CREATE TABLE `Token2` (
`CompanyUID` int(11) NOT NULL,
`Token` varchar(100) NOT NULL,
KEY `Token` (`Token`),
KEY `CompanyUID` (`CompanyUID`),
CONSTRAINT `CompanyAlias5_ibfk_1` FOREIGN KEY (`CompanyUID`) REFERENCES `CompanyMaster` (`CompanyUID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
I want to get the WebDomain from the CompanyMaster table using the Token1 and Token2 tables.
The Query i am using is,
SELECT WebDomain FROM CompanyMaster WHERE CompanyUID IN (
SELECT CompanyUID FROM Token1 WHERE Token='appleinc'
UNION
SELECT CompanyUID FROM Token2 WHERE Token='d012233:q122100:')
This query takes almost 30 Seconds to get the result. I executed the sub-query alone, which is taking < 100 milli-seconds.So the problem is with the IN condition.
I replaced the query with join and it is executing in < 200 ms,
SELECT c.CompanyUID FROM `CompanyMaster` c
JOIN `Token1` tk1
ON tk1.CompanyUID = c.CompanyUID AND tk1.Token= 'appleinc'
JOIN `Token2` tk2
ON tk2.CompanyUID = c.CompanyUID AND tk2.Token= 'd012233:q122100:'
But the problem with above query is , if tk1.Alias = 'appleinc' or tk2.Alias = 'd012233:q122100:' fails it is giving output as empty row. But i want the matched rows even if only one condition is matched.
Please help me how to solve this one ? And i also want the query to be executed in less than 10 milli-seconds. Is it achievable ?
You should certainly get better performance with UNION ALL than with UNION, as it will have no difference for your case in output, but it does not need to filter out duplicates like UNION does:
SELECT WebDomain
FROM CompanyMaster
WHERE CompanyUID IN
( SELECT CompanyUID
FROM Token1
WHERE Token = 'appleinc'
UNION ALL
SELECT CompanyUID
FROM Token2
WHERE Token = 'd012233:q122100:')
However, if you would put the UNION in the outer query, it might even give better performance, like this:
SELECT WebDomain
FROM CompanyMaster m
INNER JOIN Token1 t ON t.CompanyUID = m.CompanyUID
WHERE Token = 'appleinc'
UNION
SELECT WebDomain
FROM CompanyMaster m
INNER JOIN Token2 t ON t.CompanyUID = m.CompanyUID
WHERE Token = 'd012233:q122100:'
Here it is probably important to only get unique values, so you need UNION without ALL here.
You can use where clause to filter your record on the basis of toke1 and token2. On the basis of your requirement you can change that clause.
Please check following SQL. Hope it will solve your problem.
SELECT
c.CompanyUID, c.WebDomain
FROM
CompanyMaster c
LEFT JOIN Token1 tk1 ON tk1.CompanyUID = c.CompanyUID
LEFT JOIN Token2 tk2 ON tk2.CompanyUID = c.CompanyUID
WHERE
tk1.Token = '123' OR tk2.Token = 'xyz';
I am not even sure how to ask this question but here's my situation. I use Plex to stream movies at home. I've built a database which I translate to a webpage that I use as an index. With in this database I have a few tables. The main one is called movie_list. 1 of the fields is called Rating which has an association table called assc_movie_genre which simply stores the movie id generated from the main table and a genre id which is read from another association table. There can be multiple movie Id's that are the same which match a Genre, for instance let's say The Matrix falls under the category Action and Sci Fi there will be 2 entries for MovieId each on matching the corresponding genre code. Anyways, my question is I need a query (if possible) that can join all genres to the appropriate row. Right now I have the following query
SELECT a.`Title`,a.`Year`,b.`Rating` FROM movie_list a, assc_rating b WHERE b.`Id` = a.`Rating
But would need to expand it to I guess join the multiple genres that match. I hope that all makes sense.
Thanks in advance
Update
Thanks to your help I am also there. Here is my current query
SELECT a.Title, c.Rating,
GROUP_CONCAT(DISTINCT b.GenreId ORDER BY b.GenreId)
AS Genres FROM assc_movie_genre b, movie_list a, assc_rating c
WHERE a.Id = b.MovieId AND a.Rating = c.Id group by a.Title
ORDER BY a.Title;
But the issue remains where I am just getting the GenreId instead of the genre name. I would assume I need to put a select in there somewhere so that it is pulling the name from the assc_genres tables just not 100% sure where.
Here's what the current output looks like
Title Rating Genres
28 Days Later... R 11,16,17
The concat works great and I'm so close. Thanks again
Update
Here are the queries to create my tables, you can get the structure from here (obviously)
CREATE TABLE IF NOT EXISTS `assc_genres` (
`Id` int(11) NOT NULL AUTO_INCREMENT,
`Genre` varchar(50) NOT NULL DEFAULT '0',
PRIMARY KEY (`Id`)
) ENGINE=InnoDB AUTO_INCREMENT=20 DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `assc_movie_genre` (
`MovieId` int(11) NOT NULL DEFAULT '0',
`GenreId` int(11) NOT NULL DEFAULT '0',
KEY `FK_assc_movie_genre_movie_list` (`MovieId`),
KEY `FK_assc_movie_genre_assc_genres` (`GenreId`),
CONSTRAINT `FK_assc_movie_genre_movie_list` FOREIGN KEY (`MovieId`) REFERENCES `movie_list` (`Id`),
CONSTRAINT `FK_assc_movie_genre_assc_genres` FOREIGN KEY (`GenreId`) REFERENCES `assc_genres` (`Id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `assc_rating` (
`Id` int(11) NOT NULL AUTO_INCREMENT,
`Rating` char(50) NOT NULL DEFAULT '0',
PRIMARY KEY (`Id`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `assc_status` (
`Id` tinyint(4) NOT NULL,
`Status` char(50) NOT NULL,
PRIMARY KEY (`Id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `movie_list` (
`Id` int(11) NOT NULL AUTO_INCREMENT,
`Title` varchar(100) NOT NULL DEFAULT '0',
`Year` year(4) NOT NULL DEFAULT '2000',
`Rating` int(11) NOT NULL DEFAULT '0',
`Folder` varchar(50) NOT NULL DEFAULT '0',
PRIMARY KEY (`Id`),
UNIQUE KEY `Title_Year` (`Title`,`Year`),
KEY `FK_movie_list_assc_rating` (`Rating`),
CONSTRAINT `FK_movie_list_assc_rating` FOREIGN KEY (`Rating`) REFERENCES `assc_rating` (`Id`)
) ENGINE=InnoDB AUTO_INCREMENT=614 DEFAULT CHARSET=latin1;
I am not sure as well if this is what you are asking, but you can join all 3 tables to get the data like
SELECT a.`Title`,
a.`Year`,
b.`Rating`
FROM movie_list a
JOIN assc_movie_genre c ON a.Id = c.movie_id
JOIN assc_rating b ON b.`Id` = c.genre_id;
Per your comment you can use GROUP_CONCAT() like
SELECT a.`Title`,
a.`Year`,
b.`Rating`,
xx.genre_list
FROM movie_list a
JOIN ( select movie_id, genre_id, group_concat(genre) as genre_list
from assc_movie_genre
group by movie_id) xx ON a.Id = xx.movie_id
JOIN assc_rating b ON b.`Id` = xx.genre_id;
You can modify your query like
SELECT a.Title, c.Rating,
GROUP_CONCAT(DISTINCT d.`Genre` ORDER BY d.`Genre`) AS Genres
FROM movie_list a
JOIN assc_movie_genre b ON a.Id = b.MovieId
JOIN assc_rating c ON a.Rating = c.Id
JOIN `assc_genres` d ON b.`GenreId` = d.Id
group by a.Title
ORDER BY a.Title;
I have three tables {animal, food, animal_food}
DROP TABLE IF EXISTS `tbl_animal`;
CREATE TABLE `tbl_animal` (
id_animal INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(25) NOT NULL DEFAULT "no name",
sex CHAR(1) NOT NULL DEFAULT "M",
size VARCHAR(10) NOT NULL DEFAULT "Mini",
age VARCHAR(10) NOT NULL DEFAULT "born",
hair VARCHAR(5 ) NOT NULL DEFAULT "short",
color VARCHAR(25) NOT NULL DEFAULT "not defined",
FOREIGN KEY (sex) REFERENCES `tbl_sexes` (sex),
FOREIGN KEY (tamanio) REFERENCES `tbl_sizes` (size),
FOREIGN KEY (age) REFERENCES `tbl_ages` (age),
FOREIGN KEY (hair) REFERENCES `tbl_hair_length` (hair_length),
CONSTRAINT `uc_Info_Animal` UNIQUE (`id_animal`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `tbl_food`;
CREATE TABLE `tbl_food` (
id_food INTEGER NOT NULL PRIMARY KEY,
type_food VARCHAR(20) NOT NULL DEFAULT "Other",
label VARCHAR(50) NOT NULL,
CONSTRAINT `uc_Info_Food` UNIQUE (`id_food`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `animal_food`;
CREATE TABLE `animal_food` (
id_animal INTEGER NOT NULL,
food VARCHAR(50) NOT NULL DEFAULT "",
quantity VARCHAR(50) NOT NULL DEFAULT "",
times VARCHAR(50) NOT NULL DEFAULT "",
description VARCHAR(50) NOT NULL DEFAULT "",
date_last DATE DEFAULT '0000-00-00 00:00:00',
date_water DATE DEFAULT '0000-00-00 00:00:00',
CONSTRAINT fk_ID_Animal_Food FOREIGN KEY (id_animal) REFERENCES `tbl_animal`(id_animal)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
And I have a view where I select the values columns in animal and animal_food depending on ID
CREATE VIEW `CAT_animal_food` AS
SELECT a.name, a.sex,a.size,a.age,a.hair,a.color,
a_f.*
FROM `tbl_animal` a, `animal_food` a_f
WHERE a.id_animal = a_f.id_animal;
What would be better to create a view like above or, to join these animal and animal_food tables?
SELECT ...
FROM A.table t1
JOIN B.table2 t2 ON t2.column = t1.col
What is really the diference between that kind of view and a left join for example?
The only difference between the two SELECT statements is the syntax style. Both perform INNER JOINS. In other words, this style uses what is called "implicit" syntax:
SELECT ...
FROM A.table t1, B.table2 t2
WHERE t2.column = t1.col
It is "implicit" because the join condition is implied by the WHERE clause. This version uses "explicit" syntax:
SELECT ...
FROM A.table t1
JOIN B.table2 t2 ON t2.column = t1.col
Most people prefer to see "explicit" syntax because it make your code easier to follow; the join condition is explicitly understood and any WHERE clause is obvious.
None of this is related to LEFT JOINS of course. Here is a famous link with a great visual description of join types.
There is a big difference.
The old school style using a list of tables only allows inner joins.
Further, placing non-key conditions in the ON clause of a proper join allows greater performance and capability that a where clause can not deliver. The primary reason for this is that the ON clause is evaluated as the join is being made, but the WHERE clause is evaluated after all joins have been made.
The subject is too complex to do justice here.
I want make query select all names from table 'a' where from table 'b' i have id_one='3'. id_two is id record from table
'a', two records have relation from id_one='3'. How i can make query ?
CREATE TABLE IF NOT EXISTS `a` (
`id` int(11) NOT NULL,
`name` varchar(11) NOT NULL,
`value` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Zrzut danych tabeli `a`
--
INSERT INTO `a` (`id`, `name`, `value`) VALUES
(1, 'lalala', 0),
(2, 'allalala', 0);
CREATE TABLE IF NOT EXISTS `b` (
`id_one` int(11) NOT NULL,
`id_two` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Zrzut danych tabeli `b`
--
INSERT INTO `b` (`id_one`, `id_two`) VALUES
(3, 1),
(3, 2);
This is what you want:
select Name
from a inner join b on a.id = b.id_two
where b.id_one = 3
sorry I'm not fully understanding your schemas or your question, but I think what you're trying to ask for is:
SELECT * FROM a
JOIN b on a.id = b.id_two;
Try that.
Your question title mentions left joins, but you don't need a left join to make the query you described.
Left joins are good for finding things that don't match up the way you'd expect. So, using a left join in this case depends on what you are looking for. If you are looking for b.id_two entries that don't have corresponding table a entries,
select Name, b.* from b left join a on a.id = b.id_two
This will give you a table that lists every row in table b, with NULLs in place of the names for table a where there is no match.
Likewise, if you are looking for names that don't have entries in b.id_two, you would use
select Name, b.* from a left join b on a.id = b.id_two
If you want to enforce that there is always a correspondence, you can define a foreign key constraint between the parent and child table.
select Name
from a join b on a.id = b.id_two
where b.id_one = 3;
Will also work to get your answer. Might I also suggest you significantly improve your create table statements to include indexing. E.G.
CREATE TABLE IF NOT EXISTS `a` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`name` varchar(11) NOT NULL,
`value` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `b` (
`id_one` bigint(20) NOT NULL,
`id_two` bigint(20) NOT NULL,
KEY `FKCAFBB09382DEAC` (`id_one`),
CONSTRAINT `b_a_1` FOREIGN KEY (`id_two`) REFERENCES `a` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I would also use bigint for primary keys and use charset=utf8
these days it is just to common to want to migrate application to multi-lingual, lay the ground work now. IMHO