MySQL group by multiple columns issue - mysql

The query below collects temporary overview data for every user into memory table. Basicaly, user sees, count of items by keyword.
The problem is, it calculates total count of items by keyword_id.
What I need is, to calculate item_count by both keyword_id and item_type (Item.type).
SELECT
`Item`.`user_id` AS `user_id` ,
`ItemKeyword`.`keywordID` AS `keyword_id` ,
`Keyword`.`title` AS `keyword_title`,
count(`ItemKeyword`.`ItemID`) AS `ico_count`
FROM
(
(
`ItemKeyword`
JOIN `Item` ON(
(
`Item`.`id` = `ItemKeyword`.`ItemID`
)
)
)
JOIN `Keyword` ON(
(
`Keyword`.`id` = `ItemKeyword`.`keywordID`
)
)
)
GROUP BY
`Item`.`user_id` ,
`ItemKeyword`.`keywordID`;
Details
For example, now result looks like below
Basicaly, item_count is total of all item_types. What I need is, to separate the result below
user_id keyword_id keyword_title item_count
1 9645 surveillance 20
Into something like this:
user_id keyword_id keyword_title item_count item_type
1 9645 surveillance 18 1
1 9645 surveillance 2 2
Where, item_count are calculated by both keyword_id and item_type.
I can't figure out how to include item_type also into this query.
Any suggestions?

I do not understand your love of brackets (parenthesis). Why so many? in my opinion you lose the readability. It was just a side note.
If you need an extra grouping level you need to modify the query like this:
SELECT
`Item`.`user_id` AS `user_id` ,
`ItemKeyword`.`keywordID` AS `keyword_id` ,
`Keyword`.`title` AS `keyword_title`,
count(`ItemKeyword`.`ItemID`) AS `ico_count`,
`Item`.`type` AS `item_type`
FROM `ItemKeyword` JOIN `Item` ON
`Item`.`id` = `ItemKeyword`.`ItemID`
JOIN `Keyword` ON
`Keyword`.`id` = `ItemKeyword`.`keywordID`
GROUP BY
`Item`.`user_id` ,
`Item`.`type` ,
`ItemKeyword`.`keywordID`,
`Keyword`.`title`;

Related

MySQL delete all but one record matching two column values [duplicate]

I am doing some data clean up and I would like to remove duplicate rows by finding records that have the same "picture id" and "date" values:
Example:
picture_id - 2 date - "13-Jul-18"
picture_id - 2 date - "13-Jul-18"
picture_id - 2 date - "13-Jul-18"
picture_id - 2 date - "13-Jul-18"
DELETE FROM `pictures` WHERE `picture_id` = '2' AND `date` = '13-Jul-18'
Table columns (in order): ID (primary key), picture_id, date, followers
I would like to only delete all but one of the duplicate records. It does not matter which one. How can I accomplish this?
In MySQL, you can keep the smallest (or biggest) id using JOIN:
DELETE p
FROM pictures p JOIN
(SELECT p.picture_id, p.date, MIN(id) as min_id
FROM pictures p
WHERE p.picture_id = 2 AND p.date = '2018-07-13'
GROUP BY p.picture_id
) pp
ON p.picture_id = pp.picture_id AND p.date = pp.date AND p.id > p.min_id;
Assuming you don't care which ID you keep you can select one record all delete all those records which are not the one selected
DELETE
FROM pictures
WHERE ID NOT IN (
SELECT
ID
FROM pictures
WHERE picture_id = 2 AND
Date = '2018-07-13'
LIMIT 1
) AND
picture_id = 2 AND
Date = '2018-07-13'
The fact these are unwanted duplicates makes me think either your current Primary Key is insufficient for your purposes or you need to look at a unique constraints
you can try something like
DROP TABLE IF EXISTS pictures;
CREATE TABLE pictures(picture_id INT(11), `dt` DATE, followers INT(11));
INSERT INTO pictures VALUES
(2,'2018-07-13',4553),
(2,'2018-07-13',4552),
(2,'2018-07-13',4557),
(2,'2018-07-13',4577),
(3,'2018-07-13',4355),
(3,'2018-07-13',4351),
(3,'2018-07-13',4353),
(3,'2018-07-13',4374);
Delete query
DELETE P FROM pictures p
LEFT JOIN (
SELECT picture_id, dt, MAX(followers) AS fol
FROM pictures WHERE dt ='2018-07-13' GROUP BY picture_id
) AS main
ON main.dt = p.dt
WHERE main.picture_id = p.picture_id
AND main.fol <> p.followers;
I hope this will solve you problem.
simply use common table
With CTE_Duplicates as
(select picture_id ,date , row_number() over(partition by picture_id,date order by picture_id ,date ) rownumber
from `pictures` )
delete from CTE_Duplicates where rownumber!=1
it work for me.please check

MySQL Delete duplicate rows that have same column value

I am doing some data clean up and I would like to remove duplicate rows by finding records that have the same "picture id" and "date" values:
Example:
picture_id - 2 date - "13-Jul-18"
picture_id - 2 date - "13-Jul-18"
picture_id - 2 date - "13-Jul-18"
picture_id - 2 date - "13-Jul-18"
DELETE FROM `pictures` WHERE `picture_id` = '2' AND `date` = '13-Jul-18'
Table columns (in order): ID (primary key), picture_id, date, followers
I would like to only delete all but one of the duplicate records. It does not matter which one. How can I accomplish this?
In MySQL, you can keep the smallest (or biggest) id using JOIN:
DELETE p
FROM pictures p JOIN
(SELECT p.picture_id, p.date, MIN(id) as min_id
FROM pictures p
WHERE p.picture_id = 2 AND p.date = '2018-07-13'
GROUP BY p.picture_id
) pp
ON p.picture_id = pp.picture_id AND p.date = pp.date AND p.id > p.min_id;
Assuming you don't care which ID you keep you can select one record all delete all those records which are not the one selected
DELETE
FROM pictures
WHERE ID NOT IN (
SELECT
ID
FROM pictures
WHERE picture_id = 2 AND
Date = '2018-07-13'
LIMIT 1
) AND
picture_id = 2 AND
Date = '2018-07-13'
The fact these are unwanted duplicates makes me think either your current Primary Key is insufficient for your purposes or you need to look at a unique constraints
you can try something like
DROP TABLE IF EXISTS pictures;
CREATE TABLE pictures(picture_id INT(11), `dt` DATE, followers INT(11));
INSERT INTO pictures VALUES
(2,'2018-07-13',4553),
(2,'2018-07-13',4552),
(2,'2018-07-13',4557),
(2,'2018-07-13',4577),
(3,'2018-07-13',4355),
(3,'2018-07-13',4351),
(3,'2018-07-13',4353),
(3,'2018-07-13',4374);
Delete query
DELETE P FROM pictures p
LEFT JOIN (
SELECT picture_id, dt, MAX(followers) AS fol
FROM pictures WHERE dt ='2018-07-13' GROUP BY picture_id
) AS main
ON main.dt = p.dt
WHERE main.picture_id = p.picture_id
AND main.fol <> p.followers;
I hope this will solve you problem.
simply use common table
With CTE_Duplicates as
(select picture_id ,date , row_number() over(partition by picture_id,date order by picture_id ,date ) rownumber
from `pictures` )
delete from CTE_Duplicates where rownumber!=1
it work for me.please check

How to write correct sql with left join on some tables?

Good day.
STRUCTURE TABLES AND ERROR WHEN EXECUTE QUERY ON SQLFIDDLE
I have some sql queries:
First query:
SELECT
n.Type AS Type,
n.UserIdn AS UserIdn,
u.Username AS Username,
n.NewsIdn AS NewsIdn,
n.Header AS Header,
n.Text AS Text,
n.Tags AS Tags,
n.ImageLink AS ImageLink,
n.VideoLink AS VideoLink,
n.DateCreate AS DateCreate
FROM News n
LEFT JOIN Users u ON n.UserIdn = u.UserIdn
SECOND QUERY:
SELECT
IFNULL(SUM(Type = 'up'),0) AS Uplikes,
IFNULL(SUM(Type = 'down'),0) AS Downlikes,
(IFNULL(SUM(Type = 'up'),0) - IFNULL(SUM(Type = 'down'),0)) AS SumLikes
FROM JOIN Likes
WHERE NewsIdn=NewsIdn //only for example- in main sql NewsIdn = value NewsIdn from row table News
ORDER BY UpLikes DESC
AND TREE QUERY
SELECT
count(*) as Favorit
Form Favorites
WHERE NewsIdn=NewsIdn //only for example- in main sql NewsIdn = value NewsIdn from row table News
I would like to combine both queries, display all rows from the table News, as well as the number of Uplikes, DownLikes and number of Favorit for each value NewsIdn from the table of News (i.e. number of Uplikes, DownLikes and number of Favorit for each row of News) and make order by Uplikes Desc.
Tell me please how to make it?
P.S.: in result i would like next values
TYPE USERIDN USERNAME NEWSIDN HEADER TEXT TAGS IMAGELINK VIDEOLINK DATECREATE UPLIKES DOWNLIKES SUMLIKES FAVORIT
image 346412 test 260806 test 1388152519.jpg December, 27 2013 08:55:27+0000 2 0 2 2
image 108546 test2 905554 test2 1231231111111111111111111 123. 123 1388153493.jpg December, 27 2013 09:11:41+0000 1 0 1 0
text 108546 test2 270085 test3 123 .123 December, 27 2013 09:13:30+0000 1 0 1 0
image 108546 test2 764955 test4 1388192300.jpg December. 27 2013 19:58:22+0000 0 1 -1 0
First, your table structures with all the "Idn" of varchar(30). It appears those would actually be ID keys to the other tables and should be integers for better indexing and joining performance.
Second, this type of process, especially web-based is a perfect example of DENORMALIZING the values for likes, dislikes, and favorites by actually having those columns as counters directly on the record (ex: News table). When a person likes, dislikes or makes as a favorite, stamp it right away and be done with it. If a first time through you do a bulk sql-update do so, but also have triggers on the table to automatically handle updating the counts appropriately. This way, you just query the table directly and order by that which you need and you are not required to query all likes +/- records joined to all news and see which is best. Having an index on the news table will be your best bet.
Now, that said, and with your existing table constructs, you can do via pre-aggregate queries and joining them as aliases in the sql FROM clause... something like
SELECT
N.Type,
N.UserIdn,
U.UserName,
N.NewsIdn,
N.Header,
N.Text,
N.Tags,
N.ImageLink,
N.VideoLink,
N.DateCreate,
COALESCE( SumL.UpLikes, 0 ) as Uplikes,
COALESCE( SumL.DownLikes, 0 ) as DownLikes,
COALESCE( SumL.NetLikes, 0 ) as NetLikes,
COALESCE( Fav.FavCount, 0 ) as FavCount
from
News N
JOIN Users U
ON N.UserIdn = U.UserIdn
LEFT JOIN ( select
L.NewsIdn,
SUM( L.Type = 'up' ) as UpLikes,
SUM( L.Type = 'down' ) as DownLikes,
SUM( ( L.Type = 'up' ) - ( L.Type = 'down' )) as NetLikes
from
Likes L
group by
L.NewsIdn ) SumL
ON N.NewsIdn = SumL.NewsIdn
LEFT JOIN ( select
F.NewsIdn,
COUNT(*) as FavCount
from
Favorites F
group by
F.NewsIdn ) Fav
ON N.NewsIdn = Fav.NewsIdn
order by
SumL.UpLikes DESC
Again, I do not understand why you would have an auto-increment numeric ID column for the news table, then ANOTHER value for it as NewsIdn as a varchar. I would just have this and your other tables reference the News.ID column directly... why have two columns representing the same component. And obviously, each table you are doing aggregates (likes, favorites), should have indexes on any such criteria you would join or aggregate on (hence NewsIdn) column, UserIdn, etc.
And final reminder, this type of query is ALWAYS running aggregates against your ENTIRE TABLE of likes, favorites EVERY TIME and suggest going with denormalized columns to hold the counts when someone so selects them. You can always go back to the raw tables if you ever want to show or update for a particular person to change their like/dislike/favorite status.
You'll have to look into reading on triggers as each database has its own syntax for handling.
As for table structures, this is a SIMPLIFIED version of what I would have (removed many other columns from you SQLFiddle sample)
CREATE TABLE IF NOT EXISTS `News` (
id int(11) NOT NULL AUTO_INCREMENT,
UserID integer NOT NULL,
... other fields
`DateCreate` datetime NOT NULL,
PRIMARY KEY ( id ),
KEY ( UserID )
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=5 ;
extra key on the User ID in case you wanted all news activity created by a specific user.
CREATE TABLE IF NOT EXISTS `Users` (
id int(11) NOT NULL AUTO_INCREMENT,
other fields...
PRIMARY KEY ( id ),
KEY ( LastName, Name )
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=5 ;
additional key in case you want to do a search by a user's name
CREATE TABLE IF NOT EXISTS `Likes` (
id int(11) NOT NULL AUTO_INCREMENT,
UserId integer NOT NULL,
NewsID integer NOT NULL,
`Type` enum('up','down') NOT NULL,
`IsFavorite` enum('yes','no') NOT NULL,
`DateCreate` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY ( UserID ),
KEY ( NewsID, IsFavorite )
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=6 ;
additional keys here for joining and/or aggregates. I've also added a flag column for being a favorite too. This could prevent the need of a favorites table since they hold the same basic content of the LIKES. So someone could just LIKE/DISLIKE, against a given news item, but ALSO LIKE/DISLIKE it as a FAVORITE the end-user wants to quickly be able to reference.
Now, how do these table structures get simplified for querying? Each table has its own "id" column, but any OTHER table is uses the tableNameID (UserID, NewsID, LikesID or whatever) and that is the join.
select ...
from
News N
Join Users U
on N.UserID = U.ID
Join Likes L
on N.ID = L.NewsID
Integer columns are easier and more commonly identifiable by others when writing queries... Does this make a little more sense?
SELECT
n.Type AS Type,
n.UserIdn AS UserIdn,
u.Username AS Username,
n.NewsIdn AS NewsIdn,
n.Header AS Header,
n.Text AS Text,
n.Tags AS Tags,
n.ImageLink AS ImageLink,
n.VideoLink AS VideoLink,
n.DateCreate AS DateCreate,
IFNULL(SUM(Likes.Type = 'up'),0) AS Uplikes,
IFNULL(SUM(Likes.Type = 'down'),0) AS Downlikes,
(IFNULL(SUM(Likes.Type = 'up'),0) - IFNULL(SUM(Likes.Type = 'down'),0)) AS SumLikes,
COUNT(DISTINCT Favorites.id) as Favorit
FROM News n
LEFT JOIN Users u ON n.UserIdn = u.UserIdn
LEFT JOIN Likes ON Likes.NewsIdn = n.NewsIdn
LEFT JOIN Favorites ON n.NewsIdn=Favorites.NewsIdn
GROUP BY n.NewsIdn

Using MySQL COUNT(1), COUNT(2) ...etc using JOIN

I have 4 tables:
Table talks
table talks_fan
table talks_follow
table talks_comments
What I'm trying to achieve is counting all comments, fans, followers for every single talk.
I came up with this so far.
All tables have talk_id and only in talks table is a primary key
SELECT
g. *,
COUNT( m.talk_id ) AS num_of_comments,
COUNT( f.talk_id ) AS num_of_followers
FROM
talks AS g
LEFT JOIN talks_comments AS m
USING ( talk_id )
LEFT JOIN talks_follow AS f
USING ( talk_id )
WHERE g.privacy = 'public'
GROUP BY g.talk_id
ORDER BY g.created_date DESC
LIMIT 30;
I also tried using this method
SELECT
t.*,
COUNT(b.talk_id) AS comments,
COUNT(bt.talk_id) AS followers
FROM
talks t
LEFT JOIN talks_follow bt
ON bt.talk_id = t.talk_id
LEFT JOIN talks_comments b
ON b.talk_id = t.talk_id
GROUP BY t.talk_id;
Both give me the same results ....?!
Update: Create Statements
CREATE TABLE IF NOT EXISTS `talks` (
`talk_id` bigint(20) NOT NULL AUTO_INCREMENT,
`user_id` mediumint(9) NOT NULL,
`title` varchar(255) NOT NULL,
`content` text NOT NULL,
`created_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`privacy` enum('public','private') NOT NULL DEFAULT 'private',
PRIMARY KEY (`talk_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=7 ;
CREATE TABLE IF NOT EXISTS `talks_comments` (
`comment_id` bigint(20) NOT NULL AUTO_INCREMENT,
`talk_id` bigint(20) NOT NULL,
`user_id` mediumint(9) NOT NULL,
`comment` text NOT NULL,
`date_created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`status` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`comment_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=8 ;
CREATE TABLE IF NOT EXISTS `talks_fan` (
`fan_id` bigint(20) NOT NULL AUTO_INCREMENT,
`talk_id` bigint(20) NOT NULL,
`user_id` bigint(20) NOT NULL,
`created_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`status` tinyint(1) NOT NULL DEFAULT '1',
PRIMARY KEY (`fan_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=4 ;
CREATE TABLE IF NOT EXISTS `talks_follow` (
`follow_id` bigint(20) NOT NULL AUTO_INCREMENT,
`talk_id` bigint(20) NOT NULL,
`user_id` mediumint(9) NOT NULL,
`date_created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`follow_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=5 ;
The final query that works
SELECT t.* , COUNT( DISTINCT b.comment_id ) AS comments,
COUNT( DISTINCT bt.follow_id ) AS followers,
COUNT( DISTINCT c.fan_id ) AS fans
FROM talks t
LEFT JOIN talks_follow bt ON bt.talk_id = t.talk_id
LEFT JOIN talks_comments b ON b.talk_id = t.talk_id
LEFT JOIN talks_fan c ON c.talk_id = t.talk_id
WHERE t.privacy = 'public'
GROUP BY t.talk_id
ORDER BY t.created_date DESC
LIMIT 30
EDIT: Final answer to the whole issue...
I have modified the Query and created some code in PHP (Codeigniter) to solve my issue apone the reccomendation of #Bill Karwin
$sql="
SELECT t.*,
COUNT( DISTINCT b.comment_id ) AS comments,
COUNT( DISTINCT bt.follow_id ) AS followers,
COUNT( DISTINCT c.fan_id ) AS fans,
GROUP_CONCAT( DISTINCT c.user_id ) AS list_of_fans
FROM talks t
LEFT JOIN talks_follow bt ON bt.talk_id = t.talk_id
LEFT JOIN talks_comments b ON b.talk_id = t.talk_id
LEFT JOIN talks_fan c ON c.talk_id = t.talk_id
WHERE t.privacy = 'public'
GROUP BY t.talk_id
ORDER BY t.created_date DESC
LIMIT 30
";
$query = $this->db->query($sql);
if($query->num_rows() > 0)
{
$results = array();
foreach($query->result_array() AS $talk){
$fan_user_id = explode(",", $talk['list_of_fans']);
foreach($fan_user_id AS $user){
if($user == 1 /* this supposed to be user id or session*/){
$talk['list_of_fans'] = 'yes';
}
}
$follower_user_id = explode(",", $talk['list_of_follower']);
foreach($follower_user_id AS $user){
if($user == 1 /* this supposed to be user id or session*/){
$talk['list_of_follower'] = 'yes';
}
}
$results[] = array(
'talk_id' => $talk['talk_id'],
'user_id' => $talk['user_id'],
'title' => $talk['title'],
'created_date' => $talk['created_date'],
'comments' => $talk['comments'],
'followers' => $talk['followers'],
'fans' => $talk['fans'],
'list_of_fans' => $talk['list_of_fans'],
'list_of_follower' => $talk['list_of_follower']
);
}
}
I STILL BELIEVE IT COULD BE OPTIMIZED IN THE DB AND JUST USE THE RESULT...
Im thinking if there are 1000 follower and 2000 fans of every single TALK then the result will take much longer to load.. HOW IF YOUT MULTIPLY THE NO WITH 10. Or im mistaking hear...
EDIT: adding benchmark for the query test...
I have used codeigniter profiler to know how long it take for the query to finish excuting.
that been said i also start adding data in the tables gratually
the result as follows.
Testing the DB after answerting data into it
Query Results time
table Talks
---------------
table data 50 rows.
Time: 0.0173 seconds
Table Rows: 644 rows
Time: 0.0535 seconds
Table Rows: 1250 rows
Time: 0.0856 seconds
Adding data to other tables
--------------------------
Talks = 1250 rows
talks_follow = 4115
talks_fan = 10 rows
Time: 2.656 seconds
Adding data to other tables
--------------------------
Talks = 1250 rows
talks_follow = 4115
talks_fan = 10 rows
talks_comments = 3650 rows
Time: 10.156 seconds
After replacing LEFT JOIN with STRAIGHT_JOIN
Time: 6.675 seconds
It seems that its extremely heavy on the DB.....
NOW Im Going to another dilemma on how to enhance its performance
Edited: using #leonardo_assumpcao suggestion
After rebuilding the DB using #leonardo_assumpcao suggestion
for indexing few fields..........
Adding data to other tables
--------------------------
Talks = 6000 Rows
talks_follow = 10000 Rows
talks_fan = 10000 Rows
talks_comments = 10000 Rows
Time: 17.940 second
Is this normal for heavy data DB......?
I can say this is (at least) one of the coolest select statements I improved today.
SELECT STRAIGHT_JOIN
t.* ,
COUNT( DISTINCT b.comment_id ) AS comments,
COUNT( DISTINCT bt.follow_id ) AS followers,
COUNT( DISTINCT c.fan_id ) AS fans
FROM
(
SELECT * FROM talks
WHERE privacy = 'public'
ORDER BY created_date DESC
LIMIT 0, 30
) AS t
LEFT JOIN talks_follow bt ON (bt.talk_id = t.talk_id)
LEFT JOIN talks_comments b ON (b.talk_id = t.talk_id)
LEFT JOIN talks_fan c ON (c.talk_id = t.talk_id)
GROUP BY t.talk_id ;
But it seems to me that your problem resides on your tables; A first step to obtain efficient queries is to index every field involved on your desired joins.
I've made some modifications on the tables you shown above; You can see its code here (updated).
Quite interesting, isn't it? Since we're here, take also your ERR model:
First try it using MySQL test database. Hopefully it will solve your performance troubles.
(Forgive my english, it's my second language)
You can force this into one query like so:
SELECT COUNT(*) num, 'talks' item FROM talks
UNION
SELECT COUNT(*) num, 'talks_fan' item FROM talks_fan
UNION
SELECT COUNT(*) num, 'talks_follow' item FROM talks_follow
UNION
SELECT COUNT(*) num, 'talks_comment' item FROM talks_comment
This will give you a five row resultset with one row per table. Each row is the count in a particular table.
If you must get it all into a single row you can do a pivot like so.
SELECT
SUM( CASE item WHEN 'talks' THEN num ELSE 0 END ) AS 'talks',
SUM( CASE item WHEN 'talks_fan' THEN num ELSE 0 END ) AS 'talks_fan',
SUM( CASE item WHEN 'talks_follow' THEN num ELSE 0 END ) AS 'talks_follow',
SUM( CASE item WHEN 'talks_comment' THEN num ELSE 0 END ) AS 'talks_comment'
FROM
( SELECT COUNT(*) num, 'talks' item FROM talks
UNION
SELECT COUNT(*) num, 'talks_fan' item FROM talks_fan
UNION
SELECT COUNT(*) num, 'talks_follow' item FROM talks_follow
UNION
SELECT COUNT(*) num, 'talks_comment' item FROM talks_comment
) counts
(This doesn't take into account your WHERE g.privacy = clause because I don't understand that. But you could add a WHERE clause to one one of the four queries in the UNION item to handle that.)
Notice that this truly is four queries on four separate tables coerced into a single query.
And, by the way, there is no difference in value between COUNT(*) and COUNT(id) when id is the primary key of the table. COUNT(id) doesn't count the rows for which the id is NULL, but if id is the primary key, then it is NOT NULL. But COUNT(*) is faster, so use it.
Edit if you need the number of fan, follow, and comment rows for each distinct talk, do this. It's the same idea of doing a union and a pivot, but with an extra parameter.
SELECT
talk_id,
SUM( CASE item WHEN 'talks_fan' THEN num ELSE 0 END ) AS 'talks_fan',
SUM( CASE item WHEN 'talks_follow' THEN num ELSE 0 END ) AS 'talks_follow',
SUM( CASE item WHEN 'talks_comment' THEN num ELSE 0 END ) AS 'talks_comment'
FROM
(
SELECT talk_id, COUNT(*) num, 'talks_fan' item
FROM talks_fan
GROUP BY talk_id
UNION
SELECT talk_id, COUNT(*) num, 'talks_follow' item
FROM talks_follow
GROUP BY talk_id
UNION
SELECT talk_id, COUNT(*) num, 'talks_comment' item
FROM talks_comment
GROUP BY talk_id
) counts
GROUP BY talk_id
After doing this for (too) many years, I've discovered that the best way to describe a query you need is to say to yourself "I need a result set with one row for each xxx, with columns for yyy, zzz, and qqq."
The reason the counts are the same is that it's counting rows after the joins have combined the tables. By joining to multiple tables, you're creating a Cartesian product.
Basically, you're counting not only how many comments per talk, but how many comments * followers per talk. Then you count the followers as how many followers * comments per talk. Thus the counts are the same, and they're all way too high.
Here's a simpler way to write a query to count each distinct comment, follower, etc. only once:
SELECT t.*,
COUNT(DISTINCT b.comment_id) AS comments,
COUNT(DISTINCT bt.follow_id) AS followers
FROM talks t
LEFT JOIN talks_follow bt ON bt.talk_id = t.talk_id
LEFT JOIN talks_comments b ON b.talk_id = t.talk_id
GROUP BY t.talk_id;
Re your comment: I wouldn't fetch all the followers in the same query. You could do it this way:
SELECT t.*,
COUNT(DISTINCT b.comment_id) AS comments,
COUNT(DISTINCT bt.follow_id) AS followers,
GROUP_CONCAT(DISTINCT bt.follower_name) AS list_of_followers
FROM talks t
LEFT JOIN talks_follow bt ON bt.talk_id = t.talk_id
LEFT JOIN talks_comments b ON b.talk_id = t.talk_id
GROUP BY t.talk_id;
But what you'd get back is a single string with the follower names separated by commas. Now you have to write application code to split the string on commas, you have to worry if some follower names actually contain commas already, and so on.
I'd do a second query, fetching the followers for a given talk. It's likely you want to display the followers only for a specific talk anyway.
SELECT follower_name
FROM talks_follow
WHERE talk_id = ?

SQL - Confusing Query

I have a 'photos' table:
photoID (INT), setID (INT)....
18900 , 234 , ...
18901 , 234 , ...
18902 , 234 , ...
18903 , 249 , ...
18904 , 249 , ...
18905 , 249 , ...
I also have a 'photoKeyword' table:
photoID (INT), keywordID (INT)
18900 , 12
18900 , 21
18901 , 17
18905 , 26
18905 , 10
As you can see from my examples above, photos 18902, 18903 and 18904 do NOT have any keywords in the photoKeyword table. This is exactly what I am trying to establish.
I am trying to produce a list of photoID's that don't have keywords but one setID at a time. So as you can see, photo 18902 doesn't have keywords and so does 18903 and 18904 but these three photos have two different setID's.
So running this query once, should only return photo 18902. I would then add keywords to this photo so it won't be a problem again. The next time I run the query it should return photo 18903 and 18904, the next set (setID:249) of photos that do not have keywords.
How is this possible? Is it possible just using SQL? I hope you can understand what I looking to achieve, I lost myself just writing about it!!
Any thoughts gratefully received...
try
SELECT X.photoID FROM photos X
INNER JOIN
(SELECT DISTINCT P.setID FROM
photos P
LEFT OUTER JOIN (SELECT K.photoID, COUNT(*) C FROM photoKeyword K GROUP BY K.photoID) KC ON KC.photoID = P.photoID
GROUP BY P.setID
HAVING SUM (KC.C) < 1) Y ON X.setID = Y.SetID
SELECT photoID
, setID
FROM photos
WHERE photoID NOT IN
( SELECT photoID
FROM photoKeyword
)
AND setID =
( SELECT setID
FROM photos
WHERE photoID NOT IN
( SELECT photoID
FROM photoKeyword
)
ORDER BY setID
LIMIT 1
)
This might be what you need which is quite simple... when you think about it... Ensure your keyword table has an index on the photo id
select
p.PhotoID,
p.SetID
from
Photos p
LEFT JOIN photoKeyword pkey
on p.PhotoID = pkey.PhotoID
where
pkey.PhotoID = null
By doing a LEFT join, we know it will always attempt to the second table. Then, if the second table has no matches, we know the ID trying to join on will be null... So, left join and return only those IDs where the answer is NULL in the second table.