I have this query for example (good, it works how I want it to)
SELECT `discusComments`.`memberID`, COUNT( `discusComments`.`memberID`) AS postcount
FROM `discusComments`
GROUP BY `discusComments`.`memberID` ORDER BY postcount DESC
Example Results:
memberid postcount
3 283
6 230
9 198
Now I want to join the memberid of the discusComments table with that of the discusTopic table (because what I really want to do is only get my results from a specific GROUP, and the group id is only in the topic table and not in the comment one hence the join.
SELECT `discusComments`.`memberID`, COUNT( `discusComments`.`memberID`) AS postcount
FROM `discusComments`
LEFT JOIN `discusTopics` ON `discusComments`.`memberID` = `discusTopics`.`memberID`
GROUP BY `discusComments`.`memberID` ORDER BY postcount DESC
Example Results:
memberid postcount
3 14789
6 8678
9 6987
How can I stop this huge increase happening in the postcount? I need to preserve it as before.
Once I have this sorted I want to have some kind of line which says WHERE discusTopics.groupID = 6, for example
CREATE TABLE IF NOT EXISTS `discusComments` (
`id` bigint(255) NOT NULL auto_increment,
`topicID` bigint(255) NOT NULL,
`comment` text NOT NULL,
`timeStamp` bigint(12) NOT NULL,
`memberID` bigint(255) NOT NULL,
`thumbsUp` int(15) NOT NULL default '0',
`thumbsDown` int(15) NOT NULL default '0',
`status` int(1) NOT NULL default '1',
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=7190 ;
.
CREATE TABLE IF NOT EXISTS `discusTopics` (
`id` bigint(255) NOT NULL auto_increment,
`groupID` bigint(255) NOT NULL,
`memberID` bigint(255) NOT NULL,
`name` varchar(255) NOT NULL,
`views` bigint(255) NOT NULL default '0',
`lastUpdated` bigint(10) NOT NULL,
PRIMARY KEY (`id`),
KEY `groupID` (`groupID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=913 ;
SELECT `discusComments`.`memberID`, COUNT( `discusComments`.`memberID`) AS postcount
FROM `discusComments`
JOIN `discusTopics` ON `discusComments`.`topicID` = `discusTopics`.`id`
GROUP BY `discusComments`.`memberID` ORDER BY postcount DESC
Joining the topicid in both tables solved the memberID issue. Thanks #Andiry M
You need to use just JOIN not LEFT JOIN and you can add AND discusTopics.memberID = 6 after ON discusComments.memberID = discusTopics.memberID
You can use subqueries lik this
SELECT `discusComments`.`memberID`, COUNT( `discusComments`.`memberID`) AS postcount
FROM `discusComments` where `discusComments`.`memberID` in
(select distinct memberid from `discusTopics` WHERE GROUPID = 6)
If i understand your question right you do not need to use JOIN here at all. JOINs are needed in case when you have many to many relationships and you need for each value in one table select all corresponding values in another table.
But here you have many to one relationship if i got it right. Then you can simply do select from two tables like this
SELECT a.*, b.id FROM a, b WHERE a.pid = b.id
This is simple request and won't create a giant overhead as JOIN does
PS: In the future try to experiment with your queries, try to avoid JOINs especially in MySQL. They are slow and dangerous in their complexity. For 90% of cases when you want to use JOIN there is simple and much faster solution.
Related
Two tables are defined:
CREATE TABLE `users` (
`user_id` mediumint(6) unsigned NOT NULL AUTO_INCREMENT,
`score` tinyint(1) unsigned DEFAULT NULL,
PRIMARY KEY (`user_id`)
);
CREATE TABLE `online` (
`user_id` mediumint(6) unsigned NOT NULL AUTO_INCREMENT,
`url` varchar(255) NOT NULL,
PRIMARY KEY (`user_id`)
);
How to combine the tables so that the result would be sorted by the score field from the largest to the smallest but at the top there were records with the value NULL?
This query does not sort the second sample:
(SELECT * FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NULL)
UNION
(SELECT * FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NOT NULL ORDER BY `score` DESC)
Use two keys in the sort:
SELECT *
FROM `online` o JOIN
`users`
USING (user_id)
ORDER BY (`score` IS NULL) DESC, Score DESC;
MySQL treats booleans as numbers in a numeric context, with "1" for true and "0" for false. So, DESC puts the true values first.
Incidentally, your version would look like it works if you used UNION ALL rather than UNION. However, it is not guaranteed that the results are in any particular order unless you explicitly have an ORDER BY.
The UNION incurs overhead for removing duplicates and in doing so rearranges the data.
Try:
select * from online join users using (user_id) order by ifnull(score, 10) desc;
You can use order by Nulls Last in the end of your sql to show nulls on the first.
You can try below -
select * from
(
SELECT *,1 as ord FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NULL
UNION
SELECT *,2 FROM `online` JOIN `users` USING(`user_id`) WHERE `score` IS NOT NULL
)A ORDER BY ord asc,`score` DESC
I have this select to get chat (like facebook inbox).
It will show most recent messages, grouping by user who sent them.
SELECT c.id, c.from, c.to, c.sent, c.message, c.recd FROM chat c
WHERE c.id IN(
SELECT MAX(id) FROM chat
WHERE (`to` = 1 and `del_to_status` = '0') or (`from` = 1 and `del_from_status` = '0')
GROUP BY CASE WHEN 1 = `to` THEN `from` ELSE `to` END
)
ORDER BY id DESC
limit 60
The problem is it is taking about 8 seconds.
`chat` (
`id` int(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`from` int(11) UNSIGNED NOT NULL,
`to` int(11) UNSIGNED NOT NULL,
`message` text NOT NULL,
`sent` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`recd` tinyint(1) NOT NULL DEFAULT '0',
`del_from_status` tinyint(1) NOT NULL DEFAULT '0',
`del_to_status` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `from` (`from`),
KEY `to` (`to`),
FOREIGN KEY (`from`) REFERENCES cadastro (`id`),
FOREIGN KEY (`to`) REFERENCES cadastro (`id`)
)
any ideas of indexing or re-writing this select to get better speed?
I am assuming chat.id is indexed. If not, of course you should add an index.
If it is indexed, MySQL is often very slow with sub selects.
One thing you can do is convert your sub select to a temporary table and join with it.
It will look something like
CREATE TEMPORARY TABLE IF NOT EXISTS max_chat_ids
( INDEX(id) )
ENGINE=MEMORY
AS ( 'SELECT MAX(id) as id FROM chat
WHERE (`to` = 1 and `del_to_status` = '0') or (`from` = 1 and `del_from_status` = '0')
GROUP BY CASE WHEN 1 = `to` THEN `from` ELSE `to` END' );
then, you need to just join with the temp table:
SELECT c.id, c.from, c.to, c.sent, c.message, c.recd FROM chat c
join max_chat_ids d on c.id=d.id
ORDER BY c.id DESC
limit 60
temp tables only live during the duration of the session, so if you test this in phpmyadmin remember to execute both queries together with ';' between them.
If you try this share your result.
I'll assume the column id is already indexed since it probably is the primary key of the table. If it's not the case, add the index:
create index ix1_chat on chat (id);
Then, if the selectivity of the subquery is good then an index will help. The selectivity is the percentage of rows the select is reading compared to the total number of rows. Is it 50%, 5%, 0.5%? If it's 5% or less then the following index will help:
create index ix2_chat on chat (`to`, del_to_status, `from`, del_from_status);
As a side note, please don't use reserved words for column names: I'm talking about the from column. It just makes life difficult for everyone.
I have this select:
select t.id, c.user, t.title, pp.foto, t.data from topics t
inner join cadastro c on t.user = c.id
left join profile_picture pp on t.user = pp.user
left join (
select c.topic, MAX(c.data) cdata from comments c
group by c.topic
)c on t.id = c.topic
where t.community = ?
order by ifnull(cdata, t.data) desc
limit 15
I want to select topics and order them by their date or the date of the topic comments, if it has comments.
Unfortunately, this is taking more than 9 seconds.
I don't think the problem here is indexing, but the way I am writing the select itself.
`topics` (
`id` int(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`user` INT(11) UNSIGNED NOT NULL,
`title` varchar(100) NOT NULL,
`description` varchar(1000),
`community` INT(11) UNSIGNED NOT NULL,
`data` datetime NOT NULL,
`ip` varchar(20),
PRIMARY KEY (`id`),
FOREIGN KEY (`user`) REFERENCES cadastro (`id`),
FOREIGN KEY (`community`) REFERENCES discussion (`id`)
)
`comments` (
`id` int(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`user` INT(11) UNSIGNED NOT NULL,
`comment` varchar(1000) NOT NULL,
`topic` INT(11) UNSIGNED NOT NULL,
`data` datetime NOT NULL,
`ip` varchar(20),
`delete` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
FOREIGN KEY (`user`) REFERENCES cadastro (`id`),
FOREIGN KEY (`topic`) REFERENCES topics (`id`)
)
Your EXPLAIN gives you a strong hint. The first row in that results says, using temporary, using filesort implying that it's not using a an index.
It might be possible to improve this query by adding indexes and removing some conditionals, but I think in this case a better solution exists.
Why not add a new column to topics that indicates the last time a comment was added? (like a last_modified). Every time a comment gets added, just update that column for that topic as well.
It's effectively denormalizing this. I think this a valid usecase and it's always going to be faster than fixing this messy query.
You are performing a full table scan on the table comments on every query. How many rows does it have? At least create the following index:
comments (topic, data);
to avoid reading the whole table every time.
I know you've said you don't think the problem is indexing, but 9 out of 10 times I've had this problem that's exactly what it's been down to.
Ensure you have an index created on each table that you're using in the query and include the columns specified in the join.
Also, as NiVeR said, don't use the same alias multiple times.
Here's a refactoring of that query, unsure if I've mixed up or missed a column name/alias or two though.
select t.id, c.user, t.title, pp.foto, t.data from topics t
inner join cadastro c on t.user = c.id
left join profile_picture pp on t.user = pp.user
left join (
select com.topic, MAX(com.data) comdata from comments com
group by com.topic
)com1 on t.id = com1.topic
where t.community = ?
order by ifnull(com1.comdata, t.data) desc
limit 15
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
I have a media center website that I'm working on that has a page that displays media categories. Each category can have multiple media items assigned to it, and each media item can be assigned to multiple categories.
The page contains a text input that must be able to filter the categories that are shown.
What I need to be able to do is get every category that is associated with the current user ($user_id) as well as a media item belonging to that user (unused categories are not displayed). Normally that would be simple enough, but I also have to be able to filter the categories based on fields in other tables associated with the media.
The fields I need to be able to apply the text filter to are as follows:
message_number in the media table
keywords in the media table
speaker_name in the media_speakers table
series_name in the media_series table
book_name in the media_books table
category_name in the media_categories table
As it is now, the query takes several seconds to complete. I'm not a MySQL pro, so I'm sure there must be better ways to do what I need to do here. In case it helps, I'm using MySQLi via PHP. My query has several subqueries, which I'm positive is the cause of the problem, but I didn't know any other way to do what I'm trying to do.
Below are the relevant table structures and the current query. I've included as much info as I can think of that can help someone to help me with this, but if you need more info please just let me know.
The media table (omitting some irrelevant fields) (series,speaker, and book fields contain the ID of a record in their corresponding tables):
`id` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`date` date NOT NULL DEFAULT '0000-00-00',
`message_number` varchar(32) DEFAULT NULL,
`series` int(10) unsigned zerofill NOT NULL DEFAULT '0000000000',
`speaker` int(10) unsigned zerofill NOT NULL DEFAULT '0000000000',
`book` int(10) unsigned zerofill NOT NULL DEFAULT '0000000000',
`keywords` text NOT NULL,
PRIMARY KEY (`id`)
The media_series table:
`id` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`series_name` varchar(255) NOT NULL DEFAULT '',
`cover` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
The media_speakers table:
`id` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`speaker_name` varchar(255) NOT NULL DEFAULT '',
`cover` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
The media_books table:
`id` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`book_name` varchar(64) NOT NULL DEFAULT '',
`book_shortname` varchar(10) NOT NULL DEFAULT '',
PRIMARY KEY (`id`)
The media_categories table:
`id` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`category_name` varchar(255) NOT NULL DEFAULT '',
`cover` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `id` (`id`)
The media_categories_assoc table:
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(10) unsigned DEFAULT NULL,
`media_id` int(10) unsigned zerofill DEFAULT NULL,
`category_id` int(10) unsigned zerofill DEFAULT NULL,
`marked_for_deletion` int(1) DEFAULT '0',
PRIMARY KEY (`id`)
Finally, the over-complicated query:
SELECT media_categories.id `media_categories.id`,
media_categories.user_id `media_categories.user_id`,
media_categories.category_name `media_categories.category_name`,
media_categories.cover `media_categories.cover`,
(SELECT id
FROM media
WHERE user_id = '$user_id'
AND media_categories.id IN (SELECT category_id
FROM media_category_assoc
WHERE user_id = '$user_id')
ORDER BY `date` DESC
LIMIT 1) `media.id`,
(SELECT `date`
FROM media
WHERE user_id = '$user_id'
AND media_categories.id IN (SELECT category_id
FROM media_category_assoc
WHERE user_id = '$user_id')
ORDER BY `date` DESC
LIMIT 1) `media.date`,
(SELECT series
FROM media
WHERE user_id = '$user_id'
AND media_categories.id IN (SELECT category_id
FROM media_category_assoc
WHERE user_id = '$user_id')
ORDER BY `date` DESC
LIMIT 1) `media.series`,
(SELECT speaker
FROM media
WHERE user_id = '$user_id'
AND media_categories.id IN (SELECT category_id
FROM media_category_assoc
WHERE user_id = '$user_id')
ORDER BY `date` DESC
LIMIT 1) `media.speaker`
FROM media_categories
LEFT JOIN media
ON media.id IN (SELECT media_id
FROM media_category_assoc
WHERE media_id = media.id
AND user_id = '$user_id')
LEFT JOIN media_series
ON media.series = media_series.id
LEFT JOIN media_speakers
ON media.speaker = media_speakers.id
LEFT JOIN media_books
ON media.book = media_books.id
WHERE media_categories.user_id = '$user_id'
AND media_categories.id IN (SELECT category_id
FROM media_category_assoc
WHERE user_id = '$user_id')
AND ( media.title LIKE '%filter_text%'
OR media.message_number LIKE '%filter_text%'
OR media.keywords LIKE '%filter_text%'
OR media_speakers.speaker_name LIKE '%filter_text%'
OR media_categories.category_name LIKE '%filter_text%'
OR media_series.series_name LIKE '%filter_text%'
OR media_books.book_name LIKE '%filter_text%' )
GROUP BY `media_categories.id`
ORDER BY `media.date` DESC
LIMIT 0, 12;
If I have been able to understand the query correctly you try to get some extra information from the most recent media per category for a given user. As far as I can see all of the subqueries in the SELECT-clause can then be moved to the FROM-clause.
Maybe doing it like this could help ?
SELECT media_categories.id,
media_categories.user_id,
media_categories.category_name,
media_categories.cover,
newest_media.id,
newest_media.'date',
newest_media.series,
newest_media.speaker
FROM media_categories
LEFT JOIN media_category_assoc
ON media_categories.id = media_category_assoc.category_id AND media_categories.user_id = media_category_assoc.user_id
LEFT JOIN (
SELECT id, 'date', series, speaker
FROM media
WHERE media.id = media_category_assoc.media_id
ORDER BY `date` DESC
LIMIT 1
) newest_media ON newest_media.user_id = '$user_id'
LEFT JOIN media_series
ON newest_media.series = media_series.id
LEFT JOIN media_speakers
ON newest_media.speaker = media_speakers.id
LEFT JOIN media_books
ON newest_media.book = media_books.id
LEFT JOIN media
ON media.id = media_category_assoc.media_id AND media.user_id = '$user_id'
WHERE media_categories.user_id = '$user_id'
AND ( media.title LIKE '%filter_text%'
OR media.message_number LIKE '%filter_text%'
OR media.keywords LIKE '%filter_text%'
OR media_speakers.speaker_name LIKE '%filter_text%'
OR media_categories.category_name LIKE '%filter_text%'
OR media_series.series_name LIKE '%filter_text%'
OR media_books.book_name LIKE '%filter_text%' )
GROUP BY `media_categories.id`
ORDER BY `media.date` DESC
LIMIT 0, 12;
As I mentioned in my comment, the subqueries can be the bottlenecks in your query. First of all, run an explain select... on your query to check the execution plan.
See the reference manual:
http://dev.mysql.com/doc/refman/5.0/en/explain.html
http://dev.mysql.com/doc/refman/5.0/en/using-explain.html
Now, about the suggestion I made about using temporary tables, I'll take your first subquery to make an example.
You use this:
SELECT
...,
(SELECT id
FROM media
WHERE user_id = '$user_id'
AND media_categories.id IN (SELECT category_id
FROM media_category_assoc
WHERE user_id = '$user_id')
ORDER BY `date` DESC
LIMIT 1),
....
And you can do something like this:
drop table if exists temp_step1;
create temporary table temp_step1
select id
from media
where user_id = #user_id -- I'm assuming you are putting this in a stored procedure
and media_categories.id in (SELECT category_id
FROM media_category_assoc
WHERE user_id = #user_id)
order by `date` desc
limit 1;
Then you can use this temp_step1 table as a row source for your big query.
Notice that this example returns only one row, so there's no point on indexing this. For those temp tables that contain more than one row and which you use in the FROM ... JOIN ... clause of your query, you will need to create indexes at the very least on all the fields you are doing the joins. To do that, after creating the temp table (for example temp_step_X) you should do this:
alter table temp_step_X
add index idx_indexName(field1),
...;
Hope this helps you
I need some help with a MySQL query. I have two tables, one with offers and one with statuses. An offer can has one or more statuses. What I would like to do is get all the offers and their latest status. For each status there's a table field named 'added' which can be used for sorting.
I know this can be easily done with two queries, but I need to make it with only one because I also have to apply some filters later in the project.
Here's my setup:
CREATE TABLE `test`.`offers` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`client` TEXT NOT NULL ,
`products` TEXT NOT NULL ,
`contact` TEXT NOT NULL
) ENGINE = MYISAM ;
CREATE TABLE `statuses` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`offer_id` int(11) NOT NULL,
`options` text NOT NULL,
`deadline` date NOT NULL,
`added` datetime NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
Should work but not very optimal imho :
SELECT *
FROM offers
INNER JOIN statuses ON (statuses.offer_id = offers.id
AND statuses.id =
(SELECT allStatuses.id
FROM statuses allStatuses
WHERE allStatuses.offer_id = offers.id
ORDER BY allStatuses.added DESC LIMIT 1))
Try this:
SELECT
o.*
FROM offers o
INNER JOIN statuses s ON o.id = s.offer_id
ORDER BY s.added
LIMIT 1