Im currently building my self a forum for my school project and this query is listing all the forum categories but im having problem checking if there's a topic/thread thats not been read by the user and then TAG the forum category as having unread messages.
It says "Unknown column 'forum_category.id' in 'field list'" in the subquery... i have checked several examples on subqueries and from what i have seen i should be able to access forum_category.id and use it in the subquery? i don't see what im doing wrong at this point...
Help is much appreciated!
SELECT forum_category.id
, root.name AS root_name
, subcat.name AS subcat_name
, subcat.id AS subcat_id
, subcat.description AS subcat_description
, subcat.safe_url AS subcat_safe_url
, topics.topic_id
, topics.topic_safe_url
, topics.topic_title
, topics.last_post_time
, topics.topic_last_poster_name
, topics.topic_last_poster_id
, (
SELECT
posts_read.last_read_time
FROM
forum_topics a
LEFT JOIN
forum_posts_read AS posts_read ON
posts_read.last_read_time > a.last_post_time
AND posts_read.last_read_time > 1321004546
AND posts_read.topic_id = a.topic_id
AND posts_read.user_id = 1
AND a.forum_id = forum_category.id
LIMIT 1) AS last_read_time
FROM forum_category AS root
LEFT JOIN
forum_category AS subcat ON subcat.parent_id = root.id
LEFT JOIN
forum_topics AS topics ON topics.forum_id = subcat.id
LEFT JOIN
forum_topics AS t2 ON t2.forum_id = subcat.id AND t2.last_post_time > topics.last_post_time
WHERE
root.parent_id = 0 AND t2.forum_id IS NULL
ORDER BY
root_name, subcat_name
I know tried this, but it only checks the first topic/thread in each category...
SELECT root.name AS root_name
, subcat.name AS subcat_name
, subcat.id AS subcat_id
, subcat.description AS subcat_description
, subcat.safe_url AS subcat_safe_url
, topics.topic_id
, topics.topic_safe_url
, topics.topic_title
, topics.last_post_time
, topics.topic_last_poster_name
, topics.topic_last_poster_id
, posts_read.last_read_time
FROM forum_category AS root
LEFT JOIN
forum_category AS subcat ON subcat.parent_id = root.id
LEFT JOIN
forum_topics AS topics ON topics.forum_id = subcat.id
LEFT JOIN
forum_topics AS t2 ON t2.forum_id = subcat.id AND t2.last_post_time > topics.last_post_time
LEFT JOIN
forum_posts_read AS posts_read ON
posts_read.last_read_time > topics.last_post_time
AND posts_read.last_read_time > ?
AND posts_read.topic_id = topics.topic_id
AND posts_read.user_id = ?
AND topics.forum_id = subcat.id
WHERE
root.parent_id = 0 AND t2.forum_id IS NULL
ORDER BY
root_name, subcat_name
Who had an idea it would be this hard... i mean its rather simple thing i want to do :(
tables. I believe your problem is in the aliasing of your tables. You've told the engine not to use forum_category but instead use ROOT or subcat. So the SQL doesn't know about a forum_Category (it doesn't see that table in the "FROM". The same problem exists in the outer select too. Change it to root as well.
SELECT root.id
, root.name AS root_name
, subcat.name AS subcat_name
, subcat.id AS subcat_id
, subcat.description AS subcat_description
, subcat.safe_url AS subcat_safe_url
, topics.topic_id
, topics.topic_safe_url
, topics.topic_title
, topics.last_post_time
, topics.topic_last_poster_name
, topics.topic_last_poster_id
, (
SELECT
posts_read.last_read_time
FROM
forum_topics a
LEFT JOIN
forum_posts_read AS posts_read ON
posts_read.last_read_time > a.last_post_time
AND posts_read.last_read_time > 1321004546
AND posts_read.topic_id = a.topic_id
AND posts_read.user_id = 1
AND a.forum_id = Root.id
LIMIT 1) AS last_read_time
FROM forum_category AS root
LEFT JOIN
forum_category AS subcat ON subcat.parent_id = root.id
LEFT JOIN
forum_topics AS topics ON topics.forum_id = subcat.id
LEFT JOIN
forum_topics AS t2 ON t2.forum_id = subcat.id AND t2.last_post_time > topics.last_post_time
WHERE
root.parent_id = 0 AND t2.forum_id IS NULL
ORDER BY
root_name, subcat_name
Related
Hi I have 8 million row data and need to optimize Mysql query to fetch a row from that data. I am using below query but its server response time is too high that creating issue in page loading speed
SELECT q.id
, q.title
, q.question
, q.price
, q.created
, q.userid
, q.duedate
, q.tags
, s.id subjectid
, sc.id subcategoryid
, s.title subject
, sc.title subcategory
, q.statusid
, (SELECT COUNT(id) FROM tbl_answers a WHERE a.questionid = q.id AND a.statusid = 1 AND a.deleted = 'N') q_num_answers
, u.username
, u.image
, u.gender
, (SELECT COUNT(id) FROM tbl_answers a WHERE a.userid = q.userid AND a.statusid = 1 AND a.deleted = 'N') num_answers
, (SELECT COUNT(id) FROM tbl_questions WHERE userid = q.userid AND statusid = 1 AND deleted = 'N') AS num_questions
, 0 amt_earned
, 0 amt_spent
, 0 num_sold
, (SELECT COUNT(ur.id) FROM tbl_users_ratings ur WHERE ur.userid = q.userid AND ur.deleted = 'N') u_num_ratings
, (SELECT COALESCE(SUM(ur.rating), 0) FROM tbl_users_ratings ur WHERE ur.userid = q.userid AND ur.deleted = 'N') u_score
FROM tbl_questions q
JOIN tbl_subjects s
ON q.subject = s.id
JOIN tbl_subjects sc
ON q.subcategory = sc.id
LEFT
JOIN tbl_users u
ON u.id = q.userid
WHERE q.deleted = '$show_deleted'
AND q.id = ?
LIMIT 1
These indexes may help. (I am assuming that id is the PRIMARY KEY of each table.)
ur: (deleted, userid, rating)
a: (deleted, statusid, userid)
a: (deleted, statusid, questionid)
Please provide EXPLAIN SELECT ....
Don't use COUNT(id) unless you need to check for id not being NULL. The usual way to write it is COUNT(*).
In one place you are checking for a provided values for deleted. In another, you hard code it. Perhaps wrong?
AND ur.deleted = 'N'
If the PRIMARY KEY for q is id, then this will lead to either 1 row or no row. What am I missing?
WHERE q.deleted = '$show_deleted'
AND q.id = ?
(There may be more tips. Please change the things I suggested and follow the suggestions from others. Then let's have another look.)
I'm not sure how to make the following SQL query more efficient. Right now, the query is taking 8 - 12 seconds on a pretty fast server, but that's not close to fast enough for a Website when users are trying to load a page with this code on it. It's looking through tables with many rows, for instance the "Post" table has 717,873 rows. Basically, the query lists all Posts related to what the user is following (newest to oldest).
Is there a way to make it faster by only getting the last 20 results total based on PostTimeOrder?
Any help would be much appreciated or insight on anything that can be done to improve this situation. Thank you.
Here's the full SQL query (lots of nesting):
SELECT DISTINCT p.Id, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime, p.Content AS Content, p.Bu AS Bu, p.Se AS Se, UNIX_TIMESTAMP(p.PostCreationTime) AS PostTimeOrder
FROM Post p
WHERE (p.Id IN (SELECT pc.PostId
FROM PostCreator pc
WHERE (pc.UserId IN (SELECT uf.FollowedId
FROM UserFollowing uf
WHERE uf.FollowingId = '100')
OR pc.UserId = '100')
))
OR (p.Id IN (SELECT pum.PostId
FROM PostUserMentions pum
WHERE (pum.UserId IN (SELECT uf.FollowedId
FROM UserFollowing uf
WHERE uf.FollowingId = '100')
OR pum.UserId = '100')
))
OR (p.Id IN (SELECT ssp.PostId
FROM SStreamPost ssp
WHERE (ssp.SStreamId IN (SELECT ssf.SStreamId
FROM SStreamFollowing ssf
WHERE ssf.UserId = '100'))
))
OR (p.Id IN (SELECT psm.PostId
FROM PostSMentions psm
WHERE (psm.StockId IN (SELECT sf.StockId
FROM StockFollowing sf
WHERE sf.UserId = '100' ))
))
UNION ALL
SELECT DISTINCT p.Id AS Id, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime, p.Content AS Content, p.Bu AS Bu, p.Se AS Se, UNIX_TIMESTAMP(upe.PostEchoTime) AS PostTimeOrder
FROM Post p
INNER JOIN UserPostE upe
on p.Id = upe.PostId
INNER JOIN UserFollowing uf
on (upe.UserId = uf.FollowedId AND (uf.FollowingId = '100' OR upe.UserId = '100'))
ORDER BY PostTimeOrder DESC;
Changing your p.ID in (...) predicates to existence predicates with correlated subqueries may help. Also since both halves of your union all query are pulling from the Post table and possibly returning nearly identical records you might be able to combine the two into one query by left outer joining to UserPostE and adding upe.PostID is not null as an OR condition in the WHERE clause. UserFollowing will still inner join to UPE. If you want the same Post record twice once with upe.PostEchoTime and once with p.PostCreationTime as the PostTimeOrder you'll need keep the UNION ALL
SELECT
DISTINCT -- <<=- May not be needed
p.Id
, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime
, p.Content AS Content
, p.Bu AS Bu
, p.Se AS Se
, UNIX_TIMESTAMP(coalesce( upe.PostEchoTime
, p.PostCreationTime)) AS PostTimeOrder
FROM Post p
LEFT JOIN UserPostE upe
INNER JOIN UserFollowing uf
on (upe.UserId = uf.FollowedId AND
(uf.FollowingId = '100' OR
upe.UserId = '100'))
on p.Id = upe.PostId
WHERE upe.PostID is not null
or exists (SELECT 1
FROM PostCreator pc
WHERE pc.PostId = p.ID
and pc.UserId = '100'
or exists (SELECT 1
FROM UserFollowing uf
WHERE uf.FollowedId = pc.UserID
and uf.FollowingId = '100')
)
OR exists (SELECT 1
FROM PostUserMentions pum
WHERE pum.PostId = p.ID
and pum.UserId = '100'
or exists (SELECT 1
FROM UserFollowing uf
WHERE uf.FollowedId = pum.UserId
and uf.FollowingId = '100')
)
OR exists (SELECT 1
FROM SStreamPost ssp
WHERE ssp.PostId = p.ID
and exists (SELECT 1
FROM SStreamFollowing ssf
WHERE ssf.SStreamId = ssp.SStreamId
and ssf.UserId = '100')
)
OR exists (SELECT 1
FROM PostSMentions psm
WHERE psm.PostId = p.ID
and exists (SELECT
FROM StockFollowing sf
WHERE sf.StockId = psm.StockId
and sf.UserId = '100' )
)
ORDER BY PostTimeOrder DESC
The from section could alternatively be rewritten to also use an existence clause with a correlated sub query:
FROM Post p
LEFT JOIN UserPostE upe
on p.Id = upe.PostId
and ( upe.UserId = '100'
or exists (select 1
from UserFollowing uf
where uf.FollwedID = upe.UserID
and uf.FollowingId = '100'))
Turn IN ( SELECT ... ) into a JOIN .. ON ... (see below)
Turn OR into UNION (see below)
Some the tables are many:many mappings? Such as SStreamFollowing? Follow the tips in http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
Example of IN:
SELECT ssp.PostId
FROM SStreamPost ssp
WHERE (ssp.SStreamId IN (
SELECT ssf.SStreamId
FROM SStreamFollowing ssf
WHERE ssf.UserId = '100' ))
-->
SELECT ssp.PostId
FROM SStreamPost ssp
JOIN SStreamFollowing ssf ON ssp.SStreamId = ssf.SStreamId
WHERE ssf.UserId = '100'
The big WHERE with all the INs becomes something like
JOIN ( ( SELECT pc.PostId AS id ... )
UNION ( SELECT pum.PostId ... )
UNION ( SELECT ssp.PostId ... )
UNION ( SELECT psm.PostId ... ) )
Get what you can done of that those suggestions, then come back for more advice if you still need it. And bring SHOW CREATE TABLE with you.
I have added this simple example, to show the problem I am having. Say this is my database foo.
SELECT
`people`.`id`
, `people`.`name`
, `places`.`place_type`
, `places`.`aread`
, `family`.`family_count`
, `income`.`income_value`
, `income`.`average`
, `employment`.`address`
, `employment`.`duration`
FROM
`people`
INNER JOIN `places`
ON (`people`.`id` = `places`.`id`)
INNER JOIN `family`
ON (`people`.`id` = `family`.`id`)
INNER JOIN `income`
ON (`people`.`id` = `income`.`id`)
INNER JOIN `employment`
ON (`people`.`id` = `employment`.`id`) WHERE `people`.`full_name` = ? LIMIT 1
Now the issue here is that I get null values if people.id is not matching every id in the other columns. This is because not all of people with id are found in places, family, income ... columns.
So, I would like to know how to always returns a value from people.id even if for example people.id is not same as say .. income.id
Just change your joins to LEFT JOIN which return all the records from the LEFT table and only the matching from the RIGHT tables :
SELECT
`people`.`id`
, `people`.`name`
, `places`.`place_type`
, `places`.`aread`
, `family`.`family_count`
, `income`.`income_value`
, `income`.`average`
, `employment`.`address`
, `employment`.`duration`
FROM
`people`
LEFT JOIN `places`
ON (`people`.`id` = `places`.`id`)
LEFT JOIN `family`
ON (`people`.`id` = `family`.`id`)
LEFT JOIN `income`
ON (`people`.`id` = `income`.`id`)
LEFT JOIN `employment`
ON (`people`.`id` = `employment`.`id`) WHERE `people`.`full_name` = ? LIMIT 1
EDIT: If you want to know which tables didn't have a match, just search for null values. When left joining, the tables that didn't match the condition will get NULL values on their column selection.
Replace LEFT JOIN with INNER JOIN.
SELECT
`people`.`id`
, `people`.`name`
, `places`.`place_type`
, `places`.`aread`
, `family`.`family_count`
, `income`.`income_value`
, `income`.`average`
, `employment`.`address`
, `employment`.`duration`
FROM
`people`
LEFT JOIN `places`
ON (`people`.`id` = `places`.`id`)
LEFT JOIN `family`
ON (`people`.`id` = `family`.`id`)
LEFT JOIN `income`
ON (`people`.`id` = `income`.`id`)
LEFT JOIN `employment`
ON (`people`.`id` = `employment`.`id`) WHERE `people`.`full_name` = ? LIMIT 1
I have a set of articles. Some of these articles have been assigned x number of tags.
I want to select all articles in the format specified in the SELECT, and if the article also has any number of tags assigned, put them into a single GROUP_CONCAT field.
The SQL below does everything I need, except it only retrieves articles that have at least one tag.
How can I make it so that the tagging requirement is optional?
SELECT d.entry_id AS entry_id
, d.field_id_40 AS body
, t.title AS title
, t.url_title AS url_title
, t.entry_date AS entry_date
, GROUP_CONCAT(g.tag_name ORDER BY g.tag_name) AS tag
FROM exp_channel_data d
, exp_category_posts c
, exp_channel_titles t
, exp_tagger_links l
, exp_tagger g
WHERE t.YEAR > '2005'
AND t.site_id = d.site_id
AND d.site_id = 9
AND ( c.cat_id = 95
OR c.cat_id = 93
OR c.cat_id = 64
OR c.cat_id = 24
)
AND d.entry_id = t.entry_id
AND t.entry_id = c.entry_id
AND t.STATUS = 'open'
AND l.tag_id = g.tag_id
AND d.entry_id = l.entry_id
GROUP BY d.entry_id
Switching your JOINS to use the JOIN syntax, this should provide you with your answer.
SELECT d.entry_id AS entry_id
, d.field_id_40 AS body
, t.title AS title
, t.url_title AS url_title
, t.entry_date AS entry_date
, GROUP_CONCAT(g.tag_name ORDER BY g.tag_name) AS tag
FROM exp_channel_data AS d
INNER JOIN exp_channel_titles AS t ON d.site_id = t.site_id
AND d.entry_id = t.entry_id
INNER JOIN exp_category_posts AS c ON t.entry_id = c.entry_id
LEFT OUTER JOIN exp_tagger_links AS l ON d.entry_id = l.entry_id
LEFT OUTER JOIN exp_tagger AS g ON l.tag_id = g.tag_id
WHERE t.YEAR > '2005'
AND d.site_id = 9
AND c.cat_id IN (95, 93, 64, 24)
AND t.STATUS = 'open'
GROUP BY d.entry_id
You may also want to look at A visual explanation of SQL joins.
SELECT `v`.*
, GROUP_CONCAT(DISTINCT(`gg`.genre_id)) AS `genre_ids`
, GROUP_CONCAT(DISTINCT(`gg`.genre_ru)) AS `genres`
, GROUP_CONCAT(DISTINCT(`tg`.tag_id)) AS `tags_ids`
, GROUP_CONCAT(DISTINCT(`tg`.name)) AS `tags`
FROM `video` AS `v`
LEFT JOIN `video_categories` AS `gc` ON gc.video_id = v.video_id
LEFT JOIN `video_genres` AS `gg` ON gg.genre_id = gc.genre_id
LEFT JOIN `tags_relations` AS `tr` ON tr.department=18 AND tr.parent_id = v.video_id
LEFT JOIN `tags` AS `tg` ON tg.tag_id = tr.tag_id
WHERE (v.video_id IN (
SELECT gc2.video_id
FROM video_categories gc2
WHERE gc2.genre_id IN(44)
GROUP BY gc2.video_id
HAVING COUNT(gc2.video_id)=1))
AND (v.video_id IN (
SELECT gc2.parent_id
FROM tags_relations gc2
WHERE gc2.tag_id IN(14) AND gc2.department=18
GROUP BY gc2.parent_id
HAVING COUNT(gc2.parent_id)=1))
GROUP BY `v`.`video_id`
ORDER BY `v`.`video_id` DESC
Query takes 0.4s~ to run. It's because of subquery in WHERE statement:
v.video_id IN (
SELECT gc2.parent_id
FROM tags_relations gc2
WHERE gc2.tag_id IN(14) AND gc2.department=18
GROUP BY gc2.parent_id
HAVING COUNT(gc2.parent_id)=1
)
Without it, it runs 0.07s~
Maybe try something with advanced joins? shy
Thanks ;)
In MySQL, never use WHERE in (SELECT ... it is notoriously slow.
Use a join instead:
SELECT `v`.*
, GROUP_CONCAT(DISTINCT(`gg`.genre_id)) AS `genre_ids`
, GROUP_CONCAT(DISTINCT(`gg`.genre_ru)) AS `genres`
, GROUP_CONCAT(DISTINCT(`tg`.tag_id)) AS `tags_ids`
, GROUP_CONCAT(DISTINCT(`tg`.name)) AS `tags`
FROM video AS v
INNER JOIN video_categories AS gc ON gc.video_id = v.video_id
AND gc2.genre_id = '44'
LEFT JOIN video_genres AS gg ON gg.genre_id = gc.genre_id
LEFT JOIN tags_relations AS tr ON tr.department=18 AND tr.parent_id = v.video_id
LEFT JOIN tags AS tg ON tg.tag_id = tr.tag_id
INNER JOIN tags_relations tr2 ON (tr2.parent_id = v.video_id
AND tr2.tag_id = 14 AND tr2.department=18
GROUP BY `v`.`video_id`
ORDER BY `v`.`video_id` DESC
SELECT `v`.*
, GROUP_CONCAT(DISTINCT(`gg`.genre_id)) AS `genre_ids`
, GROUP_CONCAT(DISTINCT(`gg`.genre_ru)) AS `genres`
, GROUP_CONCAT(DISTINCT(`tg`.tag_id)) AS `tags_ids`
, GROUP_CONCAT(DISTINCT(`tg`.name)) AS `tags`
FROM `video` AS `v`
LEFT JOIN `video_categories` AS `gc` ON gc.video_id = v.video_id
LEFT JOIN `video_genres` AS `gg` ON gg.genre_id = gc.genre_id
LEFT JOIN `tags_relations` AS `tr` ON tr.department=18 AND tr.parent_id = v.video_id
LEFT JOIN `tags` AS `tg` ON tg.tag_id = tr.tag_id
JOIN (
SELECT gc2.video_id
FROM video_categories gc2
WHERE gc2.genre_id IN(44)
GROUP BY gc2.video_id
HAVING COUNT(gc2.video_id)=1
) vc ON v.video_id = vc.video_id
JOIN (
SELECT gc2.parent_id
FROM tags_relations gc2
WHERE gc2.tag_id IN(14) AND gc2.department=18
GROUP BY gc2.parent_id
HAVING COUNT(gc2.parent_id)=1
) tr ON v.video_id = tr.parent_id
GROUP BY `v`.`video_id`
ORDER BY `v`.`video_id` DESC
Edit:
I took a closer look at your query and think it can be reduced to the following:
SELECT `v1`.*
, GROUP_CONCAT(DISTINCT(`gg`.genre_id)) AS `genre_ids`
, GROUP_CONCAT(DISTINCT(`gg`.genre_ru)) AS `genres`
, GROUP_CONCAT(DISTINCT(`tg`.tag_id)) AS `tags_ids`
, GROUP_CONCAT(DISTINCT(`tg`.name)) AS `tags`
FROM (
SELECT v.*
FROM `video` AS `v`
LEFT JOIN `video_categories` AS `gc` ON gc.video_id = v.video_id
LEFT JOIN `tags_relations` AS `tr` ON tr.parent_id = v.video_id
WHERE gc.genre_id IN(44)
AND tr.tag_id IN(14)
AND tr.department=18
GROUP BY `v`.`video_id`
HAVING COUNT(*) = 1
) v1
LEFT JOIN `video_genres` AS `gg` ON gg.genre_id = gc.genre_id
LEFT JOIN `tags` AS `tg` ON tg.tag_id = tr.tag_id
GROUP BY v1.video_id
ORDER BY v1.video_id DESC