My goal is to get from the following tables - the user's unique group names and ids, the latest comments for the user's groups, the latest "done" article for the user's groups, the SUM of done articles, and total articles. Basically what is presented in the bottom sheet.
So far I've managed to get the data from the groups table and from the articles, but I can't get the latest comment.
Here is my query
SELECT `groups`.`name` , `groups`.`id` , (
SELECT MAX( `articles`.`written` )
FROM `articles`
WHERE `group` = `groups`.`id`
AND `articles`.`done` = '1'
) AS latestArt, (
SELECT MAX( `comments`.`date_added` )
FROM `comments`
WHERE `comments`.`article_id` = `a`.`id`
AND `comments`.`active` = '1'
) AS latestComm, SUM( `a`.`done` = '1' ) articlesAchieved, COUNT( `a`.`id` ) AS totalArticles
FROM `groups`
LEFT JOIN `articles` AS `a` ON `a`.`group` = `groups`.`id`
LEFT JOIN `comments` AS `c` ON `c`.`note_id` = `a`.`id`
WHERE `groups`.`user_id` = '6'
AND `n`.`active` = '1'
GROUP BY `groups`.`id`
I've also tried to get the data by joining everything to the article table but I wasn't successful with that either :(
UPDATED Your query might look like this
SELECT g.id group_id, g.name group_name,
a.last_written, a.total_articles, a.total_done,
c.last_comment
FROM groups g LEFT JOIN
(
SELECT `group`,
MAX(CASE WHEN done = 1 THEN written END) last_written,
COUNT(*) total_articles,
SUM(done) total_done
FROM articles
WHERE active = 1
AND user_id = 1
GROUP BY `group`
) a
ON g.id = a.`group` LEFT JOIN
(
SELECT a.`group`,
MAX(date_added) last_comment
FROM commants c JOIN articles a
ON c.article_id = a.id
WHERE a.active = 1
AND a.user_id = 1
GROUP BY a.`group`
) c
ON g.id = c.`group`
WHERE user_id = 1
Related
I have an incidents table which has a 1 to many relationship with a few tables - mainly, for the context of this question, people.
Basically, one incident may have many people (involved).
At the moment, I'm retrieving the incident details - plus a concatenated comma-delimited string of people's IDs using this query:
SELECT
i.`ID` AS `id`,
i.`Author_ID` AS `author_id`,
i.`Description` AS `description`,
i.`Date` AS `date`,
i.`Datetime_Created` AS `created`,
p.`Title` AS `period`,
GROUP_CONCAT(DISTINCT ip.`Person_ID` ORDER BY FIELD(ip.`Involvement`, 'V', 'P', 'W') ASC SEPARATOR ',') AS `people_ids`,
( SELECT COUNT(`ID`) FROM `reports` r WHERE r.Incident_ID = i.ID ) AS `reports`,
i.`Status` AS `status`
FROM `incidents` i
LEFT JOIN `reports` ir ON ir.Incident_ID = i.ID
LEFT JOIN `people` ip ON ip.Incident_ID = i.ID
LEFT JOIN `periods` p ON i.Period_ID = p.ID
WHERE 1 NOT IN ( SELECT Category_ID FROM `categories_link` WHERE `Incident_ID` = i.ID )
GROUP BY i.ID
ORDER BY i.`Date` DESC, p.`ID` DESC
This works fine, and produces data like:
What I'm trying to do now is filter these reports so that only incidents where one of the people involved is a student from a certain year group.
This information can be found by joining their IDs to the students table. The students table contains their ID and a Year_Group field.
One of the complexities is that some of the IDs from the people_involved table may not relate just to students - they could be staff, parents or other members of our community.
I don't want to exclude reports which have other people involved, as long as there is a student from a specific year group involved too.
I've written a query which seems to partially work:
SELECT
i.`ID` AS `id`,
i.`Author_ID` AS `author_id`,
i.`Description` AS `description`,
i.`Date` AS `date`,
i.`Datetime_Created` AS `created`,
p.`Title` AS `period`,
GROUP_CONCAT(DISTINCT ip.`Person_ID` ORDER BY FIELD(ip.`Involvement`, 'V', 'P', 'W') ASC SEPARATOR ',') AS `people_ids`,
( SELECT COUNT(`ID`) FROM `reports` r WHERE r.Incident_ID = i.ID ) AS `reports`,
i.`Status` AS `status`
FROM `incidents` i
LEFT JOIN `reports` ir ON ir.Incident_ID = i.ID
LEFT JOIN `people` ip ON ip.Incident_ID = i.ID
<< LEFT JOIN `student` stu ON ip.Person_ID = stu.db_id >>
LEFT JOIN `periods` p ON i.Period_ID = p.ID
WHERE 1 NOT IN ( SELECT Category_ID FROM `categories_link` WHERE `Incident_ID` = i.ID )
<< AND `stu`.`Year_Group` = 11 >>
GROUP BY i.ID
ORDER BY i.`Date` DESC, p.`ID` DESC
But I just can't imagine that a single simple JOIN would be sufficient for the task I'm trying to achieve.
I think a subquery might do it, but I don't know where to begin with that.
The code I would use to access this information (for year 7 students) without all of the necessary incidents data would be (I think):
SELECT DISTINCT( p.`Incident_ID` )
FROM `people` p
LEFT JOIN `student` stu ON p.Person_ID = stu.db_id
WHERE stu.Year_Group = 7
How do I bundle that into this code?
To get incidents where students of only specific age group is included,use the following query.
SELECT p.Incident_ID
FROM people p
JOIN student stu ON p.Person_ID = stu.db_id
WHERE stu.Year_Group = 11
group by p.Incident_ID
Your original query returns the incidents and the group of people involved ,So in your original query filter incidents by comparing with the above query written by me.This way you will get all incidents where students from a specific year group involved plus other people also involved(if any).I think this will solve your problem.
SELECT
i.`ID` AS `id`,
i.`Author_ID` AS `author_id`,
i.`Description` AS `description`,
i.`Date` AS `date`,
i.`Datetime_Created` AS `created`,
p.`Title` AS `period`,
GROUP_CONCAT(DISTINCT ip.`Person_ID` ORDER BY FIELD(ip.`Involvement`, 'V', 'P', 'W') ASC SEPARATOR ',') AS `people_ids`,
( SELECT COUNT(`ID`) FROM `reports` r WHERE r.Incident_ID = i.ID ) AS `reports`,
i.`Status` AS `status`
FROM `incidents` i
LEFT JOIN `reports` ir ON ir.Incident_ID = i.ID
LEFT JOIN `people` ip ON ip.Incident_ID = i.ID
LEFT JOIN `periods` p ON i.Period_ID = p.ID
WHERE 1 NOT IN ( SELECT Category_ID FROM `categories_link` WHERE `Incident_ID` = i.ID )
and i.ID in //Here you will put the above query
(
SELECT p.Incident_ID
FROM people p
JOIN student stu ON p.Person_ID = stu.db_id
WHERE stu.Year_Group = 11
group by p.Incident_ID
)
GROUP BY i.ID
ORDER BY i.`Date` DESC, p.`ID` DESC
It looks to me like you want an OUTER JOIN on students.
LEFT OUTER JOIN 'student' stu on ip.Person_ID = stu.db_id
That will include all the incidents. Then, in the WHERE clause, add the filter
WHERE 1 NOT IN ( SELECT Category_ID FROM `categories_link` WHERE `Incident_ID` = i.ID ) AND `stu`.`Year_Group` = 7
I don't know much about query optimization but I know the order in which queries get executed
FROM clause
WHERE clause
GROUP BY clause
HAVING clause
SELECT clause
ORDER BY clause
This the query I had written
SELECT
`main_table`.forum_id,
my_topics.topic_id,
(
SELECT MAX(my_posts.post_id) FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id
) AS `maxpostid`,
(
SELECT my_posts.admin_user_id FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id ORDER BY my_posts.post_id DESC LIMIT 1
) AS `admin_user_id`,
(
SELECT my_posts.user_id FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id ORDER BY my_posts.post_id DESC LIMIT 1
) AS `user_id`,
(
SELECT COUNT(my_topics.topic_id) FROM my_topics WHERE my_topics.forum_id = main_table.forum_id ORDER BY my_topics.forum_id DESC LIMIT 1
) AS `topicscount`,
(
SELECT COUNT(my_posts.post_id) FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id ORDER BY my_topics.topic_id DESC LIMIT 1
) AS `postcount`,
(
SELECT CONCAT(admin_user.firstname,' ',admin_user.lastname) FROM admin_user INNER JOIN my_posts ON my_posts.admin_user_id = admin_user.user_id WHERE my_posts.post_id = maxpostid ORDER BY my_posts.post_id DESC LIMIT 1
) AS `adminname`,
(
SELECT forum_user.nick_name FROM forum_user INNER JOIN my_posts ON my_posts.user_id = forum_user.user_id WHERE my_posts.post_id = maxpostid ORDER BY my_posts.post_id DESC LIMIT 1
) AS `nickname`,
(
SELECT CONCAT(ce1.value,' ',ce2.value) AS fullname FROM my_posts INNER JOIN customer_entity_varchar AS ce1 ON ce1.entity_id = my_posts.user_id INNER JOIN customer_entity_varchar AS ce2 ON ce2.entity_id=my_posts.user_id WHERE (ce1.attribute_id = 1) AND (ce2.attribute_id = 2) AND my_posts.post_id = maxpostid ORDER BY my_posts.post_id DESC LIMIT 1
) AS `fullname`
FROM `my_forums` AS `main_table`
LEFT JOIN `my_topics` ON main_table.forum_id = my_topics.forum_id
WHERE (forum_status = '1')
And now I want to know if there is any way to optimize it ? Because all the logic is written in Select section not From, but I don't know how to write the same logic in From section of the query ?
Does it make any difference or both are same ?
Thanks
Correlated subqueries should really be a last resort, they often end up being executed RBAR, and given that a number of your subqueries are very similar, trying to get the same result using joins is going to result in a lot less table scans.
The first thing I note is that all of your subqueries include the table my_posts, and most contain ORDER BY my_posts.post_id DESC LIMIT 1, those that don't have a count with no group by so the order and limit are redundant anyway, so my first step would be to join to my_posts:
SELECT *
FROM my_forums AS f
LEFT JOIN my_topics AS t
ON f.forum_id = t.forum_id
LEFT JOIN
( SELECT topic_id, MAX(post_id) AS post_id
FROM my_posts
GROUP BY topic_id
) AS Maxp
ON Maxp.topic_id = t.topic_id
LEFT JOIN my_posts AS p
ON p.post_id = Maxp.post_id
WHERE forum_status = '1';
Here the subquery just ensures you get the latest post per topic_id. I have shortened your table aliases here for my convenience, I am not sure why you would use a table alias that is longer than the actual table name?
Now you have the bulk of your query you can start adding in your columns, in order to get the post count, I have added a count to the subquery Maxp, I have also had to add a few more joins to get some of the detail out, such as names:
SELECT f.forum_id,
t.topic_id,
p.post_id AS `maxpostid`,
p.admin_user_id,
p.user_id,
t2.topicscount,
maxp.postcount,
CONCAT(au.firstname,' ',au.lastname) AS adminname,
fu.nick_name AS nickname
CONCAT(ce1.value,' ',ce2.value) AS fullname
FROM my_forums AS f
LEFT JOIN my_topics AS t
ON f.forum_id = t.forum_id
LEFT JOIN
( SELECT topic_id,
MAX(post_id) AS post_id,
COUNT(*) AS postcount
FROM my_posts
GROUP BY topic_id
) AS Maxp
ON Maxp.topic_id = t.topic_id
LEFT JOIN my_posts AS p
ON p.post_id = Maxp.post_id
LEFT JOIN admin_user AS au
ON au.admin_user_id = p.admin_user_id
LEFT JOIN forum_user AS fu
ON fu.user_id = p.user_id
LEFT JOIN customer_entity_varchar AS ce1
ON ce1.entity_id = p.user_id
AND ce1.attribute_id = 1
LEFT JOIN customer_entity_varchar AS ce2
ON ce2.entity_id = p.user_id
AND ce2.attribute_id = 2
LEFT JOIN
( SELECT forum_id, COUNT(*) AS topicscount
FROM my_topics
GROUP BY forum_id
) AS t2
ON t2.forum_id = f.forum_id
WHERE forum_status = '1';
I am not familiar with your schema so the above may need some tweaking, but the principal remains - use JOINs over sub-selects.
The next stage of optimisation I would do is to get rid of your customer_entity_varchar table, or at least stop using it to store things as basic as first name and last name. The Entity-Attribute-Value model is an SQL antipattern, if you added two columns, FirstName and LastName to your forum_user table you would immediately lose two joins from your query. I won't get too involved in the EAV vs Relational debate as this has been extensively discussed a number of times, and I have nothing more to add.
The final stage would be to add appropriate indexes, you are in the best decision to decide what is appropriate, I'd suggest you probably want indexes on at least the foreign keys in each table, possibly more.
EDIT
To get one row per forum_id you would need to use the following:
SELECT f.forum_id,
t.topic_id,
p.post_id AS `maxpostid`,
p.admin_user_id,
p.user_id,
MaxT.topicscount,
maxp.postcount,
CONCAT(au.firstname,' ',au.lastname) AS adminname,
fu.nick_name AS nickname
CONCAT(ce1.value,' ',ce2.value) AS fullname
FROM my_forums AS f
LEFT JOIN
( SELECT t.forum_id,
COUNT(DISTINCT t.topic_id) AS topicscount,
COUNT(*) AS postCount,
MAX(t.topic_ID) AS topic_id
FROM my_topics AS t
INNER JOIN my_posts AS p
ON p.topic_id = p.topic_id
GROUP BY t.forum_id
) AS MaxT
ON MaxT.forum_id = f.forum_id
LEFT JOIN my_topics AS t
ON t.topic_ID = Maxt.topic_ID
LEFT JOIN
( SELECT topic_id, MAX(post_id) AS post_id
FROM my_posts
GROUP BY topic_id
) AS Maxp
ON Maxp.topic_id = t.topic_id
LEFT JOIN my_posts AS p
ON p.post_id = Maxp.post_id
LEFT JOIN admin_user AS au
ON au.admin_user_id = p.admin_user_id
LEFT JOIN forum_user AS fu
ON fu.user_id = p.user_id
LEFT JOIN customer_entity_varchar AS ce1
ON ce1.entity_id = p.user_id
AND ce1.attribute_id = 1
LEFT JOIN customer_entity_varchar AS ce2
ON ce2.entity_id = p.user_id
AND ce2.attribute_id = 2
WHERE forum_status = '1';
Here is a sample SQL dump: https://gist.github.com/JREAM/99287d033320b2978728
I have a SELECT that grabs a bundle of users.
I then do a foreach loop to attach all the associated tree_processes to that user.
So I end up doing X Queries: users * tree.
Wouldn't it be much more efficient to fetch the two together?
I've thought about doing a LEFT JOIN Subselect, but I'm having a hard time getting it correct.
Below I've done a query to select the correct data in the SELECT, however I would have to do this for all 15 rows and it seems like a TERRIBLE waste of memory.
This is my dirty Ateempt:
-
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
(
SELECT tp.id FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS newest_tree_id,
#
# Don't want to have to do this below for every row
(
SELECT t.type FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS tree_type
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
This is my LEFT JOIN attempt, but I am having trouble getting the SELECT values
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
freshness.id,
# freshness.subscribers_id < -- Cant get multiples out of the LEFT join
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
LEFT JOIN ( SELECT tp.id, tp.subscribers_id AS tp FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
ORDER BY tp.id DESC
LIMIT 1 ) AS freshness
ON (
s.id = subscribers_id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
In the LEFT JOIN you are using 'freshness' as the table alias. This in you select you need to additionally state what column(s) you want from it. Since there is only one column (id) you need to add:
freshness.id
to the select clause.
Your ON clause of the left join looks pretty dodgy too. Maybe freshness.id = ss.subscribers_id?
Cheers -
I'm trying to get all topics along with the last comment in each topic. I've tried a couple of different sql statements, which haven't been working out.
SELECT
a.*,
b.*,
c.*,
(SELECT COUNT(*) FROM comments WHERE comment_topic_id = a.topic_id) AS count
FROM topics AS a
LEFT JOIN categories AS b ON a.topic_category = b.category_id
LEFT JOIN (
SELECT *
FROM comments
ORDER BY comment_date DESC
) AS c ON a.topic_id = c.comment_topic_id
WHERE b.category_id = '1' AND b.category_permission <= '2'
ORDER BY a.topic_created ASC
The above code will generate a result for each comment instead of the most recent.
Any help is appreciated, I can provide images to illustrate the database and table structures
I've changed the alias of your count because count is a reserved word.
Try this:
EDIT
SELECT
a.*,
b.*,
co.*,
(SELECT COUNT(*) FROM comments WHERE comment_topic_id = a.topic_id) AS tot_comment
FROM topics AS a
JOIN categories AS b ON a.topic_category = b.category_id
LEFT JOIN (
SELECT *
FROM comments c
WHERE NOT EXISTS(
SELECT 'NEXT'
FROM comments c2
WHERE c2.comment_topic_id = c.comment_topic_id
AND c2.comment_date > c.comment_date
)
) AS co ON a.topic_id = co.comment_topic_id
WHERE b.category_id = '1' AND b.category_permission <= '2'
ORDER BY a.topic_created ASC
I'm working with a mysql query that is supposed to select all messages addressed or sent by the user. I need to group all messages with same UID so that I show a single thread for each differente user (this means it should eliminate all messages except the last with same UID). My problem is that I started using GROUP BY to do it but sometimes the row that remains is actually the older message instead of the latest.
This is what I was trying:
SELECT `UID`, `Name`, `Text`, `A`.`Date`
FROM `Users`
INNER JOIN (
(
SELECT *, To_UID AS UID FROM `Messages` WHERE `From_UID` = '$userID' AND `To_UID` != '$userID'
)
UNION ALL
(
SELECT *, From_UID AS UID FROM `Messages` WHERE `To_UID` = '$userID' AND `From_UID` != '$userID'
)
) AS A
ON A.UID = Users.ID
GROUP BY UID // This doesn't work
How can I show only the row with the most resent date per UID?
use DISTINCT and only use ORDER BY date
GROUP BY actually sometimes displays a random row, which isn't always commonly discussed.
you can try some thing like this:
select UID, Name, Text, c.date
from User
inner join (
select if(b.From_UID = '$userID', b.To_UID, b.From_UID) as UID,
*
from Messages as b
inner join(
select if(c.From_UID = '$userID', c.To_UID, c.From_UID) as UID,
max(c.date) as date
from Messages as c
where c.From_UID = '$userID' or c.To_UID = '$userID'
group by UID
) as d on d.date = b.date and d.UID = b.UID
) as e on e.UID = Users.id
)
or create a temp table / stored procedure to make life easier
Temp table
create temp table t
select if(From_UID = '$userID', To_UID, From_UID) as UID, * from Messages
select UID, Name, Text, date
from User
inner join (
select *
from t as t1
inner join(
select
t2.UID,
max(t2.date) as date
from t as t2
group by t2.UID
) as t3 on t3.date = t1.date and t3.UID = t1.UID
) as e on e.UID = Users.id