MySQL Subquery Select very slow - mysql

One of our websites has a table with about 60'000 records in it. Recently we noticed the page was timing out and could only be resolved by setting the memory limit to -1. This allowed the page to load but it was very slow. Furthermore I did not believe this was the correct way to resolve the problem, as obviously it indicates something is not quite right.
I managed to output the query that the page was running:
SELECT u.*,
(SELECT COUNT(*) FROM enquiry e WHERE e.user_id = u.id AND e.deleted = 0 AND e.time_started != 0) AS opened_count,
(SELECT COUNT(*) FROM enquiry e WHERE e.user_id = u.id AND e.deleted = 0 AND e.confirmed = 1 AND e.time_started != 0) AS confirmed_count,
(SELECT COUNT(*) FROM enquiry e WHERE e.user_id = u.id AND e.deleted = 0 AND e.call_back = 1 AND e.time_started != 0) AS call_back_count
FROM user u
WHERE u.active = 1 AND u.deleted = 0
ORDER BY u.username
I ran this query in phpMyAdmin and it takes over 30 seconds to return the results.
I feel the query needs optimising in some way but I'm struggling to work out how. I'm guessing I need to use a JOIN of some sort?

You're really running >180,000 queries, since each of those 3 subqueries will be run once for every row in the user table.
You could try simplifying into a standard join with some groups, e.g.
SELECT user.*,
COUNT(enq.id) AS opened_count
SUM(e.confirmed = 1) AS confirmed_count
SUM(e.call_back = 1) AS call_back_count
FROM user
LEFT JOIN enquiry ON enquiry.user_id = user.id
WHERE user.active = 1 and user.deleted AND enquiry.deleted = 0
GROUP BY user.id

Related

How to find a result with multiple different values on same column?

Imagine that we have a database with a logs table and types table. I want to do a query where I figure out if UserX has entries for certain types of logs. Let's say that UserX has logged type_1 and type_2, but not type_3. I want to write a simple query to see if this is true or false.
At first I tried something like:
SELECT * FROM logs AS l
INNER JOIN types AS t
ON t.id = l.type_id
WHERE t.name = "type_1"
AND t.name = "type_2"
AND t.name != "type_3";
But I quickly realised that it was not possible to do it like this, since t.name cannot have multiple values. I have tried a bunch of different approaches now, but cannot seem to find the one right for me. I'm sure the solution is fairly simple, I just don't see it at the moment.
Hope someone can point me in the right direction.
I have made a simple test database in this fiddle, to use for testing and example: https://www.db-fiddle.com/f/nA6iKgCcJwKnXKsxaNvsLt/0
One option with conditional aggregation.
SELECT l.userID
FROM logs AS l
JOIN types AS t ON t.id = l.type_id
GROUP BY l.userID
HAVING COUNT(DISTINCT CASE WHEN t.name IN ('type_1','type_2') THEN t.name END) = 2
AND COUNT(DISTINCT CASE WHEN t.name = 'type_3' THEN t.name END) = 0
You can do it like Vamsi, but if you prefer an easier to understand SQL then you can do it like this:
SELECT * FROM logs AS l
INNER JOIN types AS t
ON t.id = l.type_id
WHERE true
AND EXISTS (SELECT 1 FROM logs ll WHERE l.user_id = ll.user_id AND type_id = 1)
AND EXISTS (SELECT 1 FROM logs ll WHERE l.user_id = ll.user_id AND type_id = 2)
AND NOT EXISTS (SELECT 1 FROM logs ll WHERE l.user_id = ll.user_id AND type_id = 3)
I do not recommend using count(distinct) for this purpose. It can be expensive. I would simply do:
SELECT l.userId
FROM logs l INNER JOIN
types t
ON t.id = l.type_id
WHERE t.name IN ('type_1', 'type_2', 'type_3')
GROUP BY l.userId
HAVING SUM(t.name = 'type_1') > 0 AND -- at least one
SUM(t.name = 'type_2') > 0 AND -- at least one
SUM(t.name = 'type_3') = 0 ; -- none

MYSQL LEFT JOINing the latest record from another table

I have a training_stats table (current due training) and I also have a completed_training table.
What I want to do is query due training with the last completed date from the completed table.
I've nearly got what I want, I get the due training, but they are duplicated with each completed record(as there are many completed records to each current due), and I only want single rows and the latest completed date.
I've been trying to use MAX, and when I run the MAX query independently, I get the last record. But when the MAX query is in the join, it is returning all completed rows.
This is the query that I am using:
SELECT s.course_stat_id
,o.org_name
,u.id
,u.first_name
,u.last_name
,a.area_id
,a.area_name
,tc.course_id
,tc.course_name
,s.assigned_on
,s.due
,s.pass_mark
,s.completed_on
,completed.complete_training_id
,completed.complete_date
FROM training_stats s
JOIN organisations o ON o.org_id = s.org_id
LEFT JOIN (
SELECT complete_training_id
,user_id
,area_id
,course_id
,max(completed_on) AS complete_date
FROM completed_training
GROUP BY complete_training_id
) completed ON completed.user_id = s.user_id
AND completed.area_id = s.area_id
AND completed.course_id = s.course_id
LEFT JOIN users u ON u.id = s.user_id
LEFT JOIN areas a ON a.area_id = s.area_id
LEFT JOIN training_courses tc ON tc.course_id = s.course_id
WHERE u.active = 1
AND o.active = 1
AND s.assigned = 1
Can you see what I am doing wrong?
Not exactly positive of your expected results, but the failure is PROBABLY for your group by and JOIN. Your group by is ONLY on the training ID, but you are also pulling user, area and course as well as max date completed for said respective training ID, user, area, course. You group by and join should match the unique characteristics.
Without seeing data, the query as I interpret it is that the "complete_training_id" is an auto-increment column for that table. Having said that, there would only ever be one record for that ID.
Having said that, the completed training table can have for a single user, area and course, multiple training days of which you want the most recent. For example someone attending college and needs to take many computer classes and they are refreshers from prior so assume all are same course ID. A person could take in 2012, 2014, 2016. You would want the instance of the user/area/course showing the 2016 dated training. So lets look at that first.
select
ct.user_id,
ct.area_id,
ct.course_id,
max(ct.completed_on) AS complete_date
FROM
completed_training ct
GROUP BY
ct.user_id,
ct.area_id,
ct.course_id
Now, for each user, area and course of study, I have one record with the most recent completion date. NOW lets pull the rest of the details, but since you need the completed training ID too, I applied the MAX() of that in the query below. The ID should by default be increasing every time a new record is added, so one completed a year ago would have a lower value than the ID completed today. So you get both the completed ID and its corresponding date for a given user, area, course.
SELECT
s.course_stat_id,
o.org_name,
u.id,
u.first_name,
u.last_name,
a.area_id,
a.area_name,
tc.course_id,
tc.course_name,
s.assigned_on,
s.due,
s.pass_mark,
s.completed_on,
ct.complete_training_id,
ct.complete_date
FROM
training_stats s
JOIN organisations o
ON s.org_id = o.org_id
AND o.active = 1
LEFT JOIN
( select
ct.user_id,
ct.area_id,
ct.course_id,
max(ct.complete_training_id ) as complete_training_id,
max(ct.completed_on) AS complete_date
FROM
completed_training ct
GROUP BY
ct.user_id,
ct.area_id,
ct.course_id ) ct
on s.user_id = ct.user_id
AND s.area_id = ct.area_id
AND s.course_id = ct.course_id
JOIN users u
ON s.user_id = u.id
AND u.active = 1
LEFT JOIN areas a
ON s.area_id = a.area_id
LEFT JOIN training_courses tc
ON s.course_id = tc.course_id
WHERE
s.assigned = 1
I'm not 100% sure of that. First, run this query. It should list all completed training, with a rnk from 1 (lastest), to n (oldest).
SELECT complete_training_id
,user_id
,area_id
,course_id
,completed_on AS complete_date
,#curRank := case when complete_training_id <> #cur_complete_training_id then 0 else #curRank + 1 end rnk
FROM completed_training, (select #curRank := 0, #cur_complete_training_id := 0)
ORDER BY complete_training_id, completed_on DESC
If true, the answer is :
SELECT s.course_stat_id
,o.org_name
,u.id
,u.first_name
,u.last_name
,a.area_id
,a.area_name
,tc.course_id
,tc.course_name
,s.assigned_on
,s.due
,s.pass_mark
,s.completed_on
,completed.complete_training_id
,completed.complete_date
FROM training_stats s
JOIN organisations o ON o.org_id = s.org_id
LEFT JOIN (
SELECT complete_training_id
,user_id
,area_id
,course_id
,completed_on AS complete_date
,#curRank := case when complete_training_id <> #cur_complete_training_id then 0 else #curRank + 1 end rnk
FROM completed_training, (select #curRank := 0, #cur_complete_training_id := 0)
ORDER BY complete_training_id, completed_on DESC
) completed ON completed.user_id = s.user_id and completed.rnk = 1
AND completed.area_id = s.area_id
AND completed.course_id = s.course_id
LEFT JOIN users u ON u.id = s.user_id
LEFT JOIN areas a ON a.area_id = s.area_id
LEFT JOIN training_courses tc ON tc.course_id = s.course_id
WHERE u.active = 1
AND o.active = 1
AND s.assigned = 1

Get other different id

I have a simple mysql query:
SELECT U.id FROM
users U
INNER JOIN friends F
ON ( U.id = F.id_exp AND F.id_des = :id_exp )
OR ( U.id = F.id_des AND F.id_exp = :id_exp ) WHERE
U.id <> :id_des
AND F.active = 1
I wish that by calling this request again, that It does not return the id that It has already previously returned.
I hope you understand me.
Thank you.
You must give RAND a seed number in order to always get the same order. This can be any number, e.g. 1:
...
ORDER BY RAND(1)
LIMIT #offset, 1;

MYSQL return 0 for filtered rows

I am trying to filter with a "WHERE" but I want to display the filtered rows as "NULL" or 0 instead of hiding them.
Here is my code:
SELECT *, IFNULL(SUM(ROUND(TIMESTAMPDIFF(MINUTE,start,end)/60,1)),0) urlaub
FROM time_entries e
LEFT JOIN users u ON e.user_id = u.id
WHERE e.project_id = 10 AND YEAR(end) = YEAR(CURRENT_DATE)
GROUP BY e.user_id
Best,
Chris
I might be just being pedantic, but in case there is a genuine misunderstanding.... They are not "hidden", they are not there. The point of the WHERE is to choose which records to retrieve, so as to not retrieve the entire table contents when only a few records of data are relevant.
But if something like that is what you want to do, you should just be able to add another result field, like so:
(e.project_id = 10 AND YEAR(end) = YEAR(CURRENT_DATE)) AS relevantRecord
Got it working! This is the code that returns what I wanted to:
SELECT
*,
IFNULL(SUM(CASE WHEN e.project_id = 10 AND YEAR(end) = YEAR(CURRENT_DATE) THEN ROUND(TIMESTAMPDIFF(MINUTE, start, end)/60,1) ELSE 0 END),0) AS urlaub
FROM time_entries e
LEFT JOIN users u ON e.user_id = u.id
GROUP BY e.user_id
Thanks!

How to scan through multiple items in MySQL?

I want to select all elements from the element table, but with an extra column stating whether or not ANY of their comments have admin status. A comment has admin status if the user that posted the status' admin column is 1.
I don't know how I'd scan through each comment for every element being queried.
In another part of the program, there is a query to draw down all comments of a single issue, and I was able to reason how to determine admin status there, but I can't think of a way to do it in a query that pulls down more than one issue.
SELECT comments.id, comments.elementID, comments.googleID, comments.time, comments.body, users.name, users.admin
FROM comments
LEFT JOIN users ON comments.googleID = users.googleID
WHERE comments.elementID = ? AND comments.approved = 1
ORDER BY comments.time DESC
To know which elements have an comment from an admin user, you could simply do:
SELECT C.elementID, MAX(U.admin) AS admin
FROM comments C
LEFT JOIN users U ON C.googleId = U.googleID
WHERE C.approved = 1
GROUP BY C.elementID;
and then to get all elements with the additional admin column:
SELECT E.*, CASE WHEN A.admin = 1 THEN 1 ELSE 0 END AS admin
FROM elements E
LEFT OUTER JOIN
(SELECT C.elementID, MAX(U.admin) AS admin
FROM comments C
LEFT JOIN users U ON C.googleId = U.googleID
WHERE C.approved = 1
GROUP BY C.elementID) A
ON E.id = A.elementID
You can use a case expression with an exists predicate and a correlated subquery like so:
SELECT
e.*,
CASE WHEN EXISTS (
SELECT 1
FROM comments c
JOIN users u ON c.googleID = u.googleID
WHERE e.googleID = c.googleID AND u.admin = 1
) THEN 'Yes' ELSE 'No' END AS AdminComment
FROM elements e;
Or you could express it using joins:
SELECT
e.*,
CASE WHEN u.admin IS NOT NULL THEN 'Yes' ELSE 'No' END AS AdminComment
FROM elements e
LEFT JOIN comments c ON e.googleID = c.googleID
LEFT JOIN users u ON c.googleID = u.googleID AND u.admin = 1