Below is a query which takes 30+ seconds to run. Based on similar queries I have running, I can't see where the hold up is here. My only thought is joining the job user id to job_applicants user id, but they need to be mapped.
SELECT DISTINCT u.user_id, u.first_name, u.last_name FROM users u
LEFT OUTER JOIN employee_access ea ON ea.user_id = u.user_id
LEFT OUTER JOIN confirmation c ON c.user_id = u.user_id
LEFT OUTER JOIN job_applicants a ON a.user_id = u.user_id
LEFT OUTER JOIN job j ON j.job_id = a.job_id
WHERE ea.access_id = 4 OR c.access_id = 4 OR (a.process_level = 0 AND j.access_id = 4)
ORDER BY u.last_name asc
Use exists:
select u.*
from users u
where exists (select 1
from employee_access ea
where ea.user_id = u.user_id and ea.access_id = 4
) or
exists (select 1
from confirmation c
where c.user_id = u.user_id and c.access_id = 4
) or
exists (select 1
from job_applicants a join
job j
on j.job_id = a.job_id
where a.user_id = u.user_id and
a.process_level = 0 AND j.access_id = 4
)
order by u.last_name;
This will prevent all the Cartesian products and the final removal of duplicates.
I would recommend indexes on:
users(last_name, user_id)
employee_access(user_id, access_id)
confirmation(user_id, access_id)
job_applicants(user_id, process_level, job_id)
job(job_id, access_id)
Yet another approach. This has the advantage of first gathering the list of user_ids, then reaching into users for the other columns:
SELECT u.user_id, u.first_name, u.last_name
FROM users u
JOIN (
( SELECT user_id FROM employee_access WHERE access_id = 4 )
UNION DISTINCT
( SELECT user_id FROM confirmation WHERE access_id = 4 )
UNION DISTINCT
( SELECT a.user_id
FROM job_applicants a
JOIN job j USING(job_id)
WHERE a.process_level = 0
AND j.access_id = 4 )
) AS x USING(user_id)
ORDER BY u.last_name ASC
Indexes:
employee_access: INDEX(access_id, user_id) -- (covering)
confirmation: INDEX(access_id, user_id) -- (covering)
job: INDEX(access_id, job_id) -- (covering)
job_applicants: INDEX(process_level, job_id, user_id) -- (covering)
users: PRIMARY KEY(user_id)
See if this will shave off most of the remaining 8 seconds.
This should work. It is similar in concept to Gordon's answer but I have a borderline pathological distrust of correlated subqueries.
SELECT DISTINCT u.user_id, u.first_name, u.last_name
FROM users u
WHERE u.user_id IN (SELECT user_id FROM employee_access WHERE access_id = 4)
OR u.user_id IN (SELECT user_id FROM confirmation WHERE access_id = 4)
OR u.user_id IN (
SELECT a.user_id
FROM job_applicants a
INNER JOIN job j ON j.job_id = a.job_id
WHERE a.process_level = 0 AND j.access_id = 4
)
ORDER BY u.last_name asc
Related
This query is taking forever to finish in MySql 8, doing some research i found out that the "EXISTS" in this code can be extremely slow in some queries.
When i remove the "OR EXISTS" sub-query part, it runs in less than a second.
So i need to substitute the "OR EXISTS" in this query so i can get all the users i need:
SELECT u.name,
u.email,
u.cpf,
u.register,
r.name AS role_name,
s.name AS sector_name,
b.name AS branch_name,
u.status
FROM users u
INNER JOIN roles r ON r.id = u.role_id
INNER JOIN sectors s ON s.id = u.sector_id
INNER JOIN branches b ON b.id = u.branch_id
WHERE u.status = 2 OR EXISTS (
SELECT *
FROM user_recovery ur
WHERE ur.user_id = u.id
AND ur.status_recovery = 1
)
Is there a way to do it without the "OR EXISTS"?
Or can enforce a full scan
try
you can't get rid of the eXISTS clause because it increases the number of returned rows.
Add a INDEX on user status and user_recovery userid,status_recovery and on the on Clause columns.
SELECT u.name,
u.email,
u.cpf,
u.register,
r.name AS role_name,
s.name AS sector_name,
b.name AS branch_name,
u.status
FROM users u
INNER JOIN roles r ON r.id = u.role_id
INNER JOIN sectors s ON s.id = u.sector_id
INNER JOIN branches b ON b.id = u.branch_id
WHERE u.status = 2
UNION
SELECT u.name,
u.email,
u.cpf,
u.register,
r.name AS role_name,
s.name AS sector_name,
b.name AS branch_name,
u.status
FROM users u
INNER JOIN roles r ON r.id = u.role_id
INNER JOIN sectors s ON s.id = u.sector_id
INNER JOIN branches b ON b.id = u.branch_id
WHERE EXISTS (
SELECT 1
FROM user_recovery ur
WHERE ur.user_id = u.id
AND ur.status_recovery = 1
)
"I'll see your UNION; and raise you a derived table."
SELECT u.name,
u.email,
u.cpf,
u.register,
r.name AS role_name,
s.name AS sector_name,
b.name AS branch_name,
u.status
FROM ( SELECT id
FROM users
WHERE status = 2
UNION DISTINCT -- or UNION ALL; see below
SELECT user_id
FROM user_recovery
WHERE status_recovery = 1 -- see new index
) AS u1
JOIN users AS u USING(id) -- self-join to pick up other columns
JOIN roles r ON r.id = u.role_id
JOIN sectors s ON s.id = u.sector_id
JOIN branches b ON b.id = u.branch_id;
Indexes:
user_recovery: INDEX(status_recovery, user_id) -- in this order
users: INDEX(status, id) -- in this order
(I assume `id` is the PRIMARY KEY in each table)
The general rule here is... When you have a bunch of JOINs, but a single table that controls which rows, but that is messy or slow (eg UNION in this case, GROUP BY or LIMIT in other cases),
Optimize finding the ids (user.id aka user_id) is the optimal way.
Then JOIN back to the original table (if needed), plus the other tables.
In doing all that, it became apparent that a new index for user_recovery might be beneficial.
(If UNION ALL won't produce any dups, switch to it for a little more speed.)
I want to optimize this query becouse it takes to much time to return records
SELECT
u.*,
s.legal_name AS structure_name,
ui.id AS userinfo_id,
ui.structure_id AS structure_id,
ui.lrn_user,
ui.gender,
ui.fiscal_code,
ui.prov,
ui.phone,
ui.school_name,
ui.school_codice_meccanografico,
us.status, us.date AS status_date,
CONCAT(u.lastname,' ',u.firstname) AS fullname,
CONCAT(u.firstname,' ',u.lastname) AS display_name,
uu.username AS created_by_name,
g.group_names,
IF(u.website_id = 0,'Sito Web principale', w.name) AS website_name
FROM fcf_users AS u
LEFT JOIN (
SELECT
gu.user_id,
GROUP_CONCAT(gg.name SEPARATOR ', ') AS group_names
FROM fcf_user_user_groups gu
JOIN fcf_user_groups gg ON gg.id = gu.group_id
GROUP BY user_id
) g ON g.user_id = u.id
LEFT JOIN fcf_users_userinfo AS ui ON ui.user_id = u.id
LEFT JOIN fcf_users_user_statuses AS us ON us.user_id = u.id
LEFT JOIN fcf_structures_structures AS s ON s.id = ui.structure_id
LEFT JOIN fcf_users AS uu ON uu.id = u.created_by
LEFT JOIN fcf_websites AS w ON w.id = u.website_id
WHERE
u.id IN (SELECT user_id FROM fcf_user_user_groups WHERE group_id = '8')
AND u.id IN (SELECT user_id FROM fcf_user_user_groups WHERE group_id = '8')
AND ui.lrn_user = '0'
ORDER BY fullname ASC
LIMIT 0,25
If anyone can help, thanks
Turn it inside-out. That is, first use a 'derived' table to locate 25 users you want. Then gather the rest of the info.
What you have gathers all the info (including all the JOIN work) for all the users, then sorts and peels off 25.
It will be something like:
SELECT -- lots of stuff
FROM ( SELECT u.id,
CONCAT(u.lastname,' ',u.firstname) AS fullname
FROM fcf_users AS u
JOIN fcf_user_user_groups AS ug ON ...
JOIN fcf_users_userinfo AS ui ON ui.user_id = u.id
WHERE ug.group_id = '8'
AND ui.lrn_user = '0'
ORDER BY u.lastname, u.firstname -- now sargeable
LIMIT 25
) AS u25
JOIN .... -- whatever tables are needed to get the rest of the columns
ORDER BY u25.fullname -- yes, again, but now using the CONCAT
-- no limit here
Also:
u: INDEX(lastname, firstname, id)
user_user_group is a "many-t0=many mapping" table? If so, follow the indexing advice here: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
Ditto for any other many:many tables.
Note how I put into the derived table only the tables needed to achieve the LIMIT.
I have the following 3 tables:
users: [id, name, admin ...]
events: [id, user_id, type ...]
messages: [id, user_id, ...]
I want to construct a query that does the following:
-> Select all users from the table users who have not scheduled an event of the type "collection"
-> And who have less than 3 messages of the type "collection_reminder"
-> And who are not admin
I've managed to figure out the first part of this query, but it all goes a bit pear shaped when I try to add the 3 table, do the count, etc.
Here is a query that might get the job done. Each of the requirement is represented as a condition in the WHERE clause, using correlated subqueries when needed:
SELECT u.*
FROM users u
WHERE
NOT EXISTS (
SELECT 1
FROM events e
WHERE e.user_id = u.id AND e.type = 'collection'
)
AND (
SELECT COUNT(*)
FROM messages m
WHERE m.userid = u.id AND m.type = 'collection_reminder'
) <= 3
AND u.admin IS NULL
Ill try this on the top of the head so expect some synthax issues, but the idea is the following.
You can filter out who have no events schedule using a left join. On a left join the elements on the second part of the query will show up as null.
select * from users u
left join events e on e.user_id = u.id
where e.user_id is null
Now, i dont think this is the most performant way, but a simple way to search for everyone that has 3 or less messages:
select * from users u
left join events e on e.user_id = u.id
where u.id in (
select COUNT(*) from messages m where m.user_id = u.id HAVING COUNT(*)>3;
)
and e.user_id is null
Then filtering who is not admin is the easiest :D
select * from users u
left join events e on e.user_id = u.id
where u.id in (
select COUNT(*) from messages m where m.user_id = u.id HAVING COUNT(*)>3;
)
and e.user_id is null
and u.admin = false
Hope it helps.
This is pretty much a direct translation of your requirements, in the order you listed them:
SELECT u.*
FROM users AS u
WHERE u.user_id NOT IN (SELECT user_id FROM events WHERE event_type = 'Collection')
AND u.user_id IN (
SELECT user_id
FROM messages
WHERE msg_type = 'Collection Reminder'
GROUP BY user_id
HAVING COUNT(*) < 3
)
AND u.admin = 0
or alternatively, this can be accomplished completely with joins:
SELECT u.*
FROM users AS u
LEFT JOIN events AS e ON u.user_id = e.user_id AND e.event_type = 'Collection'
LEFT JOIN messages AS m ON u.user_id = m.user_id AND m.msg_type = 'Collection Reminder'
WHERE u.admin = 0
AND e.event_id IS NULL -- No event of type collection
GROUP BY u.user_id -- Note: you should group on all selected fields, and
-- some configuration of MySQL will require you do so.
HAVING COUNT(DISTINCT m.message_id) < 3 -- Less than 3 collection reminder messages
-- distinct is optional, but
-- if you were to remove the "no event" condition,
-- multiple events could multiply the message count.
;
This query uses joins to link the 3 tables, filters the result using the where clause, and using Group by, having limiting the result to only those who satisfy the less than count condition..
SELECT a.id,
SUM(CASE WHEN b.type = 'collection' THEN 1 ELSE 0 END),
SUM(CASE WHEN c.type = 'collection_reminder' THEN 1 ELSE 0 END
FROM users a
left join events b on (b.user_id = a.id)
left join messages c on (c.user_id = a.id)
WHERE a.admin = false
GROUP BY a.id
HAVING SUM(CASE WHEN b.type = 'collection' THEN 1 ELSE 0 END) = 0
AND SUM(CASE WHEN c.type = 'collection_reminder' THEN 1 ELSE 0 END) < 3
I have the following query in MySQL:
(SELECT ue.id, ue.userid, ue.status, ue.timestart, ue.timeend, e.courseid,
e.id AS enrolid, ra.roleid
FROM user_enrolments ue
JOIN enrol e ON e.id = ue.enrolid
JOIN course c ON c.id = e.courseid
JOIN user u ON u.id = ue.userid
JOIN context ct ON ct.instanceid = c.id
LEFT JOIN role_assignments ra ON ra.userid = u.id AND
ra.contextid = ct.id AND
ra.itemid = e.id
WHERE e.customint1 = 1 AND u.deleted = 0 AND
ct.contextlevel = 50 AND (ue.status = 0 OR ue.status = 1))
UNION
(SELECT de.enrolid AS id, de.userid, de.status, de.date_ini, de.date_fin,
de.courseid, de.enrolid, de.roleid
FROM deleted_enrols de
JOIN user u ON u.id = de.userid
WHERE userid = ANY (SELECT userid FROM local_users WHERE clientid = 1))
ORDER BY u.firstname, u.lastname, c.fullname LIMIT 0, 100
If I delete ORBER BY and LIMIT, this query works fine... but the ORDER BY clause gives an error:
Table 'u' from one of the SELECTs cannot be used in global ORDER clause
If I delete the parentheses of both SELECT querys, the error is different:
Table 'u' from one of the SELECTs cannot be used in field list
I have also tried with UNION ALL, but it does not work either.
Any suggestion or clue? Thanks in advance for your time...
The results of your UNION do not include any fields from table 'u', so those results cannot be sorted by table 'u' fields.
You could perhaps perform the UNION and then re-join the results to table 'u', and then use that to sort the results by table 'u' fields. A similar issue exists for sorting on
course.fullname, so that would need to be joined back in, too.
SELECT x.id, x.userid, x.status, x.timestart, x.timeend, x.courseid, x.enrolid, x.roleid
FROM ((SELECT ue.id, ue.userid, ue.status, ue.timestart, ue.timeend, e.courseid,
e.id AS enrolid, ra.roleid
FROM user_enrolments ue
JOIN enrol e ON e.id = ue.enrolid
JOIN course c ON c.id = e.courseid
JOIN user u ON u.id = ue.userid
JOIN context ct ON ct.instanceid = c.id
LEFT JOIN role_assignments ra ON ra.userid = u.id
AND ra.contextid = ct.id
AND ra.itemid = e.id
WHERE e.customint1 = 1 AND u.deleted = 0
AND ct.contextlevel = 50 AND (ue.status = 0 OR ue.status = 1))
UNION
(SELECT de.enrolid AS id, de.userid, de.status, de.date_ini, de.date_fin,
de.courseid, de.enrolid, de.roleid
FROM deleted_enrols de
JOIN user u ON u.id = de.userid
WHERE userid = ANY (SELECT userid FROM local_users WHERE clientid = 1))
) x
JOIN user z ON z.id = x.userid
JOIN course d ON d.id = x.courseid
ORDER BY z.firstname, z.lastname, d.fullname LIMIT 0, 100
Assuming you want to sort the whole lot, try parentheses round the whole query with the ORDER BY done afterwards:
select id, userid, status, timestart, timeend, courseid, enrolid, roleid from
((SELECT ue.id, ue.userid, ue.status, ue.timestart, ue.timeend, e.courseid,
e.id AS enrolid, ra.roleid, u.firstname, u.lastname, c.fullname
FROM user_enrolments ue
JOIN enrol e ON e.id = ue.enrolid
JOIN course c ON c.id = e.courseid
JOIN user u ON u.id = ue.userid
JOIN context ct ON ct.instanceid = c.id
LEFT JOIN role_assignments ra ON ra.userid = u.id AND
ra.contextid = ct.id AND
ra.itemid = e.id
WHERE e.customint1 = 1 AND u.deleted = 0 AND
ct.contextlevel = 50 AND (ue.status = 0 OR ue.status = 1))
UNION
(SELECT de.enrolid AS id, de.userid, de.status, de.date_ini, de.date_fin,
de.courseid, de.enrolid, de.roleid, u.firstname, u.lastname, ' ' as fullname
FROM deleted_enrols de
JOIN user u ON u.id = de.userid
WHERE userid = ANY (SELECT userid FROM local_users WHERE clientid = 1))) s1
ORDER BY firstname, lastname, fullname LIMIT 0, 100
(obviously fullname in the second SELECT statement would be populated however seems sensible)
You need to include the data to be ordered by in the selects of the unioned queries; an ORDER BY following a UNION is handled as if it were SELECT * FROM (unions) ORDER BY ... so anything not coming out of the union cannot be used for ordering.
Ironically, a query similar to that is the key to getting what you want though, with something like
SELECT x, y, z
FROM (
SELECT x, y, z, somethingIdontactuallywant
FROM blah
UNION
SELECT a, b, c, somethingIdontactuallywant
FROM blah2
) AS u
ORDER BY u.somethingIdontactuallywant
As mysql documentation on union says:
This kind of ORDER BY cannot use column references that include a
table name (that is, names in tbl_name.col_name format). Instead,
provide a column alias in the first SELECT statement and refer to the
alias in the ORDER BY. (Alternatively, refer to the column in the
ORDER BY using its column position. However, use of column positions
is deprecated.)
Also, if a column to be sorted is aliased, the ORDER BY clause must
refer to the alias, not the column name.
So, do not refer to any table names and use columns that are actually in the resultset of the union.
I have 3 tables
users ( user_id , name )
thread ( thread_id , date_time)
subscribers ( subscriber_id , thread_id)
subscriber id is a foreign key from users table how can i get thread of 2 specefic users , i'm doing something like this but it's giving me blank rows
SELECT t.thread_id, s.subscriber_id, u.user_id
FROM subscribers s
LEFT JOIN thread t ON t.thread_id = s.thread_id
LEFT JOIN user u ON s.subscriber_id = u.user_id
WHERE s.subscriber_id = 1 AND s.subscriber_id = 2
You should check your query as #Tim Biegeleisen said.
I will highlight the portion of your query which is causing the empty set result:
WHERE s.subscriber_id = 1 AND s.subscriber_id =2
^ really?
You are asking MySQL to return records where the subscriber_id is both 1 and 2. This doesn't make any sense. In the corrected query below I have changed the WHERE condition to use OR instead of AND:
SELECT u.user_id, u.name, t.thread_id
FROM users u INNER JOIN susbscribers s ON u.user_id = s.subscriber_id
INNER JOIN thread t ON s.thread_id = t.thread_id
WHERE u.user_id = 1 OR u.user_id = 2