How to scan through multiple items in MySQL? - mysql

I want to select all elements from the element table, but with an extra column stating whether or not ANY of their comments have admin status. A comment has admin status if the user that posted the status' admin column is 1.
I don't know how I'd scan through each comment for every element being queried.
In another part of the program, there is a query to draw down all comments of a single issue, and I was able to reason how to determine admin status there, but I can't think of a way to do it in a query that pulls down more than one issue.
SELECT comments.id, comments.elementID, comments.googleID, comments.time, comments.body, users.name, users.admin
FROM comments
LEFT JOIN users ON comments.googleID = users.googleID
WHERE comments.elementID = ? AND comments.approved = 1
ORDER BY comments.time DESC

To know which elements have an comment from an admin user, you could simply do:
SELECT C.elementID, MAX(U.admin) AS admin
FROM comments C
LEFT JOIN users U ON C.googleId = U.googleID
WHERE C.approved = 1
GROUP BY C.elementID;
and then to get all elements with the additional admin column:
SELECT E.*, CASE WHEN A.admin = 1 THEN 1 ELSE 0 END AS admin
FROM elements E
LEFT OUTER JOIN
(SELECT C.elementID, MAX(U.admin) AS admin
FROM comments C
LEFT JOIN users U ON C.googleId = U.googleID
WHERE C.approved = 1
GROUP BY C.elementID) A
ON E.id = A.elementID

You can use a case expression with an exists predicate and a correlated subquery like so:
SELECT
e.*,
CASE WHEN EXISTS (
SELECT 1
FROM comments c
JOIN users u ON c.googleID = u.googleID
WHERE e.googleID = c.googleID AND u.admin = 1
) THEN 'Yes' ELSE 'No' END AS AdminComment
FROM elements e;
Or you could express it using joins:
SELECT
e.*,
CASE WHEN u.admin IS NOT NULL THEN 'Yes' ELSE 'No' END AS AdminComment
FROM elements e
LEFT JOIN comments c ON e.googleID = c.googleID
LEFT JOIN users u ON c.googleID = u.googleID AND u.admin = 1

Related

mysql query takes 3 hours to run and process

I have a query that is ran on a cron job late at night. This query is then processed through a generator as it has to populate another database and I make some additional processes and checks before it is sent to the other DB.
I am wondering is there anyway for me to speed up this query and hopefully keep it as a single query. Or will I be forced to create other queries and join the data within PHP? This queries the main mautic database.
SELECT c.id as "campaign_id",
c.created_by_user,
c.name,
c.date_added,
c.date_modified,
(SELECT DISTINCT COUNT(cl.lead_id)) as number_of_leads,
GROUP_CONCAT(lt.tag) as tags,
cat.title as category_name,
GROUP_CONCAT(ll.name) as segment_name,
GROUP_CONCAT(emails.name) as email_name,
CASE WHEN c.is_published = 1 THEN "Yes" ELSE "No" END AS "published",
CASE WHEN c.publish_down > now() THEN "Yes"
WHEN c.publish_down > now() AND c.is_published = 0 THEN "Yes"
ELSE "No" END AS "expired"
FROM campaigns c
LEFT JOIN campaign_leads cl ON cl.campaign_id = c.id
LEFT JOIN lead_tags_xref ltx on cl.lead_id = ltx.lead_id
LEFT JOIN lead_tags lt on ltx.tag_id = lt.id
LEFT JOIN categories cat on c.category_id = cat.id
LEFT JOIN lead_lists_leads llist on cl.lead_id = llist.lead_id
LEFT JOIN lead_lists ll on llist.leadlist_id = ll.id
LEFT JOIN email_list_xref el on ll.id = el.leadlist_id
LEFT JOIN emails on el.email_id = emails.id
GROUP BY c.id;
Here is a image of the explain
https://prnt.sc/qQtUaLK3FIpQ
Definitions
Campaign Table:
https://prnt.sc/6JXRGyMsWpcd
Campaign_leads table
https://prnt.sc/pOq0_SxW2spe
lead_tags_xref table
https://prnt.sc/oKYn92O82gHL
lead_tags table
https://prnt.sc/ImH81ECF6Ly1
categories table
https://prnt.sc/azQj_Xwq3dw9
lead_lists_lead table
https://prnt.sc/x5C5fiBFP2N7
lead_lists table
https://prnt.sc/bltkM0f3XeaH
email_list_xref table
https://prnt.sc/kXABVJSYWEUI
emails table
https://prnt.sc/7fZcBir1a6QT
I am only expected 871 rows to be completed, I have identified that the joins can be very large, in the tens of thousands.
Seems you have an useless select DISTINCT .. could you are looking for a count(distinct .. )
In this way you can avoid nested select for each rows in main select ..
SELECT c.id as "campaign_id",
c.created_by_user,
c.name,
c.date_added,
c.date_modified,
COUNT(DISTINCT cl.lead_id) as number_of_leads,
GROUP_CONCAT(lt.tag) as tags,
cat.title as category_name,
GROUP_CONCAT(ll.name) as segment_name,
GROUP_CONCAT(emails.name) as email_name,
CASE WHEN c.is_published = 1 THEN "Yes" ELSE "No" END AS "published",
CASE WHEN c.publish_down > now() THEN "Yes"
WHEN c.publish_down > now() AND c.is_published = 0 THEN "Yes"
ELSE "No" END AS "expired"
FROM campaigns c
LEFT JOIN campaign_leads cl ON cl.campaign_id = c.id
LEFT JOIN lead_tags_xref ltx on cl.lead_id = ltx.lead_id
LEFT JOIN lead_tags lt on ltx.tag_id = lt.id
LEFT JOIN categories cat on c.category_id = cat.id
LEFT JOIN lead_lists_leads llist on cl.lead_id = llist.lead_id
LEFT JOIN lead_lists ll on llist.leadlist_id = ll.id
LEFT JOIN email_list_xref el on ll.id = el.leadlist_id
LEFT JOIN emails on el.email_id = emails.id
GROUP BY c.id;
anyway be sure you have a proper composite index on
table campaign_leads columns campaign_id, lead_id
table lead_tags_xref columns lead_id, tag_id
table lead_lists_leads columns lead_id, leadlist_id
table email_list_xref columns leadlist_id, email_id

MYSQL return 0 for filtered rows

I am trying to filter with a "WHERE" but I want to display the filtered rows as "NULL" or 0 instead of hiding them.
Here is my code:
SELECT *, IFNULL(SUM(ROUND(TIMESTAMPDIFF(MINUTE,start,end)/60,1)),0) urlaub
FROM time_entries e
LEFT JOIN users u ON e.user_id = u.id
WHERE e.project_id = 10 AND YEAR(end) = YEAR(CURRENT_DATE)
GROUP BY e.user_id
Best,
Chris
I might be just being pedantic, but in case there is a genuine misunderstanding.... They are not "hidden", they are not there. The point of the WHERE is to choose which records to retrieve, so as to not retrieve the entire table contents when only a few records of data are relevant.
But if something like that is what you want to do, you should just be able to add another result field, like so:
(e.project_id = 10 AND YEAR(end) = YEAR(CURRENT_DATE)) AS relevantRecord
Got it working! This is the code that returns what I wanted to:
SELECT
*,
IFNULL(SUM(CASE WHEN e.project_id = 10 AND YEAR(end) = YEAR(CURRENT_DATE) THEN ROUND(TIMESTAMPDIFF(MINUTE, start, end)/60,1) ELSE 0 END),0) AS urlaub
FROM time_entries e
LEFT JOIN users u ON e.user_id = u.id
GROUP BY e.user_id
Thanks!

MySQL Subquery Select very slow

One of our websites has a table with about 60'000 records in it. Recently we noticed the page was timing out and could only be resolved by setting the memory limit to -1. This allowed the page to load but it was very slow. Furthermore I did not believe this was the correct way to resolve the problem, as obviously it indicates something is not quite right.
I managed to output the query that the page was running:
SELECT u.*,
(SELECT COUNT(*) FROM enquiry e WHERE e.user_id = u.id AND e.deleted = 0 AND e.time_started != 0) AS opened_count,
(SELECT COUNT(*) FROM enquiry e WHERE e.user_id = u.id AND e.deleted = 0 AND e.confirmed = 1 AND e.time_started != 0) AS confirmed_count,
(SELECT COUNT(*) FROM enquiry e WHERE e.user_id = u.id AND e.deleted = 0 AND e.call_back = 1 AND e.time_started != 0) AS call_back_count
FROM user u
WHERE u.active = 1 AND u.deleted = 0
ORDER BY u.username
I ran this query in phpMyAdmin and it takes over 30 seconds to return the results.
I feel the query needs optimising in some way but I'm struggling to work out how. I'm guessing I need to use a JOIN of some sort?
You're really running >180,000 queries, since each of those 3 subqueries will be run once for every row in the user table.
You could try simplifying into a standard join with some groups, e.g.
SELECT user.*,
COUNT(enq.id) AS opened_count
SUM(e.confirmed = 1) AS confirmed_count
SUM(e.call_back = 1) AS call_back_count
FROM user
LEFT JOIN enquiry ON enquiry.user_id = user.id
WHERE user.active = 1 and user.deleted AND enquiry.deleted = 0
GROUP BY user.id

MySQL Joins where one value can be optional

Situation
I have a database which heavily makes use of joins due to the various situations in which each entity is used. Here is a simplified diagram:
Goal
I would like to be able to get details of all modules and the "name" fields regardless of whether the "fk_chapter_id" within user_has_module is set or not.
In the case where "user_has_module.fk_chapter_id" is null, the system can return details of the module and then null chapter.
In the case where there is a user_has_module, I would like to get the status
Issue
Whenever I perform SQL statements, I get the results only partially returned. I.E. If I have 4 module records in total, two of which where the user has an entry in "user_has_module" returns the two records in full and then 2 null records for the other modules.
Update based on feedback, almost there
Now, the only problem is I get duplicates. Using some test data
SELECT DISTINCT
chapter_id,
chapter_name,
module_id,
module_name,
(null ) AS user_module_progress,
(SELECT COUNT(fk_chapter_id) FROM module_has_chapter WHERE fk_module_id = m.module_id) AS chapter_count
FROM
module as m
LEFT JOIN
module_has_chapter as mhc ON m.module_id = mhc.fk_module_id
LEFT JOIN
chapter as c ON mhc.fk_chapter_id = c.chapter_id
group by m.module_id
UNION
SELECT DISTINCT
chapter_id,
chapter_name,
module_id,
module_name,
user_module_progress,
(SELECT COUNT(fk_chapter_id) FROM module_has_chapter WHERE fk_module_id = m.module_id) AS chapter_count
FROM
module as m
LEFT JOIN
user_has_module as uhm ON m.module_id = uhm.fk_module_id
LEFT JOIN
user as u ON uhm.fk_user_id = u.user_id
LEFT JOIN
chapter as c ON uhm.fk_latest_chapter_id = c.chapter_id
WHERE u.user_id = 2
group by m.module_id;
I got there in the end but, not particularly happy about it. This works but, it's a bloody mess...Does anyone have a better solution please?
SELECT DISTINCT
(null) AS chapter_id,
(null) AS chapter_name,
module_id,
module_name,
(null ) AS user_module_progress,
(SELECT COUNT(fk_chapter_id) FROM module_has_chapter WHERE fk_module_id = m.module_id) AS chapter_count
FROM
module as m
LEFT JOIN
user_has_module as uhm ON m.module_id = uhm.fk_module_id
WHERE
uhm.fk_user_id IS NULL
UNION ALL
SELECT DISTINCT
chapter_id,
chapter_name,
module_id,
module_name,
user_module_progress,
(SELECT COUNT(fk_chapter_id) FROM module_has_chapter WHERE fk_module_id = m.module_id) AS chapter_count
FROM
module as m
LEFT JOIN
user_has_module as uhm ON m.module_id = uhm.fk_module_id
INNER JOIN
user as u ON uhm.fk_user_id = u.user_id
INNER JOIN
chapter as c ON uhm.fk_latest_chapter_id = c.chapter_id
WHERE
u.user_id = 2;

Count number of occurrences based on user id

I'm very new to SQL/MySQL and Stackoverflow for that matter, and I'm trying to create a query through iReport (though I don't have to use iReport) for SugarCRM CE. What I need is to create a report that displays the number of "Referrals", "Voicemails", "Emails", and "Call_ins" that are linked to a specific "user" (employee). The query I currently have set up works; however it is running through the data multiple times generating a report that is 200+ pages. This is the code that I am currently using:
SELECT
( SELECT COUNT(*) FROM `leads` INNER JOIN `leads_cstm` ON `leads`.`id` = `leads_cstm`.`id_c` WHERE (leadtype_c = 'Referral' AND users.`id` = leads.`assigned_user_id`) ),
( SELECT COUNT(*) FROM `leads` INNER JOIN `leads_cstm` ON `leads`.`id` = `leads_cstm`.`id_c` WHERE (leadtype_c = 'VM' AND users.`id` = leads.`assigned_user_id`) ),
( SELECT COUNT(*) FROM `leads` INNER JOIN `leads_cstm` ON `leads`.`id` = `leads_cstm`.`id_c` WHERE (leadtype_c = 'Email' AND users.`id` = leads.`assigned_user_id`) ),
users.`first_name`,users.`last_name`
FROM
`users` users,
`leads` leads
I would appreciate any guidance!
You want to use conditional summation. The following uses MySQL syntax:
SELECT sum(leadtype_c = 'Referral') as Referrals,
sum(leadtype_c = 'VM') as VMs,
sum(leadtype_c = 'Email') as Emails,
users.`first_name`, users.`last_name`
FROM users join
`leads`
on users.`id` = leads.`assigned_user_id` INNER JOIN
`leads_cstm`
ON `leads`.`id` = `leads_cstm`.`id_c`
group by users.id;
You can use COUNT with CASE for this:
SELECT u.first_name,
u.last_name,
count(case when leadtype_c = 'Referral' then 1 end),
count(case when leadtype_c = 'VM' then 1 end),
count(case when leadtype_c = 'Email' then 1 end)
FROM users u
JOIN leads l ON u.id = l.assigned_user_id
JOIN leads_cstm lc ON l.id = lc.id_c
GROUP BY u.id
To match your exact results, you should probably use an OUTER JOIN instead, but this gives you the idea.
A Visual Explanation of SQL Joins