Optimize MYSQL query having lots of data - mysql

Is there a better way to write this query as it is searching in around 400k rows and taking 50 seconds to get executed?
SELECT pfile_print_media_id
, market.market_name
, full_name
, image
, pfile_date
FROM pfile_print_media
LEFT
JOIN pfiles
ON pfiles.pfile_id = pfile_print_media.pfile_id
LEFT
JOIN publications
ON publications.publication_id = pfile_print_media.publication_id
LEFT
JOIN clients
ON pfiles.client_id = clients.client_id
LEFT
JOIN client_markets
ON client_markets.client_id = clients.client_id
LEFT
JOIN market
ON market.market_id = clients.market
WHERE pfiles.media_type = 'PRINT'
AND clients.market = 17
OR client_markets.market_id = 17
OR client_type = 'INTERNATIONAL'
GROUP
BY pfiles.pfile_id
ORDER
BY pfiles.pfile_date DESC
LIMIT 4
Explain query output:

If no columns are used from publications table - may be replacing join with sub query to just check the existence of key will speed up the query.

Related

MySQL complicated join doesn't return what I would expect

I have query like this
SELECT a1.user_id
, cwp1.value as status
, COUNTa1.user_id) count
, MIN(cwp1.display_order) lowest_status
FROM a1
JOIN ms1
ON ms1.model_id = a1.id
JOIN wp1
ON ms1.status_id = wp1.id
JOIN cwp1
ON cwp1.workflow_parameter_id = wp1.id
WHERE wp1.type = 12
AND cwp1.company_id = 999
GROUP
BY a.user_id
ORDER
BY lowest_status ASC
I then get x rows for distinct user_id, the issue is that a user can have more than one record in table a1.
So i said OK, i don't want random records, I want the one that has lowest cwp1.display_order. Unfortunately above query doesn't work, and still brings me random a1 for a user.
What am I doing wrong here?

Understanding why this query is slow

The below query is very slow (takes around 1 second), but is only searching approx 2500 records (+ inner joined tables).
if i remove the ORDER BY, the query runs in much less time (0.05 or less)
OR if i remove the part nested select below "# used to select where no ProfilePhoto specified" it also runs fast, but i need both of these included.
I have indexes (or primary key) on :tPhoto_PhotoID, PhotoID, p.Enabled, CustomerID, tCustomer_CustomerID, ProfilePhoto (bool), u.UserName, e.PrivateEmail, m.tUser_UserID, Enabled, Active, m.tMemberStatuses_MemberStatusID, e.tCustomerMembership_MembershipID, e.DateCreated
(do i have too many indexes? my understanding is add them anywhere i use WHERE or ON)
The Query :
SELECT e.CustomerID,
e.CustomerName,
e.Location,
SUBSTRING_INDEX(e.CustomerProfile,' ', 25) AS Description,
IFNULL(p.PhotoURL, PhotoTable.PhotoURL) AS PhotoURL
FROM tCustomer e
LEFT JOIN (tCustomerPhoto ep INNER JOIN tPhoto p ON (ep.tPhoto_PhotoID = p.PhotoID AND p.Enabled=1))
ON e.CustomerID = ep.tCustomer_CustomerID AND ep.ProfilePhoto = 1
# used to select where no ProfilePhoto specified
LEFT JOIN ((SELECT pp.PhotoURL, epp.tCustomer_CustomerID
FROM tPhoto pp
LEFT JOIN tCustomerPhoto epp ON epp.tPhoto_PhotoID = pp.PhotoID
GROUP BY epp.tCustomer_CustomerID) AS PhotoTable) ON e.CustomerID = PhotoTable.tCustomer_CustomerID
INNER JOIN tUser u ON u.UserName = e.PrivateEmail
INNER JOIN tmembers m ON m.tUser_UserID = u.UserID
WHERE e.Enabled=1
AND e.Active=1
AND m.tMemberStatuses_MemberStatusID = 2
AND e.tCustomerMembership_MembershipID != 6
ORDER BY e.DateCreated DESC
LIMIT 12
i have similar queries that but they run much faster.
any opinions would be grateful:
Until we get more clarity on your question between working in other query etc..Try EXPLAIN {YourSelectQuery} in MySQL client and see the suggestions to improve the performance.

MySQL join multiple tables and limit the output from "ON"

when i write a MySQL query, there occur a problem. here is my query
SELECT
SUM(view_product_count_details.view_product_count) AS count_sum,
product_details.product_name,
product_details.product_url,
product_details.product_price,
product_image_details.product_image_name,
main_category_details.main_category_url,
sub_category_details.sub_category_url
FROM
view_product_count_details
JOIN
product_details ON view_product_count_details.product_id_fk = product_details.product_id
JOIN
product_image_details ON product_image_details.product_id_fk = view_product_count_details.product_id_fk
JOIN
main_category_details ON product_details.product_main_cat_id = main_category_details.main_category_id
JOIN
sub_category_details ON product_details.product_sub_cat_id_fk = sub_category_details.sub_category_id
WHERE
view_product_count_details.view_product_status = 'active'
GROUP BY view_product_count_details.product_id_fk
ORDER BY count_sum DESC
LIMIT 4
Here I have multiple images for one product.the images are in table "product_image_details". this query returns count as the number of images, where I need the count of product viewed by people which is stored in table "view_product_count_details". when I just pick the count, i got the count as it is. but when i join the table "product_image details", result become wrong. Is there any way to do it in single query?
Please help me... Thanks in advance.... :)
You can do it by having an inline query. I am not sure how this will perform when you have more data.
SELECT table1.*,product_image_details.product_image_name FROM
(
SELECT
SUM(view_product_count_details.view_product_count) AS count_sum,
product_details.product_id,
product_details.product_name,
product_details.product_url,
product_details.product_price,
main_category_details.main_category_url,
sub_category_details.sub_category_url
FROM
view_product_count_details
JOIN
product_details ON view_product_count_details.product_id_fk = product_details.product_id
JOIN
product_image_details ON product_image_details.product_id_fk = view_product_count_details.product_id_fk
JOIN
main_category_details ON product_details.product_main_cat_id = main_category_details.main_category_id
JOIN
sub_category_details ON product_details.product_sub_cat_id_fk = sub_category_details.sub_category_id
WHERE
view_product_count_details.view_product_status = 'active'
GROUP BY view_product_count_details.product_id_fk
ORDER BY count_sum DESC
LIMIT 4
) table1
JOIN
product_image_details ON product_image_details.product_id_fk = table1.product_id
LIMIT 4

Increase speed of slow sql-code

I'm trying to increase the speed of the sql code below. Load time right now is around 0.662 sec. The problem is that i need to loop this code for each day of the selected month and then 31*0.662 sec ~30sec is way to long time for loading.
select fname,lname,(TIME_TO_SEC(TIMEDIFF(r.edate,r.sdate))-r.break) as TotalDiff from tbluser u LEFT JOIN
tblregtime r
on (r.userid = u.id and
r.projectid = 21
and sdate='2013-11-27'
)
INNER JOIN tblgroup_users gU ON gU.userID = u.id
INNER JOIN tblgroup_brukare gB on gB.tblGroupID=gU.tblGroupID where (gB.tblprojectID = 21 AND (gU.status=0 OR gU.status=2))
order by u.fname ASC,u.lname ASC
Instead of running your sql query 31 times for each day, you could try running a single query for all days and handle them appropriately in your code (php or whatever).
Here's a suggested alternate of your query which will run only one time (you may need to rephrase it a bit). Can you try it and let us know how long that takes? Also, to further optimize, it will be helpful to post your query plan, and maybe create an sql fiddle.
select fname,lname,(TIME_TO_SEC(TIMEDIFF(r.edate,r.sdate))-r.break) as TotalDiff, sdate
from tbluser u
LEFT JOIN tblregtime r on (r.userid = u.id and r.projectid = 21 and sdate between '2013-11-01' and '2013-11-31')
INNER JOIN tblgroup_users gU ON gU.userID = u.id
INNER JOIN tblgroup_brukare gB on gB.tblGroupID=gU.tblGroupID where (gB.tblprojectID = 21 AND (gU.status=0 OR gU.status=2))
order by sdate ASC, u.fname ASC,u.lname ASC

Converting subquery to joins for performance

I have taken over a big project, and as the database is becoming large, some of the code stopped working,
Here is the query to find those rendering_requests who's last rending_log is pending, sometimes there are log entries which have no status change and recorded as noaction we dont need to count them. That is what I understood from the query.
SELECT
COUNT(rr.rendering_id) AS recordCount
FROM
rendering_request rr, rendering_log rl
WHERE
rl.rendering_id = rr.rendering_id
AND rl.status = 'pending' AND
rl.log_id = (
SELECT rl1.log_id
FROM rendering_log rl1
WHERE
rl.rendering_id = rl1.rendering_id AND
rl1.status = 'pending'
AND rl1.log_id = (
SELECT rl2.log_id
FROM rendering_log rl2
WHERE rl1.rendering_id = rl2.rendering_id AND rl2.status!='noaction'
ORDER BY rl2.log_id DESC LIMIT 1
)
ORDER BY rl1.log_id DESC
LIMIT 1
)
for example
rendering_id=1 is having multiple logs
status=noaction
status=noaction
status=pending
and
rendering_id=2 is having multiple logs
status=noaction
status=assigned
status=noaction
status=pending
when we run this query it should display count=1 as only the rendering_id=1 is our desired record.
Right now this query has stopped working, and it hangs the mysql server
Not 100% sure I have got this right, but something like this. Think you still need to use a couple of subselects but (depending on the version of MySQL) doing it this way with JOINs should be a lot faster
SELECT COUNT(rr.rendering_id) AS recordCount
FROM rendering_request rr
INNER JOIN rendering_log rl
ON rl.rendering_id = rr.rendering_id
INNER JOIN (SELECT rendering_id, MAX(log_id) FROM rendering_log WHERE status = 'pending' GROUP BY rendering_id) rl1
ON rl1.rendering_id = rl.rendering_id
AND rl1.log_id = rl.log_id
INNER JOIN (SELECT rendering_id, MAX(log_id) FROM rendering_log WHERE status!='noaction' GROUP BY rendering_id) rl2
ON rl2.rendering_id = rl1.rendering_id
AND rl2.log_id = rl1.log_id
WHERE rl.status = 'pending'