I'm trying to increase the speed of the sql code below. Load time right now is around 0.662 sec. The problem is that i need to loop this code for each day of the selected month and then 31*0.662 sec ~30sec is way to long time for loading.
select fname,lname,(TIME_TO_SEC(TIMEDIFF(r.edate,r.sdate))-r.break) as TotalDiff from tbluser u LEFT JOIN
tblregtime r
on (r.userid = u.id and
r.projectid = 21
and sdate='2013-11-27'
)
INNER JOIN tblgroup_users gU ON gU.userID = u.id
INNER JOIN tblgroup_brukare gB on gB.tblGroupID=gU.tblGroupID where (gB.tblprojectID = 21 AND (gU.status=0 OR gU.status=2))
order by u.fname ASC,u.lname ASC
Instead of running your sql query 31 times for each day, you could try running a single query for all days and handle them appropriately in your code (php or whatever).
Here's a suggested alternate of your query which will run only one time (you may need to rephrase it a bit). Can you try it and let us know how long that takes? Also, to further optimize, it will be helpful to post your query plan, and maybe create an sql fiddle.
select fname,lname,(TIME_TO_SEC(TIMEDIFF(r.edate,r.sdate))-r.break) as TotalDiff, sdate
from tbluser u
LEFT JOIN tblregtime r on (r.userid = u.id and r.projectid = 21 and sdate between '2013-11-01' and '2013-11-31')
INNER JOIN tblgroup_users gU ON gU.userID = u.id
INNER JOIN tblgroup_brukare gB on gB.tblGroupID=gU.tblGroupID where (gB.tblprojectID = 21 AND (gU.status=0 OR gU.status=2))
order by sdate ASC, u.fname ASC,u.lname ASC
Related
Is there a better way to write this query as it is searching in around 400k rows and taking 50 seconds to get executed?
SELECT pfile_print_media_id
, market.market_name
, full_name
, image
, pfile_date
FROM pfile_print_media
LEFT
JOIN pfiles
ON pfiles.pfile_id = pfile_print_media.pfile_id
LEFT
JOIN publications
ON publications.publication_id = pfile_print_media.publication_id
LEFT
JOIN clients
ON pfiles.client_id = clients.client_id
LEFT
JOIN client_markets
ON client_markets.client_id = clients.client_id
LEFT
JOIN market
ON market.market_id = clients.market
WHERE pfiles.media_type = 'PRINT'
AND clients.market = 17
OR client_markets.market_id = 17
OR client_type = 'INTERNATIONAL'
GROUP
BY pfiles.pfile_id
ORDER
BY pfiles.pfile_date DESC
LIMIT 4
Explain query output:
If no columns are used from publications table - may be replacing join with sub query to just check the existence of key will speed up the query.
Awhile back I got some help with a specific query. Here's the link: SQL Group BY using strings in new columns
My query looks similar to this:
SELECT event_data, class_40_winner, class_30_winner
FROM events e
LEFT JOIN (SELECT result_event, name AS class_40_winner
FROM results
WHERE class = 40 AND position = 1) c40 ON e.id = c40.result_event
LEFT JOIN (SELECT result_event, name AS class_30_winner
FROM results
WHERE class = 30 AND position = 1) c30 ON e.id = c30.result_event
I have now entered enough data in my database (22,000 rows) that this query is taking over 6 seconds to complete. (My actual query is bigger than the above, in that it now has 4 joins in it.)
I used the "Explain" function on my query to take a look. Each of the queries from the "results" table is pulling in the 22,000 rows, so this seems to be the problem.
I have done some research and it sounds like I should be able to INDEX the relevant column on the "results" table to help speed things up. But when I did that, it actually slowed my query down to about 10 seconds.
Any suggestions for what I can do to improve this query?
AFAIK, you are pivoting your data and I think using max(case ...) ... group by has good performance in pivoting data.
I can suggest you to use this query instead:
select event_date
, max(case when r.class = 40 then name end) `Class 40 Winner`
, max(case when r.class = 30 then name end) `Class 30 Winner`
from events e
left join results r on e.event_id = r.result_event and r.position = 1
group by event_date;
[SQL Fiddle Demo]
Try this query:
SELECT
e.event_date,
r1.name as class_40_winner,
r2.name as class_30_winner
FROM
events e,
results r1,
results r2
WHERE
r1.class = 40 AND
r2.class = 30 AND
r1.position = 1 AND
r2.position = 1 AND
r1.result_event = e.id AND
r2.result_event = e.id
SELECT e.event_data
, r.class
, r.name winner
FROM events e
JOIN results r
ON r.result_event = e.id
WHERE class IN (30,40)
AND position = 1
The rest of this problem is a simple display issue, best resolved in application code.
There are 3 tables, persontbl1, persontbl2 (each 7500 rows) and schedule (~3000 active schedules i.e. schedule.status = 0). Person tables contain data for the same persons as one to one relationship and INNER join between two takes less than a second. And schedule table contains data about persons to be interviewed and not all persons have schedules in schedule table. With Left join query instantly takes around 45 seconds, which is causing all sorts of issues.
SELECT persontbl1._CREATION_DATE, persontbl2._TOP_LEVEL_AURI,
persontbl2.RESP_CNIC, persontbl2.RESP_CNIC_NAME,
persontbl1.MOB_NUMBER1, persontbl1.MOB_NUMBER2,
schedule.id, schedule.call_datetime, schedule.enum_id,
schedule.enum_change, schedule.status
FROM persontbl1
INNER JOIN persontbl2 ON (persontbl2._TOP_LEVEL_AURI = persontbl1._URI)
AND (AGR_CONTACT=1)
LEFT JOIN SCHEDULE ON (schedule.survey_id = persontbl1._URI)
AND (SCHEDULE.status=0)
AND (DATE(SCHEDULE.call_datetime) <= CURDATE())
ORDER BY schedule.call_datetime IS NULL DESC, persontbl1._CREATION_DATE ASC
Here is the explain for query:
Schedule Table structure:
Schedule Table indexes:
Please let me know if any further information is required.
Thanks.
Edit: Added fully qualified table names and their columns.
You should just replace this line:
AND (DATE(SCHEDULE.call_datetime) <= CURDATE())
to this one:
AND SCHEDULE.call_datetime <= '2015-04-18 00:00:00'
so mysql will not call 2 functions per every record but will use static constant '2015-04-18 00:00:00'.
So you can just try for performance improvements if your query is:
SELECT persontbl1._CREATION_DATE, persontbl2._TOP_LEVEL_AURI,
persontbl2.RESP_CNIC, persontbl2.RESP_CNIC_NAME,
persontbl1.MOB_NUMBER1, persontbl1.MOB_NUMBER2,
schedule.id, schedule.call_datetime, schedule.enum_id,
schedule.enum_change, schedule.status
FROM persontbl1
INNER JOIN persontbl2 ON (persontbl2._TOP_LEVEL_AURI = persontbl1._URI)
AND (AGR_CONTACT=1)
LEFT JOIN SCHEDULE ON (schedule.survey_id = persontbl1._URI)
AND (SCHEDULE.status=0)
AND (SCHEDULE.call_datetime <= '2015-02-01 00:00:00')
ORDER BY schedule.call_datetime IS NULL DESC, persontbl1._CREATION_DATE ASC
EDIT 1 So you said without LEFT JOIN part it was fast enough, so you can try then:
SELECT persontbl1._CREATION_DATE, persontbl2._TOP_LEVEL_AURI,
persontbl2.RESP_CNIC, persontbl2.RESP_CNIC_NAME,
persontbl1.MOB_NUMBER1, persontbl1.MOB_NUMBER2,
s.id, s.call_datetime, s.enum_id,
s.enum_change, s.status
FROM persontbl1
INNER JOIN persontbl2 ON (persontbl2._TOP_LEVEL_AURI = persontbl1._URI)
AND (AGR_CONTACT=1)
LEFT JOIN
(SELECT *
FROM SCHEDULE
WHERE status=0
AND call_datetime <= '2015-02-01 00:00:00'
) s
ON s.survey_id = persontbl1._URI
ORDER BY s.call_datetime IS NULL DESC, persontbl1._CREATION_DATE ASC
I'm guessing that AGR_CONTACT comes from p1. This is the query you want to optimize:
SELECT p1._CREATION_DATE, _TOP_LEVEL_AURI, RESP_CNIC, RESP_CNIC_NAME,
MOB_NUMBER1, MOB_NUMBER2,
s.id, s.call_datetime, s.enum_id, s.enum_change, s.status
FROM persontbl1 p1 INNER JOIN
persontbl2 p2
ON (p2._TOP_LEVEL_AURI = p1._URI) AND (p1.AGR_CONTACT = 1) LEFT JOIN
SCHEDULE s
ON (s.survey_id = p1._URI) AND
(s.status = 0) AND
(DATE(s.call_datetime) <= CURDATE())
ORDER BY s.call_datetime IS NULL DESC, p1._CREATION_DATE ASC;
The best indexes for this query are: persontbl2(agr_contact), persontbl1(_TOP_LEVEL_AURI, _uri), and schedule(survey_id, status, call_datime).
The use of date() around the date time is not recommended. In general, that precludes the use of indexes. However, in this case, you have a left join, so it doesn't make a difference. That column is not being used for filtering anyway. The index on schedule is only for covering the on clause.
The below query is very slow (takes around 1 second), but is only searching approx 2500 records (+ inner joined tables).
if i remove the ORDER BY, the query runs in much less time (0.05 or less)
OR if i remove the part nested select below "# used to select where no ProfilePhoto specified" it also runs fast, but i need both of these included.
I have indexes (or primary key) on :tPhoto_PhotoID, PhotoID, p.Enabled, CustomerID, tCustomer_CustomerID, ProfilePhoto (bool), u.UserName, e.PrivateEmail, m.tUser_UserID, Enabled, Active, m.tMemberStatuses_MemberStatusID, e.tCustomerMembership_MembershipID, e.DateCreated
(do i have too many indexes? my understanding is add them anywhere i use WHERE or ON)
The Query :
SELECT e.CustomerID,
e.CustomerName,
e.Location,
SUBSTRING_INDEX(e.CustomerProfile,' ', 25) AS Description,
IFNULL(p.PhotoURL, PhotoTable.PhotoURL) AS PhotoURL
FROM tCustomer e
LEFT JOIN (tCustomerPhoto ep INNER JOIN tPhoto p ON (ep.tPhoto_PhotoID = p.PhotoID AND p.Enabled=1))
ON e.CustomerID = ep.tCustomer_CustomerID AND ep.ProfilePhoto = 1
# used to select where no ProfilePhoto specified
LEFT JOIN ((SELECT pp.PhotoURL, epp.tCustomer_CustomerID
FROM tPhoto pp
LEFT JOIN tCustomerPhoto epp ON epp.tPhoto_PhotoID = pp.PhotoID
GROUP BY epp.tCustomer_CustomerID) AS PhotoTable) ON e.CustomerID = PhotoTable.tCustomer_CustomerID
INNER JOIN tUser u ON u.UserName = e.PrivateEmail
INNER JOIN tmembers m ON m.tUser_UserID = u.UserID
WHERE e.Enabled=1
AND e.Active=1
AND m.tMemberStatuses_MemberStatusID = 2
AND e.tCustomerMembership_MembershipID != 6
ORDER BY e.DateCreated DESC
LIMIT 12
i have similar queries that but they run much faster.
any opinions would be grateful:
Until we get more clarity on your question between working in other query etc..Try EXPLAIN {YourSelectQuery} in MySQL client and see the suggestions to improve the performance.
I have a bulk query with subquery. My query works fine when I run it on development server, but when I've try it pn the live server, the query takes too much time to produce an output. I think it's because of a big data on the live server. Can anyone help me on how to index query on MySQL so that it will lessen the time execution.
Here is my query:
SELECT prd.fldemployeeno AS Empno,
(SELECT fldemployeename FROM tblprofile prf WHERE prf.fldemployeeno = prd.fldemployeeno LIMIT 0,1) AS Empname,
'01' AS `Week`,
COUNT(DISTINCT isAud.fldid) AuditedFiles,
COUNT(qua.seqid) ErrorCount,
COUNT(DISTINCT qua.fldid) OrdersWithError
FROM tbldownloadITL dwn
INNER JOIN tblproductionITL prd
ON dwn.fldid = prd.fldglobalid
INNER JOIN (SELECT p.fldemployeeno,fldglobalid,p.fldstarttime,COALESCE(q.fldstarttime,p.fldstarttime) `AuditDate`
FROM tblproductionitl p
LEFT JOIN tblqualityaudit q
ON p.fldemployeeno=q.fldemployeeno
AND p.fldstarttime=q.fldprodstarttime
AND p.fldglobalid=q.fldid
WHERE p.fldprojectgroup='PROJGROUP') temp
ON prd.fldglobalid=temp.fldglobalid
AND prd.fldemployeeno=temp.fldemployeeno
AND prd.fldstarttime=temp.fldstarttime
INNER JOIN tblisauditedITL isAud
USING (fldid)
LEFT JOIN tblqualityaudit qua
ON qua.fldid = dwn.fldid
AND qua.fldbusunit = dwn.fldbusunit
AND qua.fldprojectGroup = dwn.fldprojectGroup
AND qua.fldemployeeno = prd.fldemployeeno
AND qua.fldprodstarttime = prd.fldstarttime
AND qua.flderrorstatus != 'NOT ERROR'
LEFT JOIN tblerrorcategory
USING (flderrorcategoryid)
LEFT JOIN tblerrortypes
USING (flderrortypeid)
WHERE dwn.fldbusunit = 'BUSUNIT'
AND dwn.fldprojectGroup = 'PROJGROUP'
AND temp.AuditDate BETWEEN '2011-07-29 00:00:00' AND '2011-07-29 23:59:59'
GROUP BY prd.fldemployeeno
ORDER BY Empname
Here is also the description of the query:
I would suggest installing Sphinx on the your server if you have the access. That way you can have an indexed resource at your finger tips for extremely fast searching, on top of that you can add the execution of what is called a 'delta' index to allow for real time updating of your mysql database. It is highly customizable. Hopefully this will help you out.
http://sphinxsearch.com/