I am trying to optimize mysql to decrease my server load.. And here i have a complex query which will be used about 1k times/minute on a quad core server with 8gb ram and my server is going down.
I have tried many ways to rewrite this query :
SELECT *
FROM (
SELECT a.id,
a.url
FROM surf a
LEFT JOIN users b
ON b.id = a.user
LEFT JOIN surfed c
ON c.user = 'asdf' AND c.site = a.id
WHERE a.active = '0'
AND (b.coins >= a.cpc AND a.cpc >= '2')
AND (c.site IS NULL AND a.user !='asdf')
ORDER BY a.cpc DESC, b.premium DESC
LIMIT 100) AS records
ORDER BY RAND()
LIMIT 1
But it didn't work. So can you guys help me to rewrite the above query so that it would not waste any resources ?
Also this query doesn't have any indexes :( . It would be very helpful to guide me creating indexes for this.
The problem is most likely the inner sort.
You should have indexes on surfed(site, user, cpc), surf(active, user, site), and user(id, coins).
You can also perhaps make minor improvements by switching the join to inner joins from outer joins. The where clause is undoing the left outer join anyway, so this won't affect the results.
But I don't think these changes will really help. The problem is the sort of the result set in the inner query. The outer sort by rand() is a minor issue, because you are only doing that on 100 rows.
If you are running this 1,000/minute, you will need to rethink your data structures and develop an approach that has the data you need more readily available.
Related
I have the below query, which I appreciate probably isn't well written, but on my local PC with Xampp and MariaDB it executes in 0.1719 seconds, which is about the speed I would hope for.
However, on my development server with Plesk and MariaDB the same query with the same data takes over 12 seconds. Obviously would be no use.
Probably the query could be modified to make it better, but can somebody explain why the performance difference? The server is a VPS, it has no shortage of resources - it isn't live so usage is almost none at all, yet still 12+ seconds for this query.
The query:
SELECT m.id AS match_id, e.event AS event1
FROM matches m
JOIN competitions co ON co.id = m.competition
JOIN clubs h ON h.id = m.hometeam
JOIN clubs a ON a.id = m.awayteam
LEFT JOIN match_events e ON e.match = m.id
AND e.player = '7138'
WHERE (m.hometeam = '1'
OR m.awayteam = '1'
)
AND m.season = '121'
Are you sure you need AND e.player = '7138' in the ON clause of a LEFT JOIN and not in the WHERE clause?
Better indexing
Recommend these composite, covering, indexes:
m: (season, awayteam, hometeam, competition, id)
e: (player, match, event)
Avoiding OR
OR optimizes poorly. A common trick is to turn it into UNION. Such may work for your query:
SELECT ...
FROM matches JOIN ...
WHERE m.season = 121
AND m.hometeam = 1
UNION ALL
SELECT ...
FROM matches JOIN ...
WHERE m.season = 121
AND m.awayteam = 1
And have these two indexes:
INDEX(season, hometeam) -- will be used by one part of the UNION
INDEX(season, awayteam) -- will be used by the other
I chose UNION ALL because it is faster than UNION DISTINCT. But if you get unwanted dups, change it.
My query runs a bit slow (slower than I would like). I know the general basics of MySQL but not enough to optimize queries. My query is as follows:
SELECT a.index, a.date, b.description, a.type, a.place, a.value, b.space,a.val2
FROM a LEFT JOIN b ON a.index = b.index
WHERE a.date >= ? and a.date < ?
AND a.type = 100;
I have an index on A's index and type as well as B's index. I know this query could be faster, so I was wondering if anyone might know of some proper optimization.
Thanks
I have one query that is loading db so much and my hosting provider complains
SELECT count(a.id),
a.*,
CASE
WHEN CHAR_LENGTH(a.alias) THEN CONCAT_WS(":", a.id, a.alias)
ELSE a.id
END AS slug,
CASE
WHEN CHAR_LENGTH(cc.alias) THEN CONCAT_WS(":", cc.id, cc.alias)
ELSE cc.id
END AS catslug
FROM jos_chrono_comments AS com,
jos_content AS a
LEFT JOIN jos_content_frontpage AS f ON f.content_id = a.id
INNER JOIN jos_content AS c ON f.content_id = c.id
INNER JOIN jos_categories AS cc ON cc.id = a.catid
INNER JOIN jos_sections AS s ON s.id = a.sectionid
WHERE (a.state = 1
AND s.id > 0)
AND s.published = 1
AND cc.published = 1
AND a.id = com.pageid
AND DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= c.publish_up
GROUP BY (com.pageid)
ORDER BY 1 DESC LIMIT 0,
10
it's Joomla 1.5 related and Chronocomments module for most commented in 30 days
I have some hints here https://goo.gl/0wF2ex but I am not so good to rewerite that in better way without using temp table
Looking for help to make that query not so heavy for mysql server, maybe eliminating group by or any hint will be usefull
Thanks,
K#m0
There are multiple possible answers, this is focusing on the database with no change to the PHP code.
Explain query
With the MySQL EXPLAIN command, that you can use e.g. from phpMyAdmin, you can have optimization tips from MySQL itself, mostly related to adding indexes if required.
Possible indexes
As a wild guess, I would make sure that all the following fields are indexed:
jos_content.catid
jos_content.sectionid
jos_content.state
jos_content_frontpage.content_id (if you have many items in frontpage)
jos_categories.published (if you have many categories)
jos_sections.published (if you have many sections)
jos_chrono_comments.pageid
Final notes
The ORDER BY 1 DESC part seems useless to me.
The increase of performance is really dependant from the size of your tables, so don't expect miracles. But it's definitely worth a try.
I am currently running into an issue where, when I use a "LIKE" in my query I get the result in 2 seconds. But when I use the '=' instead, it takes around 1 minute for the result to show up.
The following is my query:
QUERY1
The following query takes 2 seconds:
`select distinct p.Name from Timeset s
join table1 f on (f.id = s.id)
join table2 p on (p.source=f.table_name)
join table3 d on (d.Name = p.Name) WHERE
s.Active = 'Y' AND **p.sourcefrom like '%sometable%'`
QUERY2
The same query replacing the 'like' by '=' takes 1 minute:
select distinct p.Name from Timeset s
join table1 f on (f.id = s.id)
join table2 p on (p.source=f.table_name)
join table3 d on (d.Name = p.Name) WHERE
s.Active = 'Y' AND **p.sourcefrom = 'sometable'
I am really puzzled because I know that 'LIKE' is usually slower (than '=') since mysql need to look for different possibilities. But I am sure why in my case, "=" is slower with such a substantial difference.
thank you kindly for the help in advance,
regards,
When you use = MySQL is probably using a different index compared to when you use LIKE. Check the output from the two execution plans and see what the differnce is. Then you can FORCE the use of the better performing index. Might be worth running ANALYZE TABLE for each of the tables involved.
I have a bulk query with subquery. My query works fine when I run it on development server, but when I've try it pn the live server, the query takes too much time to produce an output. I think it's because of a big data on the live server. Can anyone help me on how to index query on MySQL so that it will lessen the time execution.
Here is my query:
SELECT prd.fldemployeeno AS Empno,
(SELECT fldemployeename FROM tblprofile prf WHERE prf.fldemployeeno = prd.fldemployeeno LIMIT 0,1) AS Empname,
'01' AS `Week`,
COUNT(DISTINCT isAud.fldid) AuditedFiles,
COUNT(qua.seqid) ErrorCount,
COUNT(DISTINCT qua.fldid) OrdersWithError
FROM tbldownloadITL dwn
INNER JOIN tblproductionITL prd
ON dwn.fldid = prd.fldglobalid
INNER JOIN (SELECT p.fldemployeeno,fldglobalid,p.fldstarttime,COALESCE(q.fldstarttime,p.fldstarttime) `AuditDate`
FROM tblproductionitl p
LEFT JOIN tblqualityaudit q
ON p.fldemployeeno=q.fldemployeeno
AND p.fldstarttime=q.fldprodstarttime
AND p.fldglobalid=q.fldid
WHERE p.fldprojectgroup='PROJGROUP') temp
ON prd.fldglobalid=temp.fldglobalid
AND prd.fldemployeeno=temp.fldemployeeno
AND prd.fldstarttime=temp.fldstarttime
INNER JOIN tblisauditedITL isAud
USING (fldid)
LEFT JOIN tblqualityaudit qua
ON qua.fldid = dwn.fldid
AND qua.fldbusunit = dwn.fldbusunit
AND qua.fldprojectGroup = dwn.fldprojectGroup
AND qua.fldemployeeno = prd.fldemployeeno
AND qua.fldprodstarttime = prd.fldstarttime
AND qua.flderrorstatus != 'NOT ERROR'
LEFT JOIN tblerrorcategory
USING (flderrorcategoryid)
LEFT JOIN tblerrortypes
USING (flderrortypeid)
WHERE dwn.fldbusunit = 'BUSUNIT'
AND dwn.fldprojectGroup = 'PROJGROUP'
AND temp.AuditDate BETWEEN '2011-07-29 00:00:00' AND '2011-07-29 23:59:59'
GROUP BY prd.fldemployeeno
ORDER BY Empname
Here is also the description of the query:
I would suggest installing Sphinx on the your server if you have the access. That way you can have an indexed resource at your finger tips for extremely fast searching, on top of that you can add the execution of what is called a 'delta' index to allow for real time updating of your mysql database. It is highly customizable. Hopefully this will help you out.
http://sphinxsearch.com/