MySQL Query Optimization Tips - mysql

Question

I have a game application in which users users answer questions and rating is based on the time elapsed on answering this questions.
I am trying to build a query that returns a the rating for top 20 players. the game has some stages and I need to retrieve the players which played all stages (assume the number of stages are 5)
This is what have I wrote:
SELECT `usersname` , `time`
FROM `users`
WHERE `users`.`id`
IN (
SELECT `steps`.`user_id`
FROM `steps`
GROUP BY `steps`.`user_id`
HAVING COUNT( `steps`.`id` ) = 5
)
ORDER BY `time` ASC
LIMIT 20
In the inner Select I am selecting all user_id-s who have played 5 stages (steps). The query works correctly but It's horribly slow. It takes about minute and a half to execute. can you provide some tips on optimizing it. Inner Select returns about 2000 rows.
Feel free to ask me if you need additional information.

Answer 1

Try with JOIN, instead of IN (SELECT ...):
SELECT usersname , `time`
FROM users
JOIN
( SELECT steps.user_id
FROM steps
GROUP BY steps.user_id
HAVING COUNT(*) = 5
) grp
ON grp.user_id = users.id
ORDER BY `time` ASC
LIMIT 20

Answer 2

Assuming that you have an index on users.time, which is the first obvious optimization, replacing HAVING with WHERE in the inner query may be worth a try.
The query optimizer might do this already if you are lucky, but you cannot rely on it, and strictly to the specification, HAVING runs after fetching every record whereas WHERE prunes them before.
If that does not help, simply having a counter that increments for every stage completed in the users table might speed up things, eleminating the sub-query. This will make completing a stage minimally slower (but this won't happen a million times per second!), but will be very fast to query only the users who have completed all 5 stages (especially if you have an index on that field).
Also, using memcached or some similar caching technology may be worthwile for something like a highscore, which is typically of the kind of "not necessarily 100% accurate to the second, changing slowly, queried billions of times" data.
If memcached is not an option, even writing the result to a temp file and re-using that for 1-2 seconds (or even longer) would be an option. Nobody will notice. Even if you cache highscores for as long as 1-2 minutes, still nobody will take offense because that is just "how long it takes".

Answer 3

I think you should use where instead of having. Also, in my opinion you should do this in a stored function. In my opinion the best way is to use where instead of having, also, run the inner query, store the results and run the outer query based on the results of your inner query.

Answer 4

This use case may benefit from de-normalization. There is no need to search through all 2000 user records to determine if a user is better than 20 records.
Create a Top_20_Users table.
After the 5th stage check if user's time is less than any in the Top_20_Users table. If yes, update the slowest/worst record.
Things you can do with this.
Since the Top_20_Users table will be so small, add a field for stage and include Top 20 times for each stage as well as for all five stages completed.
Let the Top_20_Users table grow. A history of all top 20 users ever, their times and the date when that time was good enough to be a top 20. Show trends as users learn the game and the top 20 times get better and better.