SELECT count, item, itemid
FROM items
ORDER BY count DESC
LIMIT 20
Takes .0011
Explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE items index NULL count 4 NULL 20
I have indexes on itemid(primary key) and count (INDEX)
Does anyone have suggestions for how this could be better accomplished?
It seems like your long_query_time variable/setting is extremely short. The default is 10 seconds, but if your query is taking 0.0011 seconds, it obviously shouldn't be logged with the default setting. Try increasing it to something reasonable for your setup (1 second+ probably) and see if this still happens.
Related
I have an explain result below , problem is how to calculate the total examined rows? please explain this detail ~ （first ask question~ If there is any mistake, please correct me, I will be very grateful）
id select_type type possible_keys key_len rows
1 PRIMARY ALL 1423656
1 PRIMARY eq_ref PRIMARY 8 1
1 PRIMARY ref 152 1
1 PRIMARY ALL 138
1 PRIMARY ALL 1388
1 PRIMARY ALL 1564
3 DERIVED ALL 1684
3 DERIVED eq_ref PRIMARY 8 1
2 DERIVED ALL 141
From the manual : https://dev.mysql.com/doc/refman/5.7/en/explain-output.html
rows (JSON name: rows)
The rows column indicates the number of rows MySQL believes it must
examine to execute the query.
For InnoDB tables, this number is an estimate, and may not always be
exact.
You have a very high number of 1.4 million for one of your table but the possible_keys column is empty. That means this is a table that is desperately crying out to be indexed.
A large number of rows to be examined, means just that. Mysql needs to read all those rows to give you your result.
If you had posted your tables and your query, we could have helped you figure out what those indexes ought to be.
Hi i have little long time query which takes almost 400ms .
this is my query
SELECT id, `from`
FROM messages
WHERE `to` = ?
AND `to_viewed` = '0'
AND `to_deleted` = '0'
AND TIMESTAMPDIFF(SECOND,created,?)< 20 AND TIMESTAMPDIFF(SECOND,created,?)>= 0
How can i optimize this ?
obs i have no idex .
EDIT to show my EXPLAIN
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE messages ALL created NULL NULL NULL 10 Using where
Make sure you have an index on the created column. Then use
AND created BETWEEN ? AND DATE_ADD(?, INTERVAL 20 SECOND);
An index can't be used when you perform a calculation on the column. So you should calculate the ends of the time range and then compare the column to that.
Below is my query to get 20 rows with genre_id 1.
EXPLAIN SELECT * FROM (`content`)
WHERE `genre_id` = '1'
AND `category` = 1
LIMIT 20
I have total 654 rows in content table with genre_id 1, I have index on genre_id and in above query I am limiting result to display only 20 records which is working fine but explain is showing 654 records under rows, I tried to add index on category but still same result and then also I removed AND category = 1 but same rows count:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE content ref genre_id genre_id 4 const 654 Using where
HERE I found the answer
LIMIT is not taken into account while estimating number of rows Even
if you have LIMIT which restricts how many rows will be examined MySQL
will still print full number
But also In comments another reply was posted:
LIMIT is now taken into account when estimating number of rows. I’m
not sure which version addressed this, but in 5.1.30, EXPLAIN
accurately takes LIMIT into account.
I am using MySQL 5.5.16 with InnoDB. so as per above comment its still not taking into account. So my question is does mysql go through all 654 rows to return 20 rows even I have set limit? Thanks
Reply from Rick James at MySQL
Does mysql LIMIT is taken into account when estimating number of rows in Explain?
No. (5.7 with JSON may be a different matter.)
I have been going through my slow queries and doing what I can to property optimize each one. I ran across this one, that I have been stuck on.
EXPLAIN SELECT pID FROM ds_products WHERE pLevel >0
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ds_products ALL pLevel NULL NULL NULL 45939 Using where
I have indexed pLevel [tinyint(1)], but the query is not using it and doing a full table scan.
Here is the row count of this table for each value of pLevel:
pLevel count
0 34040
1 3078
2 7143
3 865
4 478
5 279
6 56
if I do the query for a specific value of pLevel, it does use the index:
EXPLAIN SELECT pID FROM ds_products WHERE pLevel =6
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ds_products ref pLevel pLevel 1 const 1265
I've tried pLevel>=1 and pLevel<=6... but it still does a full scan
I've tried (pLevel=1 or pLevel=2 or pLevel=3 or pLevel=4 or pLevel=5 or pLevel=6) .... but it still does a full table scan.
Try using MySQL GROUP BY.
SELECT pLevel, COUNT(*)
FROM ds_products
GROUP BY pLevel
Edit:
This MySQL documentation article may be useful to you. How to Avoid Table Scans
First, here is the query I have:
SELECT
COUNT(*) as velocity_count,
SUM(`disbursements`.`amount`) as summation_amount
FROM `disbursements`
WHERE
`disbursements`.`accumulation_hash` = '40ad7f250cf23919bd8cc4619850a40444c5e90c978f88635a09ccf66a82ffb38e39ea51cdfd651b0ebdac5f5ca37cd7a17e0f60fea6cbce1397ccff5fa37346'
AND `disbursements`.`caller_id` = 1
AND `disbursements`.`active` = 1
AND (version_hash != '86b4111677294b27a1805643d193b8d437b6ddb170b4ed5dec39aa89bf070d160cbbcd697dfc1988efea8429b1f1557625bf956180c65d3dcd3a318280e0d2da')
AND (`disbursements`.`created_at` BETWEEN '2012-12-15 23:33:22'
AND '2013-01-14 23:33:22') LIMIT 1
Explain extended returns the following:
+----+-------------+---------------+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+---------+------+--------+----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------------+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+---------+------+--------+----------+--------------------------+
| 1 | SIMPLE | disbursements | range | unique_request_index,index_disbursements_on_caller_id,disbursement_summation_index,disbursement_velocity_index,disbursement_version_out_index | disbursement_summation_index | 1543 | NULL | 191422 | 100.00 | Using where; Using index |
+----+-------------+---------------+-------+-----------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+---------+------+--------+----------+--------------------------+
The actual query counts about 95,000 rows. If I explain another query that hits ~50 rows the explain is identical, just with fewer rows estimated.
The index being chosen covers accumulation_hash, caller_id, active, version_hash, created_at, amount in that order.
I've tried playing around with doing COUNT(id) or COUNT(caller_id) since these are non-null fields and return the same thing as count(*), but it doesn't have any impact on the plan or the run time of the actual query.
This is also a heavy insert table, essentially every single query will have had a row inserted or updated since the last time it was run, so the mysql query cache isn't entirely useful.
Before I go and make some sort of bucketed time sequence cache with something like memcache or redis, is there an obvious solution to getting this to work much faster? A normal ~50 row query returns in 5MS, the ones across 90k+ rows are taking 500-900MS and I really can't afford anything much past 100MS.
I should point out the dates are a rolling 30 day window that needs to be essentially real time. Expiration could probably happen with ~1 minute granularity, but new items need to be seen immediately upon commit. I'm also on RDS, Read IOPS are essentially 0, and cpu is about 60-80%. When I'm not querying the giant 90,000+ record items, CPU typically stays below 10%.
You could try an index that has created_at before version_hash (might get a better shot at having an index range scan... not clear how that non-equality predicate on the version_hash affects the plan, but I suspect it disables a range scan on the created_at column.
Other than that, the query and the index look about as good as you are going to get, the EXPLAIN output shows the query being satisfied from the index.
And the performance of the statement doesn't sound too unreasonable, given that it's aggregating 95,000+ rows, especially given the key length of 1543 bytes. That's a much larger size than I normally deal with.
What are the datatypes of the columns in the index, and what is the cluster key or primary key?
accumulation_hash - 128-character representation of 512-bit value
caller_id - integer or numeric (?)
active - integer or numeric (?)
version_hash - another 128-characters
created_at - datetime (8bytes) or timestamp (4bytes)
amount - numeric or integer
95,000 rows at 1543 bytes each is on the order of 140MB of data.