I have a table with an indexed column named date_c with type Date. The problem is when I query something like this
explain select * from table where date_c > '2021-06-29'
the possible_keys is idx_..75 and key is idx_..75 too but when I change comparison like this:
explain select * from table where date_c < '2021-06-29'
in this case possible_keys is idx_..75 again but key is NULL
Why is that? Also I tried these but got the same results:
explain select * from table where date_c > DATE(NOW()) - 2 DAY key is not NULL
explain select * from table where date_c > DATE(NOW()) - 20 DAY key is NULL
Why mysql does not use possible_keys sometimes?
Related
I have an index on the created_at column on my actions table. When I run the following query
explain
select * from `actions` where `created_at` between '2022-01-18 06:00:00' and '2022-01-30 05:59:59'
I get the following which shows that the index is being used.
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
actions
NULL
range
actions_created_index
actions_created_index
5
NULL
3775484
100.00
Using index condition
However, if I change the date to something like
explain
select * from `actions` where `created_at` between '2022-01-01 06:00:00' and '2022-01-30 05:59:59'
The output is now
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
actions
NULL
ALL
actions_created_index
NULL
NULL
NULL
28446203
25.04
Using where
This query now does not use the created_at index. Is this an expected behavior? What can I do to ensure the created_at index is always being used regardless of the date range?
MySQL uses a query optimizer to come up with a query plan. In your second query, it predicted that ignoring the b-tree index would be faster since the range was so large.
Also see this answer: https://dba.stackexchange.com/questions/48072/why-does-mysql-ignore-the-index-even-on-force-for-this-order-by
You can use
...
FORCE INDEX FOR ORDER BY created_at
but even this is only a hint to the optimizer. https://dev.mysql.com/doc/refman/8.0/en/index-hints.html
I'll show my query first, then the tables. I've tried forcing the index, but it just won't. When that part of the query is ran as a new query on its own, it uses the index and is fast, but since it won't in my full query, it's incredibly slow / never completes.
SELECT p.*, INET_NTOA(p.ip) AS ipStr, qs.score, db.*
FROM proxies p
LEFT JOIN ipdb db FORCE INDEX FOR JOIN (`iprange`)
ON db.ipStart <= p.ip AND db.ipEnd >= p.ip
LEFT JOIN ipqs qs ON qs.ip = p.ip
WHERE expiration_date < '2021-09-18'
ORDER BY expiration_date
LIMIT 500
'iprange' is an index on ipStart + ipEnd.
There are indexes on p.ip and expiration_date
Explain results:
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
p
range
expirationdate
expirationdate
4
NULL
2547
Using index condition
1
SIMPLE
db
ALL
iprange
NULL
NULL
NULL
8334413
Range checked for each record (index map: 0x2)
1
SIMPLE
qs
eq_ref
PRIMARY
PRIMARY
4
adscend_Aff.p.ip
1
NULL
The query of ipdb, ran by itself, sometimes uses the index and sometimes doesn't.... When it doesn't it takes 17 seconds, when it does it takes 0.4 seconds.
explain SELECT * FROM ipdb db WHERE db.ipStart <= 785476891 AND db.ipEnd >= 785476891;
explain SELECT * from ipdb db where db.ipStart <= 16941057 AND db.ipEnd >= 16941057;
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
db
ALL
iprange
NULL
NULL
NULL
8334413
Using where
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
db
range
iprange
iprange
4
NULL
86
Using index condition
When I force the index:
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
db
range
iprange
iprange
4
NULL
1818402
Using index condition
and takes 1.8 seconds.
Tried FORCE INDEX instead of FORCE INDEX FOR JOIN in the larger query, but no difference. Not sure how to address this. Tried splitting this into two steps and doing the second step within a php loop but it's still crazy slow that way
If startIp is at the wrong end of the table, forcing that index will force it to go through most of the table. You can't win.
Start over on designing the table and the queries. Here is a technique that runs O(1) instead of O(N): http://mysql.rjweb.org/doc.php/ipranges
here is the "explain" of my query:
explain
select eil.sell_fmt, count(sell_fmt) as itemCount
from table_items eil
where eil.cl_Id=123 and eil.si_Id='0'
and start_date <= now() and end_date is not null and end_date < NOW()
group by eil.sell_fmt
without date (start_date, end_date) filters:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE eil ref table_items_clid_siid_sellFmt 39 const,const 7393 Using where; Using index
With date filters:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE eil ref table_items_clid_siid_sellFmt 39 const,const 8400 Using where
possible_keys are:
table_items_clid_siid, table_items_clid_siid_itemId, table_items_clid_siid_startDate_endDate, table_items_clid_siid_sellFmt
The query without date filters is very fast (0.4 sec), but with date filters, its taking about 30 seconds. total records are 14K only.
Table field types:
`cl_Id` int(11) NOT NULL,
`si_Id` varchar(11) NOT NULL,
`start_date` datetime DEFAULT NULL,
`end_date` datetime DEFAULT NULL,
`sell_fmt` varchar(20) DEFAULT NULL
I concatenated field-names to give index names, so you can estimate combined fields available in the index.
Can somebody guide me here? what's going on here? what is the best course of action i should take here, or where i'm doing wrong?
I need one more suggestion: in another query on same table, a user can filter based on UPTO 10 fields, and in no definite order of fields (random no of fields in random order). Then this type search would be too slow again. What's the best strategy then? one covering index with "all" possible searchable fields? if yes, does the order of fields in index matter? (i.e. if that order is different than that of fields in query, will the index be used?
First, without seeing your create table statement, I can offer the following... create composite index (multiple fields) that best match your common querying elements applicable to the where clause, starting with the smaller nominal count basis. Since you are explicitly looking for a "cl_ID" and "si_ID" plus start and end dates. Since you have a group by, I would add that to the index for optimization purposes and be a completely COVERING index so the engine does not need to go back to the raw data to complete the query. It can resolve by all fields in the index directly.
I would have an index on
( cl_id, si_id, start_date, end_date, sell_fmt )
Finally, change your count from count(sell_fmt) to just count(*) indicating an "i don't care about a specific field, just as long as a record is found, count it"
I'm trying to get this query to use an index.
explain SELECT SQL_NO_CACHE * FROM products WHERE (stock_disabled = 1 OR negative_stock_allowed = 1 OR stock > 0)
This is what it returns:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE p ALL multi_index NULL NULL NULL 2890 Using where
The multi_index is an that contains stock_disabled, negative_stock_allowed and stock in the same order. I think the index is not working because of the multiple OR statements. What can I do here?
"USE INDEX foo" if you explicitly want to use an index
SELECT * FROM products USE INDEX(`multi_index`)
I've got a table with a column called "date".
The table looks somthing like this
CREATE TABLE IF NOT EXISTS `offers_log_archive` (
...
`date` date DEFAULT NULL,
...
KEY `date` (`date`)
) ENGINE=InnoDB
I perform the following query on this table:
SELECT
*
FROM
offers_log_archive as ola
WHERE
ola.date >= "2012-12-01" and
ola.date <= "2012-12-31"
Then I did the following:
explain (SELECT
*
FROM
offers_log_archive as ola
WHERE
ola.date >= "2012-12-01" and
ola.date <= "2012-12-31" );
The result of this explain is
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ola ALL date NULL NULL NULL 6206460 Using where
Why do I get type ALL? From what I know that basically means that the query will inspect every row in the table and ignores the index on date. Although I would expect that mysql would use this.
What happens here and why is the date index ignored?
Almost all values in your column are within the range of the query, so not only would the index be next to useless (it would add little value), but it would actually be much more expensive to use the index than do a simple table scan.
Edit
Try first running ANALYZE on the table:
ANALYZE TABLE MYTABLE
If that doesn't help, try changing the syntax to use BETWEEN:
WHERE ola.date BETWEEN '2012-12-01' AND '2012-12-31'