i am facing heavy load from my one query . i have 10 million data in my table and i have done indexing of both pid and fid columns then also mysql explain shows that it is not using any index
here is my query
select * from tableA where pid < '94898' and fid='37' order by id desc limit 1;
my mysql explain output says this
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tableA index fid PRIMARY 4 NULL 152 Using where
but mysql slow shows its scanning millions of data
Related
In this example, id is pk.
And the total number of rows is 541.
In the first example below, all rows with id > 50 are accessed, but because of limit 5, the
rows value is expected to be 5. What's the problem?
The order by column is pk, and the where comparison column is also pk.
=======
explain select * from mytable where id > 50 order by id limit 5;
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
SIMPLE
cpc_pre_booking
range
PRIMARY
PRIMARY
8
491
100
Using where
explain select * from mytable where id > 200 order by id limit 5;
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
SIMPLE
cpc_pre_booking
range
PRIMARY
PRIMARY
8
341
100
Using where
explain select * from mytable where id > 400 order by id limit 5;
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
SIMPLE
cpc_pre_booking
range
PRIMARY
PRIMARY
8
141
100
Using where
I want to know what am i missing
The "Rows" in EXPLAIN are estimates based on statistics, and usually without taking into account LIMIT value. Don't trust them very far.
After the fact, the slowlog will provide exact "Rows_examined" and "Rows_returned".
Here's a another way (also by actually running the query):
FLUSH STATUS;
SELECT ...;
SHOW SESSION STATUS LIKE 'Handler%';
As mention in the title, I would like to know any solution for this by not using store prodecure, temporarily table etc.
Compare Query#1 and Query#3, Query#3 get worst performance result. Does it have any workaround to put in variable but without impact the performance result.
Schema (MySQL v5.7)
create table `order`(
id BIGINT(20) not null auto_increment,
transaction_no varchar(20) not null,
primary key (`id`),
unique key uk_transaction_no (`transaction_no`)
);
insert into `order` (`id`,`transaction_no`)
value (1,'10001'),(2,'10002'),(3,'10003'),(4,'10004');
Query #1
explain select * from `order` where transaction_no in ('10001','10004');
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
order
range
uk_transaction_no
uk_transaction_no
22
2
100
Using where; Using index
Query #2
set #transactionNos = "10001,10004";
There are no results to be displayed.
Query #3
explain select * from `order` where find_in_set(transaction_no, #transactionNos);
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
order
index
uk_transaction_no
22
4
100
Using where; Using index
Short Answer: See Sargeability
Long Answer:
MySQL makes no attempt to optimize an expression when an indexed column when it is hidden inside a function call such as FIND_IN_SET(), DATE(), etc. Your Query 3 will always be performed as a full table scan or a full index scan.
So the only way to optimize what you are doing is to construct
IN ('10001','10004')
That is often difficult to achieve when "binding" a list of values. IN(?) will not work in most APIs to MySQL.
I'll show my query first, then the tables. I've tried forcing the index, but it just won't. When that part of the query is ran as a new query on its own, it uses the index and is fast, but since it won't in my full query, it's incredibly slow / never completes.
SELECT p.*, INET_NTOA(p.ip) AS ipStr, qs.score, db.*
FROM proxies p
LEFT JOIN ipdb db FORCE INDEX FOR JOIN (`iprange`)
ON db.ipStart <= p.ip AND db.ipEnd >= p.ip
LEFT JOIN ipqs qs ON qs.ip = p.ip
WHERE expiration_date < '2021-09-18'
ORDER BY expiration_date
LIMIT 500
'iprange' is an index on ipStart + ipEnd.
There are indexes on p.ip and expiration_date
Explain results:
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
p
range
expirationdate
expirationdate
4
NULL
2547
Using index condition
1
SIMPLE
db
ALL
iprange
NULL
NULL
NULL
8334413
Range checked for each record (index map: 0x2)
1
SIMPLE
qs
eq_ref
PRIMARY
PRIMARY
4
adscend_Aff.p.ip
1
NULL
The query of ipdb, ran by itself, sometimes uses the index and sometimes doesn't.... When it doesn't it takes 17 seconds, when it does it takes 0.4 seconds.
explain SELECT * FROM ipdb db WHERE db.ipStart <= 785476891 AND db.ipEnd >= 785476891;
explain SELECT * from ipdb db where db.ipStart <= 16941057 AND db.ipEnd >= 16941057;
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
db
ALL
iprange
NULL
NULL
NULL
8334413
Using where
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
db
range
iprange
iprange
4
NULL
86
Using index condition
When I force the index:
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
db
range
iprange
iprange
4
NULL
1818402
Using index condition
and takes 1.8 seconds.
Tried FORCE INDEX instead of FORCE INDEX FOR JOIN in the larger query, but no difference. Not sure how to address this. Tried splitting this into two steps and doing the second step within a php loop but it's still crazy slow that way
If startIp is at the wrong end of the table, forcing that index will force it to go through most of the table. You can't win.
Start over on designing the table and the queries. Here is a technique that runs O(1) instead of O(N): http://mysql.rjweb.org/doc.php/ipranges
i have very simple mysql table with 5 columns but 5 million data. earlier when data was less my server load was very less but now the load is increasing as the data is more than 5 million and i expect it to reach 10 million by this year end so server will be more slow.i have used indexed wisely
structure is very simple with id as auto increment and primary key and i am filtering the data based on id only which is automatically indexed as it is primary key(i tried indexing it as well but no benefit)
table A
id pid title app get
my query is
EXPLAIN SELECT * FROM tableA ORDER BY id DESC LIMIT 4061280 , 10
and explain says
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tableA ALL NULL NULL NULL NULL 4700461 Using filesort
i dont want to go through all rows as it will slow down my server and create heavy load for file sorting as it will create temporary files either in buffer or in disc.
please advice any good idea to solve this issue.
when my id is indexed why it will go through all rows and reach to desired row.it can not jump directly to that row????
Assuming you don't have "gaps" (read deleted records) in your id..
SELECT * FROM tableA WHERE id > 4061279 and id <= 4061290 ORDER BY id DESC
Ok next
SELECT * FROM tableA WHERE id <= 4061290 ORDER BY id DESC LIMIT 10
I tried the SQL code:
explain SELECT * FROM myTable LIMIT 1
As a result I got:
id select_type table type possible_keys key key_len ref **rows**
1 SIMPLE myTable ALL NULL NULL NULL NULL **32117**
Do you know why the query would run though all rows instead of simply picking the first row?
What can I change within the query (or in my table) to reduce the line amount for a similar result?
The rows count shown is only an estimate of the number of rows to examine. It is not always equal to the actual number of rows examined when you run the query.
In particular:
LIMIT is not taken into account while estimating number of rows Even if you have LIMIT which restricts how many rows will be examined MySQL will still print full number.
Source
When the query actually runs only one row will be examined.
Edited for use of subselect:
Assuming the primary key is "my_id" , use WHERE. For instance:
select * from mytable
where my_id = (
select max(my_id) from mytable
)
While this seems less efficient at first, the result is as such in explain, resulting in just one row returned and a read on the index to find max. I do not suggest doing this against partitioned tables in MySQL:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY mytable const PRIMARY PRIMARY 4 const 1
2 SUBQUERY NULL NULL NULL NULL NULL NULL NULL Select tables optimized away