My table has 1,000,000 rows and 4 columns:
id cont stat message
1 rgrf 0 ttgthyhtg
2 frrgt 0 tthyrt
3 4r44 1 rrttttg
...
I am performing a select query which is very slow even though I have done indexing
SELECT * FROM tablea WHERE stat='0' order by id LIMIT 1
This query is making my mysql very slow, I checked with mysql explain and found this
explain SELECT * FROM tablea WHERE stat='0' order by id LIMIT 1
and I was shocked by the output but I don't know how to optimize it.
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tablea ref stat stat 4 const 216404 Using where
There are 216,404 rows for optimizing that I have to reduce to 1 or 2 but how?
The problem is that MySQL can only use one index per table in a query, this is index stat in your case. So, the ORDER BY is performed without use of index, which is very slow on 1M rows.
Try the following:
Implicitly use the correct index:
SELECT * FROM tablea USE INDEX(PRIMARY) WHERE stat='0' order by id LIMIT 1
Create a composite index, just as Ollie Jones said above.
I suggest you try creating a compound index on (stat, id). This may allow your search / order operation to be optimized. There's a downside, of course: you'll incur extra overhead with insertions and updates.
CREATE INDEX ON tablea (stat,id) USING BTREE
Give it a try.
Related
I have table with around 60 000 rows. I have this two queries that are drastically different in speed. Can you explain why?
SELECT COUNT(id) FROM table;
300ms - 58936 rows
Explain:
id
select_type
table
partitions
type
possible_keys
key
key_length
ref
rows
filtered
Extra
1
SIMPLE
table
NULL
index
NULL
table_id_index
8
NULL
29325
100.00
using index
SELECT COUNT(id) FROM table WHERE dummy = 1;
50ms - 58936 rows
Explain:
id
select_type
table
partitions
type
possible_keys
key
key_length
ref
rows
filtered
Extra
1
SIMPLE
table
NULL
index
NULL
dummy_index
5
const
14662
100.00
using index
It depends.
COUNT(id) may be slower than COUNT(*). The former checks id for being NOT NULL; the latter simply counts the rows. (If id is the PRIMARY KEY, then this is unlikely to make any measurable difference.)
The Optimizer may decide to scan the entire table rather than use an index.
The Optimizer may pick an irrelevant index if not forced to by the WHERE clause. In your example, any index with id can be used for the first, and any index with both dummy and id for the second.
If you run the same query twice, it may run much faster the second time due to caching. This can happen even for a 'similar' query. I suspect this is the "answer". I often see a speedup of 10x if the first run was from disk and the second found everything needed in cache (the buffer_pool).
To get more insight, do EXPLAIN SELECT ...
The optimal index for your second query is INDEX(dummy, id).
I have the following query:
SELECT *
FROM table
WHERE
structural_type=1
AND parent_id='167F2-F'
AND points_to_id=''
# AND match(search) against ('donotmatch124213123123')
The search takes about 10ms to run, running on the composite index (structural_type, parent_id, points_to_id). However, when I add in the fts index, the query balloons to taking ~1s, regardless of what is contained in the match criteria. Basically it seems like it 'skips the index' whenever I have a fts search applied.
What would be the best way to optimize this query?
Update: a few explains:
EXPLAIN SELECT... # without fts
id select_type table partitions type possible_keys key key_len ref rows filtered Extra
1 SIMPLE table NULL ref structural_type structural_type 209 const,const,const 2 100.00 NULL
With fts (also adding 'force index'):
explain SELECT ... force INDEX (structural_type) AND match...
id select_type table partitions type possible_keys key key_len ref rows filtered Extra
1 SIMPLE table NULL fulltext structural_type,search search 0 const 1 5.00 Using where; Ft_hints: sorted
The only thing I can think of which would be incredibly hack-ish, would be to add an additional term to the fts so it does the filter 'within' that. For example:
fts_term = fts_term += " StructuralType1ParentID167F2FPointsToID"
The MySQL optimizer can only use one index for your WHERE clause, so it has to choose between the composite one and the FULLTEXT one.
Since it can't run both queries to bench which one is faster, it will estimate how fast will different execution plans be.
To do so, MySQL uses some internal stats it keeps about each table. But those stats can be very different from the reality if they aren't updated and the data changes in the table.
Running a OPTIMIZE TABLE table query allows MySQL to refresh its table stats, so it will be able to perform better estimates and choose the better index.
Try expressing this without the full text logic, using like:
SELECT *
FROM table
WHERE structural_type = 1 AND
parent_id ='167F2-F' AND
points_to_id = '' AND
search not like '%donotmatch124213123123%';
The index should still be used for the first three columns. LIKE might be slow, but if not many rows match the first three, this might not be as bad as using the full text index.
I have a pretty long insert query that inserts data from a select query in a table. The problem is that the select query takes too long to execute. The table is MyISAM and the select locks the table which affects other users who also use the table. I have found that problem of the query is a join.
When I remove this part of the query, it takes less then a second to execute but when I leave this part the query takes more than 15 minutes:
LEFT JOIN enq_217 Pex_217
ON e.survey_panelId = Pex_217.survey_panelId
AND e.survey_respondentId = Pex_217.survey_respondentId
AND Pex_217.survey_respondentId != 0
db.table_1 contains 5,90,145 rows and e contains 4,703 rows.
Explain Output:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY e ALL survey_endTime,survey_type NULL NULL NULL 4703 Using where
1 PRIMARY Pex_217 ref survey_respondentId,idx_table_1 idx_table_1 8 e.survey_panelId,e.survey_respondentId 2 Using index
2 DEPENDENT SUBQUERY enq_11525_timing eq_ref code code 80 e.code 1
How can I edit this part of the query to be faster?
I suggest creating an index on the table db.table_1 for the fields panelId and respondentId
You want an index on the table. The best index for this logic is:
create index idx_table_1 on table_1(panelId, respondentId)
The order of these two columns in the index should not matter.
You might want to include other columns in the index, depending on what the rest of the query is doing.
Note: a single index with both columns is different from two indexes with each column.
Why is it a LEFT join?
How many rows in Pex_217?
Run ANALYZE TABLE on each table used. (This sometimes helps MyISAM; rarely is needed for InnoDB.)
Since the 'real problem' seems to be that the query "holds up other users", switch to InnoDB.
Tips on conversion
The JOIN is not that bad (with the new index -- note Using index): 4703 rows scanned, then reach into the other table's index about 2 times each.
Perhaps the "Dependent subquery" is the costly part. Let's see that.
I have a table in MySQL (5.5.31) which has about 20M rows. The following query:
SELECT DISTINCT mytable.name name FROM mytable
LEFT JOIN mytable_c ON mytable_c.id_c = mytable.id
WHERE mytable.deleted = 0 ORDER BY mytable.date_modified DESC LIMIT 0,21
is causing full table scan, with explain saying type is ALL and extra info is Using where; Using temporary; Using filesort. Explain results:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE mytable ALL NULL NULL NULL NULL 19001156 Using where; Using temporary; Using filesort
1 SIMPLE mytable_c eq_ref PRIMARY PRIMARY 108 mytable.id 1 Using index
Without the join explain looks like:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE mytable index NULL mytablemod 9 NULL 21 Using where; Using temporary
id_c is the primary key for mytable_c and mytable_c does not have more than one row for every row in mytable. date_modified is indexed. But looks like MySQL does not understand that. If I remove the DISTINCT clause, then explain uses index and touches only 21 rows just as expected. If I remove the join it also does this. Is there any way to make it work without the full table scan with the join? explain shows mysql knows it needs only one row from mytable_c and it is using the primary key, but still does full scan on mytable.
The reason DISTINCT is there that the query is generated by the ORM system in which there might be cases where multiple rows may be produced by JOINs, but the values of SELECT fields will always be unique (i.e. if JOIN is against multi-value link only fields that are the same in every joined row will be in SELECT).
These are just generic comments, not mysql specific.
To find all the possible name values from mytable a full scan of either the table or an index needs to happen. Possible options:
full table scan
full index scan of an index starting with deleted (take advantage of the filter)
full index scan of an index starting with name (only column of concern for output)
If there was an index on deleted, the server could find all the deleted = 0 index entries and then look up the corresponding name value from the table. But if deleted has low cardinality or the statistics aren't there to say differently, it could be more expensive to do the double reads of first the index then the corresponding data item. In that case, just scan the table.
If there was an index on name, an index scan could be sufficient, but then the table needs to be checked for the filter. Again frequent hopping from index to table.
The join columns also need to be considered in a similar manner.
If you forget about the join part and had a multi-part index on columns name, deleted then an index scan would probably happen.
Update
To me the DISTINCT and ORDER BY parts are a bit confusing. Of which name record is the date_modified to be used for sorting? I think something like this would be a bit more clear:
SELECT mytable.name name --, MIN(mytable.date_modified)
FROM mytable
LEFT JOIN mytable_c ON mytable_c.id_c = mytable.id
WHERE mytable.deleted = 0
GROUP BY mytable.name
ORDER BY MIN(mytable.date_modified) DESC LIMIT 0,21
Either way, once the ORDER BY comes into play, a full scan needs to be done to find the order. Without the ORDER BY, the first 21 found could suffice.
Why do not you try to move condition mytable.deleted = 0 from WHERE to the JOIN ON ? You can also try FORCE INDEX (mytablemod)
i have very simple mysql table with 5 columns but 5 million data. earlier when data was less my server load was very less but now the load is increasing as the data is more than 5 million and i expect it to reach 10 million by this year end so server will be more slow.i have used indexed wisely
structure is very simple with id as auto increment and primary key and i am filtering the data based on id only which is automatically indexed as it is primary key(i tried indexing it as well but no benefit)
table A
id pid title app get
my query is
EXPLAIN SELECT * FROM tableA ORDER BY id DESC LIMIT 4061280 , 10
and explain says
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tableA ALL NULL NULL NULL NULL 4700461 Using filesort
i dont want to go through all rows as it will slow down my server and create heavy load for file sorting as it will create temporary files either in buffer or in disc.
please advice any good idea to solve this issue.
when my id is indexed why it will go through all rows and reach to desired row.it can not jump directly to that row????
Assuming you don't have "gaps" (read deleted records) in your id..
SELECT * FROM tableA WHERE id > 4061279 and id <= 4061290 ORDER BY id DESC
Ok next
SELECT * FROM tableA WHERE id <= 4061290 ORDER BY id DESC LIMIT 10