I am running this command on a QOVENDOR table.
EXPLAIN SELECT *
FROM QOVENDOR
WHERE V_NAME LIKE "B%"
ORDER BY V_AREACODE
QOVENDOR table:
CREATE TABLE `qovendor` (
`V_CODE` int(11) NOT NULL,
`V_NAME` varchar(35) NOT NULL,
`V_CONTACT` varchar(15) NOT NULL,
`V_AREACODE` char(3) NOT NULL,
`V_PHONE` char(8) NOT NULL,
`V_STATE` char(2) NOT NULL,
`V_ORDER` char(1) NOT NULL,
PRIMARY KEY (`V_CODE`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
The output I get is:
+------+-------------+----------+------+---------------+------+---------+------+------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+----------+------+---------------+------+---------+------+------+-----------------------------+
| 1 | SIMPLE | QOVENDOR | ALL | NULL | NULL | NULL | NULL | 15 | Using where; Using filesort |
+------+-------------+----------+------+---------------+------+---------+------+------+-----------------------------+
This information should help me build more efficient queries, but I am having a hard time understanding how. The only non null column with some intersteing information is Extra, select_ype, Type and Rows. I am indeed using where clause, not sure what Using filesort means, besides that it relates to order by. How can I deduce if this query is the most efficient as it can be?
For evaluating performance I should have some sort of CPU Cost, and time data (like Oracle DBMS provides with EXPLAIN command).
In MySQL explain will not provide you with time, nor with CPU cost information. MySQL documentation on explain describes exactly the meaning of each column in the output:
Column JSON Name Meaning
id select_id The SELECT identifier
select_type None The SELECT type
table table_name The table for the output row
partitions partitions The matching partitions
type access_type The join type
possible_keys possible_keys The possible indexes to choose
key key The index actually chosen
key_len key_length The length of the chosen key
ref ref The columns compared to the index
rows rows Estimate of rows to be examined
filtered filtered Percentage of rows filtered by table condition
Extra None Additional information
From a query optimisation point of view, probably the type, possible_keys, key, rows, and extra fields are the most important. Their detailed description can be found on the linked MySQL documentation.
Related
Version - 10.4.25-MariaDB
I have a table where column(name) is a second part of primary key(idarchive,name).
When i run count(*) on table where name like 'done%', its using the index on field name properly but when i run select * its not using the separate index instead using primary key and slowing down the query.
Any idea what we can do here ?
any changes in optimizer switch or any other alternative which can help ?
Note - we can't use force index as queries are not controlable.
Table structure:
CREATE TABLE `table1 ` (
`idarchive` int(10) unsigned NOT NULL,
`name` varchar(255) NOT NULL,
`idsite` int(10) unsigned DEFAULT NULL,
`date1` date DEFAULT NULL,
`date2` date DEFAULT NULL,
`period` tinyint(3) unsigned DEFAULT NULL,
`ts_archived` datetime DEFAULT NULL,
`value` double DEFAULT NULL,
PRIMARY KEY (`idarchive`,`name`),
KEY `index_idsite_dates_period` (`idsite`,`date1`,`date2`,`period`,`ts_archived`),
KEY `index_period_archived` (`period`,`ts_archived`),
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Queries:
explain select count(*) from table1 WHERE name like 'done%' ;
+------+-------------+-------------------------------+-------+---------------+------+---------+------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------------------------------+-------+---------------+------+---------+------+---------+--------------------------+
| 1 | SIMPLE | table1 | range | name | name | 767 | NULL | 9131455 | Using where; Using index |
+------+-------------+-------------------------------+-------+---------------+------+---------+------+---------+--------------------------+
1 row in set (0.000 sec)
explain select * from table1 WHERE name like 'done%' ;
+------+-------------+-------------------------------+------+---------------+------+---------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------------------------------+------+---------------+------+---------+------+----------+-------------+
| 1 | SIMPLE | table1 | ALL | name | NULL | NULL | NULL | 18262910 | Using where |
+------+-------------+-------------------------------+------+---------------+------+---------+------+----------+-------------+
1 row in set (0.000 sec) ```
Your SELECT COUNT(*) ... LIKE 'constant%' query is covered by your index on your name column. That is, the entire query can be satisfied by reading the index. So the query planner decides to range-scan your index to generate the result.
On the other hand, your SELECT * query needs all columns from all rows of the table. That can't be satisfied from any of your indexes. And, it's possible your WHERE name like 'done%' filter reads a significant fraction of the table, enough so the query planner decides the fastest way to satisfy it is to scan the entire table. The query planner figures this out by using statistics on the contents of the table, plus some knowledge of the relative costs of IO and CPU.
If you have just inserted many rows into the table you could try doing ANALYZE TABLE table1 and then rerun the query. Maybe after the table's statistics are updated you'll get a different query plan.
And, if you don't need all the columns, you could stop using SELECT * and instead name the columns you need. SELECT * is a notorious query-performance antipattern, because it often returns column data that never gets used. Once you know exactly what columns you want, you could create a covering index to provide them.
These days the query planner does a pretty good job of optimizing simple queries such as yours.
In MariaDB you can say ANALYZE FORMAT=JSON SELECT.... It will run the query and show you details of the actual execution plan it used.
I tried the following SQL query to update the table INDEXED_MERCHANT where I have 10000 records in the table. I indexed both "Name" and "A" as Indexed keys to improve the update query performance. By executing the command SHOW CREATE TABLE INDEXED_MERCHANT; I'll get an output result as follows:
INDEXED_MERCHANT | CREATE TABLE `INDEXED_MERCHANT` (
`ID` varchar(50) NOT NULL,
`NAME` varchar(200) DEFAULT NULL,
`ONLINE_STATUS` varchar(10) NOT NULL,
`A` varchar(100) DEFAULT NULL,
`B` varchar(200) DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `NAME` (`NAME`,`A`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
Here this means both my keys are recognized as indexed keys. When I execute the following command, the "Extra" column result says the query doesn't using the index keys. How should I achieve my goal ?
Executed query : EXPLAIN EXTENDED UPDATE INDEXED_MERCHANT SET ONLINE_STATUS = '0' WHERE NAME = 'A 205' AND A = 'P 205';
Result:
+----+-------------+------------------+-------+---------------+------+---------+-------------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | **Extra** |
+----+-------------+------------------+-------+---------------+------+---------+-------------+------+----------+-------------+
| 1 | SIMPLE | INDEXED_MERCHANT | range | NAME | NAME | 906 | const,const | 1 | 100.00 | **Using where** |
+----+-------------+------------------+-------+---------------+------+---------+-------------+------+----------+-------------+
Your query is hitting "NAME" index as evident in explain output column "key".
Here is the explanation of key and extra columns from mysql documentation
key
The key column indicates the key (index) that MySQL actually decided to use. If MySQL decides to use one of the possible_keys indexes to look up rows, that index is listed as the key value.
Using where
A WHERE clause is used to restrict which rows to match against the next table or send to the client. Unless you specifically intend to fetch or examine all rows from the table, you may have something wrong in your query if the Extra value is not Using where and the table join type is ALL or index. Even if you are using an index for all parts of a WHERE clause, you may see Using where if the column can be NULL.
I am using COUNT(*) with MATCH() ... AGAINST(). My specific query is as follows:
SELECT COUNT(*) FROM `source_code` WHERE MATCH(`html`) AGAINST ('title');
I get results after a few seconds:
+----------+
| count(*) |
+----------+
| 17346 |
+----------+
1 row in set (16.30 sec)
After running the query multiple times, the query always takes around 16 seconds to complete.
Is there any way to speed up this query? Why isn't query cache caching the results of this query?
In case it's helpful, here is the EXPLAIN and CREATE TABLE statements:
EXPLAIN SELECT COUNT(*) FROM `source_code` WHERE MATCH(`html_w`) AGAINST ('title');
+----+-------------+-------------+----------+---------------+--------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+----------+---------------+--------+---------+------+------+-------------+
| 1 | SIMPLE | source_code | fulltext | html | html | 0 | | 1 | Using where |
+----+-------------+-------------+----------+---------------+--------+---------+------+------+-------------+
Looks like the index is being used. (Maybe the overhead is that the query is still Using where? Is it normal for key_len to be 0?)
SHOW CREATE TABLE `source_code`;
CREATE TABLE `source_code` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`url` varchar(255) NOT NULL,
`domain` varchar(255) DEFAULT NULL,
`title` varchar(255) DEFAULT NULL,
`html` longtext,
`crawled` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `url` (`url`),
KEY `crawled` (`crawled`),
KEY `domain` (`domain`),
FULLTEXT KEY `html` (`html`)
) ENGINE=MyISAM AUTO_INCREMENT=78707 DEFAULT CHARSET=latin1
Nothing too crazy in the CREATE TABLE statement.
Unlike many other databases, mysql is very good had handling select count(*) queries when there is an index that covers the entire table. In your case you do have an index that covers the whole table but it's different from a normal primary key since it's a full text index.
You can see that the query analyzer tries to use that index (possible_keys) but it's actually unable to us it.
The key_len column indicates the length of the key that MySQL decided
to use. The length is NULL if the key column says NULL. Note that the
value of key_len enables you to determine how many parts of a
multiple-part key MySQL actually uses
It's most unusual for key_len to be 0 instead of null, but what it means is that 0 parts of your index was used for the query.
As for how to optimize this? THe answer is it's very difficult. The only thing I can think of is to create a stop word list and the other is to set the minimum word length. Both these go into your my.cnf file.
I have a table with 25 million rows, indexed appropriately.
But adding the clause AND status IS NULL turns a super fast query into a crazy slow query.
Please help me speed it up.
Query:
SELECT
student_id,
grade,
status
FROM
grades
WHERE
class_id = 1
AND status IS NULL -- This line delays results from <200ms to 40-70s!
AND grade BETWEEN 0 AND 0.7
LIMIT 25;
Table:
CREATE TABLE IF NOT EXISTS `grades` (
`student_id` BIGINT(20) NOT NULL,
`class_id` INT(11) NOT NULL,
`grade` FLOAT(10,6) DEFAULT NULL,
`status` INT(11) DEFAULT NULL,
UNIQUE KEY `unique_key` (`student_id`,`class_id`),
KEY `class_id` (`class_id`),
KEY `status` (`status`),
KEY `grade` (`grade`)
) ENGINE=INNODB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Local development shows results instantly (<200ms). Production server is huge slowdown (40-70 seconds!).
Can you point me in the right direction to debug?
Explain:
+----+-------------+--------+-------------+-----------------------+-----------------+---------+------+-------+--------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+-------------+-----------------------+-----------------+---------+------+-------+--------------------------------------------------------+
| 1 | SIMPLE | grades | index_merge | class_id,status,grade | status,class_id | 5,4 | NULL | 26811 | Using intersect(status,class_id); Using where |
+----+-------------+--------+-------------+-----------------------+-----------------+---------+------+-------+--------------------------------------------------------+
A SELECT statement can only use one index per table.
Presumably the query before just did a scan using the sole index class_id for your condition class_id=1. Which will probably filter your result set nicely before checking the other conditions.
The optimiser is 'incorrectly' choosing an index merge on class_id and status for the second query and checking 26811 rows which is probably not optimal. You could hint at the class_id index by adding USING INDEX (class_id) to the end of the FROM clause.
You may get some joy with a composite index on (class_id,status,grade) which may run the query faster as it can match the first two and then range scan the grade. I'm not sure how this works with null though.
I'm guessing the ORDER BY pushed the optimiser to choose the class_id index again and returned your query to it's original speed.
Probably I'm missing some silly thing... Apparently MySQL 5.1 keeps doing a Filesort even when there is an index that matches exactly the column in the ORDER BY clause. To post it here, I've oversimplified the data model, but the issue is still happening:
Table definition:
CREATE TABLE `event` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
`owner_id` int(11) DEFAULT NULL,
`date_created` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `owner_id` (`owner_id`),
KEY `date_created` (`date_created`),
CONSTRAINT `event_ibfk_1` FOREIGN KEY (`owner_id`) REFERENCES `user_profile` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8;
My problem is that event a simple SELECT is showing "Using filesort":
explain select * from event order by date_created desc;
And the result for the query explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE event ALL NULL NULL NULL NULL 6 Using filesort
Is there any way for this type of queries to use the index insteas of doing a filesort?
Thanks in advance to everybody.
Since your CREATE TABLE statement indicates that you have less than 10 rows (AUTO_INCREMENT=7) and using FORCE INDEX on my installation will make MySQL use the index, I'm guessing the optimizer thinks a table scan is faster (less random I/O) than an index scan (since you're selecting all columns, not just date_created). This is confirmed by the following:
mysql> explain select date_created from event order by date_created;
+----+-------------+-------+-------+---------------+--------------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+--------------+---------+------+------+-------------+
| 1 | SIMPLE | event | index | NULL | date_created | 9 | NULL | 1 | Using index |
+----+-------------+-------+-------+---------------+--------------+---------+------+------+-------------+
1 row in set (0.00 sec)
In the above case, the index scan is faster because only the indexed column needs to be returned.
The MySQL documentation has some cases where using an index is considered slower: http://dev.mysql.com/doc/refman/5.1/en/how-to-avoid-table-scan.html
Q: Is there any way for this type of queries to use the index instead of doing a filesort?
A: To have MySQL use the index if at all possible, try:
EXPLAIN SELECT * FROM event FORCE INDEX (date_created) ORDER BY date_created DESC;
By using the FORCE INDEX (index_name), this tells MySQL to make use of the index if it's at all possible. Absent that directive, MySQL will choose the most efficient way to return the result set. A filesort may be more efficient than using the index.