I have a query.
SELECT id_id FROM videos_member ORDER BY date_id DESC LIMIT 0,30
Here is the table
CREATE TABLE IF NOT EXISTS `videos` (
`id_id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`date_id` int(11) NOT NULL,
PRIMARY KEY (`id_id`),
KEY `date_id` (`date_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=3 ;
I keep getting this
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE videos ALL NULL NULL NULL NULL 342 Using filesort
Why isn't is using the index?
The table contains (or at least MySQL thinks it contains) 342 rows. This is tiny and likely fits into a single block of physical storage, which means it can be read in a single read operation. Using the index would require at least two read operations. So MySQL might be smart here and realize that reading the whole table at once is just more efficient than reading the index and then using it to access the table.
In other words if you insert more rows into the table the plan might change to using index.
Related
Table:
CREATE TABLE `stores` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`slug` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` json DEFAULT NULL,
`html` mediumtext COLLATE utf8mb4_unicode_ci
GENERATED ALWAYS AS (json_unquote(json_extract(`value`,'$.html')))
STORED,
PRIMARY KEY (`id`),
KEY `slug` (`slug`),
FULLTEXT KEY `html` (`html`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Query:
select id from `stores` where MATCH(stores.html) AGAINST ('forum*' IN BOOLEAN MODE) limit 20
Takes 0.14 seconds
Explain:
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
stores
NULL
fulltext
html
html
0
const
1
100.00
Using where; Ft_hints: no_ranking, limit = 20
When query:
select id from `stores` where stores.html like '%forum%' limit 20
Takes only 0.003 seconds
Explain:
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
stores
NULL
ALL
NULL
NULL
NULL
NULL
134101
100.00
Using where
I remember, when I first implemented this virtual generated fields over json it appeared to be faster than like, but now after implementing it over all the fields I noticed the site got slower. So I started analyzing the simple queries and found out that fulltext is actually significantly slower!
When I add SQL_NO_CACHE after select it makes no difference.
What am I missing?
Thanks
This query is abnormally fast:
select id from `stores`
where stores.html like '%forum%' limit 20
Because it only looked at enough rows to find 20 that had that string. I think you will find that this takes significantly longer since it will check every row:
select id from `stores`
where stores.html like '%non-existent-text%' limit 20
Another possible reason is that MATCH found hundreds, maybe thousands, or rows before it got to doing the LIMIT. So time this:
select id from `stores`
where MATCH(stores.html) AGAINST ('qwertyui' IN BOOLEAN MODE) limit 20
The bottom line is that you may need to live with such inconsistencies. I believe (without solid proof your your dataset) than MATCH will usually be faster than LIKE. Note the word "usually".
I am using mysql db for my application.I have one table with 40 million rows data.I want to run one update query in this table.Table contains three fields.Id,TaxiId,date&distance.I want to update distance to zero for particular Taxiid's.I am using one query like this
update abc set distance=0 where TaxiId in(2026,2031,2033,2035,2037,2039);
When using explain I am getting below result
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE abc range abc_trip_fk abc_trip_fk 5 1916397 Using where
Table Declare
CREATE TABLE `testdb`.`abc` (
`id` int(8) NOT NULL AUTO_INCREMENT,
`TaxiId` int(8) DEFAULT NULL,
`distance` double DEFAULT NULL,
`date` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id),
KEY `trip_fk` (`TaxiId`) USING BTREE,
) ENGINE=InnoDB AUTO_INCREMENT=125127797 DEFAULT CHARSET=latin1;
When I run this query entire application hangs.
Innodb buffer pool option is very important option for mysql performance.
Try to make following settings
innodb_flush_log_at_trx_commit=2
innodb_buffer_pool_size=25GB
Disable indexes while insert/update/delete operation in database.
For more about mysql performance setting find here
I am performing a very simple select over a simple table, where the column that I am filtering over has an index.
Here is the schema:
CREATE TABLE IF NOT EXISTS `tmp_inventory_items` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`transmission_id` int(11) unsigned NOT NULL,
`inventory_item_id` int(11) unsigned DEFAULT NULL,
`material_id` int(11) unsigned NOT NULL,
PRIMARY KEY (`id`),
KEY `transmission_id` (`transmission_id`)
KEY `inventory_item_id` (`inventory_item_id`),
KEY `material_id` (`material_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=21 ;
Here is the SQL:
SELECT * FROM `tmp_inventory_items` WHERE `transmission_id` = 330
However, when explaining the query, I see that the index is NOT being used, why is that (the table has about 20 rows on my local machine)?
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tmp_inventory_items... ALL transmission_id NULL NULL NULL 13 Using where
No key is being used even if I hint the mysql with USE INDEX(transmission_id)... this looks very strange to me (MySQL Version 5.5.28)
Because MySQL's algorithms tell it that preparing an index and using it would use more resources than simply performing the query without one.
When you feed query syntax to a DBMS, one of the things it does is attempts to determine the most efficient way to process the query (usually there are at least tens of ways).
If you want to, you can use FORCE INDEX(transmission_id) (documented here) which will inform MySQL that a table scan is assumed to be very expensive, but it's not recommended as to determine for 20 rows, it's just not valuable.
I have the following mysql query
select points_for_user from items where user_id = '38415';
explain on the query returns this
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE items index NULL points_for_user_index 2 NULL 1000511 Using index
The problem is, shouldn't the number of rows be FAR less then the number of rows in the table because of the index?
user_id is the primary index, so I tried creating an index on just points_for_user and that still look through every row. An index on user_id AND points_for_user still searches every row.
What am I missing?
Thanks!
CREATE TABLE IF NOT EXISTS `items` (
`capture_id` int(11) NOT NULL AUTO_INCREMENT,
`id` int(11) NOT NULL,
`creator_user_id` bigint(20) NOT NULL DEFAULT '0',
`user_id` int(11) NOT NULL,
`accuracy` int(11) NOT NULL,
`captured_at` timestamp NOT NULL DEFAULT '2011-01-01 06:00:00',
`ip` varchar(30) NOT NULL,
`capture_type_id` smallint(6) NOT NULL DEFAULT '0',
`points` smallint(6) NOT NULL DEFAULT '5',
`points_for_user` smallint(6) NOT NULL DEFAULT '3',
PRIMARY KEY (`capture_id`),
KEY `capture_user` (`capture_id`,`id`,`user_id`),
KEY `user_id` (`user_id`,`id`),
KEY `id` (`id`),
KEY `capture_creator_index` (`capture_id`,`creator_user_id`),
KEY `points_capture_index` (`points_for_user`,`creator_user_id`),
KEY `points_for_user_index` (`points_for_user`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1008992 ;
select count(*) from items where user_id = '38415'
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE captures ref user_munzee_id user_munzee_id 4 const 81 Using index
the mysql optimizer try to use the best possible index during the query.
In your first query the optimizer is considering points_for_user_index the best choice, in fact the Extra column show the "Using index" status, this means a "Covering index".
The "Covering index" occurs when all fields required for a query (in your case select points_for_user from ... ) are contained in an index, this avoid the access to the full mysql data (.MYD) in favour of the direct index access (.MYI)
First of all you can try to rebuild the index tree analyzing table
ANALYZE TABLE itemes;
Note for very large tables:
ANALYZE TABLE analyzes and stores the key distribution for a table.
During the analysis, the table is locked with a read lock for InnoDB
and MyISAM. This statement works with InnoDB, NDB, and MyISAM tables.
For MyISAM tables, this statement is equivalent to using myisamchk
--analyze.
If "the problem" persist and you want to bypass the optimizer choice you can explicit try to force the usage of an index
EXPLAIN SELECT points_for_user FROM items USE INDEX ( user_id ) WHERE user_id = '38415'
More details: http://dev.mysql.com/doc/refman/5.5/en/index-hints.html
Cristian
This query:
explain
SELECT `Lineitem`.`id`, `Donation`.`id`, `Donation`.`order_line_id`
FROM `order_line` AS `Lineitem`
LEFT JOIN `donations` AS `Donation`
ON (`Donation`.`order_line_id` = `Lineitem`.`id`)
WHERE `Lineitem`.`session_id` = '1'
correctly uses the Donation.order_line_id and Lineitem.id indexes, shown in this EXPLAIN output:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE Lineitem ref session_id session_id 97 const 1 Using where; Using index
1 SIMPLE Donation ref order_line_id order_line_id 4 Lineitem.id 2 Using index
However, this query, which simply includes another field:
explain
SELECT `Lineitem`.`id`, `Donation`.`id`, `Donation`.`npo_id`,
`Donation`.`order_line_id`
FROM `order_line` AS `Lineitem`
LEFT JOIN `donations` AS `Donation`
ON (`Donation`.`order_line_id` = `Lineitem`.`id`)
WHERE `Lineitem`.`session_id` = '1'
Shows that the Donation table does not use an index:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE Lineitem ref session_id session_id 97 const 1 Using where; Using index
1 SIMPLE Donation ALL order_line_id NULL NULL NULL 3
All of the _id fields in the tables are indexed, but I can't figure out how adding this field into the list of selected fields causes the index to be dropped.
As requested by James C, here are the table definitions:
CREATE TABLE `donations` (
`id` int(10) unsigned NOT NULL auto_increment,
`npo_id` int(10) unsigned NOT NULL,
`order_line_detail_id` int(10) unsigned NOT NULL default '0',
`order_line_id` int(10) unsigned NOT NULL default '0',
`created` datetime default NULL,
`modified` datetime default NULL,
PRIMARY KEY (`id`),
KEY `npo_id` (`npo_id`),
KEY `order_line_id` (`order_line_id`),
KEY `order_line_detail_id` (`order_line_detail_id`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8
CREATE TABLE `order_line` (
`id` bigint(20) unsigned NOT NULL auto_increment,
`order_id` bigint(20) NOT NULL,
`npo_id` bigint(20) NOT NULL default '0',
`session_id` varchar(32) collate utf8_unicode_ci default NULL,
`created` datetime default NULL,
PRIMARY KEY (`id`),
KEY `order_id` (`order_id`),
KEY `npo_id` (`npo_id`),
KEY `session_id` (`session_id`)
) ENGINE=InnoDB AUTO_INCREMENT=23 DEFAULT CHARSET=utf8
I also did some reading about cardinality, and it looks like both the Donations.npo_id and Donations.order_line_id have a cardinality of 2. Hopefully this suggests something useful?
I'm thinking that a USE INDEX might solve the problem, but I'm using an ORM that makes this a bit tricky, and I don't understand why it wouldn't grab the correct index when the JOIN specifically names indexed fields?!?
Thanks for your brainpower!
The first explain has "uses index" at the end. This means that it was able to find the rows and return the result for the query by just looking at the index and not having to fetch/analyse any row data.
In the second query you add a row that's likely not indexed. This means that MySQL has to look at the data of the table. I'm not sure why the optimiser chose to do a table scan but I think it's likely that if the table is fairly small it's easier for it to just read everything than trying to pick out details for individual rows.
edit: I think adding the following indexes will improve things even more and let all of the join use indexes only:
ALTER TABLE order_line ADD INDEX(session_id, id);
ALTER TABLE donations ADD INDEX(order_line_id, npo_id, id)
This will allow order_line to to find the rows using session_id and then return id and also allow donations to join onto order_line_id and then return the other two columns.
Looking at the auto_increment values can I assume that there's not much data in there. It's worth noting that the amount of data in the tables will have an effect on the query plan and it's good practice to put some sample data in there to test things out. For more detail have a look in this blog post I made some time back: http://webmonkeyuk.wordpress.com/2010/09/27/what-makes-a-good-mysql-index-part-2-cardinality/