Add Indexes correctly to a large database table - mysql

The problem is that after I insert 200.000-300.000 rows of data into those columns the search moves very slow and my first thing that came in mind is the indexes that I may have not added correctly. I have tried adding as many as possible BTREE indexes phpmyadmin did not let me to add for all. What would be the correct indexes for my table? I have the following table with the following indexes:
CREATE TABLE IF NOT EXISTS `carads` (
`ADID` int(7) NOT NULL AUTO_INCREMENT,
`LINK` varchar(255) CHARACTER SET latin1 NOT NULL,
`TITLE` varchar(255) NOT NULL,
`MAKE` varchar(50) CHARACTER SET latin1 NOT NULL,
`MODEL` varchar(100) CHARACTER SET latin1 NOT NULL,
`FUEL` varchar(50) CHARACTER SET latin1 NOT NULL,
`LOC` varchar(100) NOT NULL,
`TRANS` varchar(50) NOT NULL,
`YEAR` varchar(4) CHARACTER SET latin1 NOT NULL,
`BODY` varchar(255) CHARACTER SET latin1 NOT NULL,
`DESCRIPT` text CHARACTER SET latin1 NOT NULL,
`PHONENR` varchar(20) NOT NULL,
`MILEAGE` int(11) NOT NULL,
`PRICE` int(20) NOT NULL,
`DISTANCE` int(250) NOT NULL,
`POSTCODE` varchar(250) NOT NULL,
`IMAGE1` varchar(255) NOT NULL,
`IMAGE2` varchar(255) NOT NULL,
`IMAGE3` varchar(255) NOT NULL,
`IMAGE4` varchar(255) NOT NULL,
`IMAGE5` varchar(255) NOT NULL,
`CPHONE` varchar(250) NOT NULL,
`CEMAIL` varchar(500) NOT NULL,
`COLOUR` varchar(250) NOT NULL,
`EQUIPMENT` text NOT NULL,
`STATUS` tinyint(1) NOT NULL DEFAULT '1',
`DATE` date NOT NULL,
`DEL` int(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`ADID`),
KEY `ix_MakeModelPrice` (`STATUS`,`MAKE`(25),`MODEL`(25),`PRICE`),
KEY `ix_Price` (`PRICE`,`STATUS`,`DEL`,`TITLE`(30),`ADID`),
KEY `ix_Date` (`DATE`,`STATUS`,`DEL`,`TITLE`(30),`ADID`),
KEY `LINK` (`LINK`),
FULLTEXT KEY `MODEL` (`MODEL`),
FULLTEXT KEY `SearchIndex` (`TITLE`,`LOC`,`TRANS`,`CPHONE`,`CEMAIL`,`COLOUR`,`EQUIPMENT`),
FULLTEXT KEY `MAKE` (`MAKE`)
)
ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=2478687;

This is very complicated and we cannot give you the correct answer, you have to understand and find the best answer by yourself.
You have to keep following in mind:
The query optimizer will choose only one index.
Indexes which start with something like "status" and or "del" (boolean values or values where 95% of the rows have the selected values) don't add any value, besides these dummy columns are followed with often queried, highly selective values.
You should first find the attributes which are
filled in most of the queries (I could imagine that "make", "price" and "year" are good candidates)
are most selective (meaning that the resulting rows are < 10%)
You have to find out which distribution of values for each of the columns exist in your table. Examples:
Make:
BMW: 5%
Alfa Romeo: 1%
VW: 7%
...
Price-Range:
0..999: 3%
1000..1999: 4%
2000..3000: 5%
...
If 80% of all searches contain "make", "price" and "year", then create an index with all 3 columns. Put the columns which are most selective and/or are mentioned in most searches to the front, followed by the other columns.
With some luck you can improve response time of many searches dramatically. You can then dig deeper into statistics and add some other indexes. Maybe 80% of all searches have a selection for "make", but in the rest there are still many searches without "make", but with focus on "price" and "fuel", then create an index for those searches.
You could as well improve performance when you use "codes" (e.g. Alfa Romeo=1, BMW=2, VW=3, ...) or cluster ranges of values (e.g. price_range: 0..999, 1000..2000, ...). This could help MySQL to build a bit more efficient indexes (smaller leads to less memory footprint and less I/0).
And to understand indexes better, try to submit a query like this (I want that index ix_MakeModelPrice is used):
-- ix_MakeModelPrice: STATUS`,`MAKE`(25),`MODEL`(25),`PRICE`
SELECT * FROM carads
where STATUS=1 AND MAKE='Alfa Romeo'
AND MODEL='159' and PRICE BETWEEN 100 and 1000
order by ADID Desc
LIMIT 0
This query should be fast (hopefully with some matching rows). Do you see why it is fast? "STATUS" is not selective, but the rest should reduce the number of rows found with an index-scan to probably way below 1%. The number of physical reads (rows) is reduced to a minimum => faster response.

Related

MySQL ORDER BY using filesort (2 joined tables)

I have strange problem on query optimization. SQL is generated by ORM-like library and something bad has been detected only after reading megabytes of SQL logs.
SELECT
`ct_pricelistentry`.`uid` as `uid`, `ct_pricelistentry`.`skuGroup` as `skuGroup`
FROM
`ct_pricelistentry` INNER JOIN `lct_set`
ON `lct_set`.`parent_uid`='SET:ALLPRICELISTENTRIES' AND
`lct_set`.`ref_uid`=`ct_pricelistentry`.`uid`
WHERE
(`isGroup` IS FALSE) AND
(`isService` IS FALSE) AND
(`brand` = 'BRAND:5513f43697d637.00632331' OR `brand` IS NULL)
ORDER BY `skuGroup` ASC
EXPLAIN says:
'1', 'SIMPLE', 'ct_pricelistentry', 'ALL', 'PRIMARY', NULL, NULL, NULL, '34591', 'Using where; Using filesort'
'1', 'SIMPLE', 'lct_set', 'eq_ref', 'PRIMARY', 'PRIMARY', '292', 'const,dealers_v2.ct_pricelistentry.uid', '1', 'Using where; Using index'
Note: all needed indexes are presented including skuGroup. But the index skuGroupstill is not listed in EXPLAIN possible_keys. It also cannot be forced by FORCE INDEX (it just disables all indexes).
After some research I found hacky solution but not sure it works as indended:
Add FORCE INDEX (skuGroup),
Add to the WHERE clause dummy AND (skuGroup IS NULL OR skuGroup IS NOT NULL) part.
Following query
SELECT
`ct_pricelistentry`.`uid` as `uid`, `ct_pricelistentry`.`skuGroup` as `skuGroup`
FROM
`ct_pricelistentry` FORCE INDEX (`skuGroup`) INNER JOIN `lct_set`
ON `lct_set`.`parent_uid`='SET:ALLPRICELISTENTRIES' AND
`lct_set`.`ref_uid`=`ct_pricelistentry`.`uid`
WHERE
(`isGroup` IS FALSE) AND
(`isService` IS FALSE) AND
(`brand` = 'BRAND:5513f43697d637.00632331' OR `brand` IS NULL) AND
(`skuGroup` IS NULL OR `skuGroup` IS NOT NULL)
ORDER BY `skuGroup` ASC
gives EXPLAIN without filesort so it seems to use the index to fetch ordered rows:
'1', 'SIMPLE', 'ct_pricelistentry', 'range', 'skuGroup', 'skuGroup', '768', NULL, '16911', 'Using where'
'1', 'SIMPLE', 'lct_set', 'eq_ref', 'PRIMARY', 'PRIMARY', '292', 'const,dealers_v2.ct_pricelistentry.uid', '1', 'Using where; Using
index'
Whats happening at all?
Is it a MySQL bug? I've tested on MySQL 5.1 - 5.5 - the same results. Do you have more predictable/stable solutions?
---- CREATE TABLE ----
CREATE TABLE IF NOT EXISTS `lct_set` (
`parent_uid` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`ref_uid` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`parent_uid`,`ref_uid`),
UNIQUE KEY `BACK_PRIMARY` (`ref_uid`,`parent_uid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `ct_pricelistentry` (
`uid` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`refcount` int(11) NOT NULL,
`isDisposed` tinyint(1) DEFAULT NULL,
`tag` text,
`isGroup` tinyint(1) DEFAULT NULL,
`parentEntry` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`externalUID` varchar(255) DEFAULT NULL,
`productCode` varchar(16) DEFAULT NULL,
`name` varchar(255) DEFAULT NULL,
`sku` varchar(255) DEFAULT NULL,
`skuGroup` varchar(255) DEFAULT NULL,
`measureUnit` varchar(16) DEFAULT NULL,
`image` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`itemClassExternalUID` varchar(255) DEFAULT NULL,
`itemClassName` varchar(255) DEFAULT NULL,
`itemClassDescription` text,
`itemClassComments` text,
`itemClassAttachments` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`brand` varchar(48) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`priceGroups` text,
`productAttributes` text,
`constituents` text,
`position` int(11) DEFAULT NULL,
`isService` tinyint(1) DEFAULT NULL,
`stackability` varchar(255) DEFAULT NULL,
PRIMARY KEY (`uid`),
UNIQUE KEY `test1` (`uid`,`skuGroup`),
KEY `name` (`name`),
KEY `sku` (`sku`),
KEY `itemClassExternalUID` (`itemClassExternalUID`),
KEY `parentEntry` (`parentEntry`),
KEY `position` (`position`),
KEY `externalUID` (`externalUID`),
KEY `productCode` (`productCode`),
KEY `skuGroup` (`skuGroup`),
KEY `brand` (`brand`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The fix Using the INDEX(skuGroup) avoids the filesort, but prevents any useful filtering. Optimizing the filtering is more important than avoiding the filesort.
Remove the FORCE and add this 'composite' index
INDEX(isGroup, isService, brand) -- (in any order)
It should help, but probably won't prevent "using filesort". The OR is the killer.
To prevent the use of filesort for ORDER BY, you need a single (usually composite) index that includes all of the WHERE clause, plus the ORDER BY column(s). In constructing such an index, the only things that can be handled in the WHERE are and'd together '=' clauses. Anything else (such as your OR) prevents the optimization.
Why OR hurts Think of it this way... Suppose there were a long printed list of names sorted by last name + first name. And the query asked for WHERE last = 'Karakulov' ORDER BY first. You would jump to the first Karakulov, and there would be all the first names in order. Now suppose you wanted WHERE (last = 'Karakulov' OR last = 'James') ORDER BY first. You could get all your relatives and all my relatives, but you still need to shuffle them together to do the ORDER BY first. MySQL has one technique for that: filesort (and a tmp table leading up to it.)
As a consolation, filesort's temp table is usually an in-memory MEMORY table, so it is reasonably fast.
A workaround is sometimes to turn OR into UNION. (It probably would not help for your query.)
Some schema critique, and other notes...
The UNIQUE key is useless because the PRIMARY KEY already declares uid to be "unique".
VARCHAR(48) utf8 is a rather clumsily big key. Is it some form of UUID? If so, I have nasty things to say about randomness and charset and size.
Some uids are (48), some are (255); was this deliberate?
Get rid of (skuGroupIS NULL ORskuGroupIS NOT NULL) -- The Optimizer is probably not smart enough to realize that this is always "TRUE" !
FORCE INDEX may work today, but could backfire tomorrow. Get rid of it.
What is the value of innodb_buffer_pool_size? It should be about 70% of available RAM if you have at least 4GB of ram. If you left it at some low default, then you are probably I/O-bound, hence slow.
Please provide SHOW CREATE TABLE lct_set -- something strange is going on in the JOIN.

MySQL indexing optimization

I have following table - 'element'
CREATE TABLE `element` (
`eid` bigint(22) NOT NULL AUTO_INCREMENT,
`tag_name` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`text` text COLLATE utf8_bin,
`depth` tinyint(2) DEFAULT NULL,
`classes` tinytext COLLATE utf8_bin,
`webarchiver_uniqueid` int(11) DEFAULT NULL,
`created` datetime DEFAULT NULL,
`updated` datetime DEFAULT NULL,
`rowstatus` char(1) COLLATE utf8_bin DEFAULT 'A',
PRIMARY KEY (`eid`)
) ENGINE=InnoDB AUTO_INCREMENT=12090 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
Column details and current index details are given above. Almost 90% of queries on this table are like:
select * from element
where tag_name = 'XXX'
and text = 'YYYY'
and depth = 20
and classes = 'ZZZZZ'
and rowstatus = 'A'
What would be the most optimal way to create index on this table? The table has around 60k rows.
Change classes from TINYTEXT to VARCHAR(255) (or some more reasonable size), then have
INDEX(tag_name, depth, classes)
with the columns in any order. I left out rowstatus because it smells like a column that is likely to change. (Anyway, a flag does not add much to an index.)
You can't include TEXT or BLOB columns in an index. And it is not worth it to do a 'prefix' index.
Since a PRIMARY KEY is a UNIQUE key, DROP INDEX eid_UNIQUE.
Is there some reason for picking "binary" / "utf8_bin" for all the character fields?

MySQL indexes creation strategy and inner logic

This question expects a generic answer to the wide problematic of indexes creation on MySQL database.
Let's take this table example :
CREATE TABLE IF NOT EXISTS `article` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`published` tinyint(1) NOT NULL DEFAULT '0',
`author_id` int(11) unsigned NOT NULL,
`modificator_id` int(11) unsigned DEFAULT NULL,
`category_id` int(11) unsigned DEFAULT NULL,
`title` varchar(200) COLLATE utf8_unicode_ci NOT NULL,
`headline` text COLLATE utf8_unicode_ci NOT NULL,
`content` text COLLATE utf8_unicode_ci NOT NULL,
`url_alias` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`priority` mediumint(11) unsigned NOT NULL DEFAULT '50',
`publication_date` datetime NOT NULL,
`creation_date` datetime NOT NULL,
`modification_date` datetime NOT NULL,
PRIMARY KEY (`id`)
);
Over such a sample there is a wide range of queries that could be performed on different criterions :
category_id
published
publication_date
e.g.:
SELECT id FROM article WHERE NOT published AND category_id = '2' ORDER BY publication_date;
On many tables you can see a wide range of state fields (like published here), date fields or reference fields (like author_id or category_id). What strategy should be picked to make indexes ?
Which can be developed under the following points:
Make an index on every fields that can be used in query (either as where argument or order by) even if this can lead to have a lot of indexes per table ?
Also make an index on fields that have only a small set of values like boolean or enum, this just does reduce the scope size of the scan by a n factor (assuming n being the number of inputs and every value homogeneously used) ?
I've read that MySQL prior to 5.0 used only one index per request how do the system picks it ? (by choosing the more restrictive one ?)
How does a OR statement is processed ?
How much does this is going to slow insert ?
Does InnoDB/MyISAM change anything to this problem ?
I know the EXPLAIN statement could be used to know whether a request is optimized or not, but a bit of concrete theoretical stuff would really be more constructive than a purely empirical approach !

MYSQL: Find and delete similar records - Updated with example

I'm trying to dedup a table, where I know there are 'close' (but not exact) rows that need to be removed.
I have a single table, with 22 fields, and uniqueness can be established through comparing 5 of those fields. Of the remaining 17 fields, (including the unique key), there are 3 fields that cause each row to be unique, meaning the dedup proper method will not work.
I was looking at the multi table delete method outlined here: http://blog.krisgielen.be/archives/111 but I can't make sense of the final line of code (AND M1.cd*100+M1.track > M2.cd*100+M2.track) as I am unsure what the cd*100 part achieves...
Can anyone assist me with this? I suspect I could do better exporting the whole thing to python, doing something with it, then re-importing it, but then (1)I'm stuck with knowing how to dedup the string anyway! and (2) I had to break the record into chunks to be able to import it into mysql as it was timing out after 300 seconds so it turned into a whole debarkle to get into mysql in the first place.... (I am very novice at both mysql and python)
The table is a dump of some 40 log files from some testing. The test set for each log is some 20,000 files. The repeating values are either the test conditions, the file name/parameters or the results of the tests.
CREATE SHOW TABLE:
CREATE TABLE `t1` (
`DROID_V` int(1) DEFAULT NULL,
`Sig_V` varchar(7) DEFAULT NULL,
`SPEED` varchar(4) DEFAULT NULL,
`ID` varchar(7) DEFAULT NULL,
`PARENT_ID` varchar(10) DEFAULT NULL,
`URI` varchar(10) DEFAULT NULL,
`FILE_PATH` varchar(68) DEFAULT NULL,
`NAME` varchar(17) DEFAULT NULL,
`METHOD` varchar(10) DEFAULT NULL,
`STATUS` varchar(14) DEFAULT NULL,
`SIZE` int(10) DEFAULT NULL,
`TYPE` varchar(10) DEFAULT NULL,
`EXT` varchar(4) DEFAULT NULL,
`LAST_MODIFIED` varchar(10) DEFAULT NULL,
`EXTENSION_MISMATCH` varchar(32) DEFAULT NULL,
`MD5_HASH` varchar(10) DEFAULT NULL,
`FORMAT_COUNT` varchar(10) DEFAULT NULL,
`PUID` varchar(15) DEFAULT NULL,
`MIME_TYPE` varchar(24) DEFAULT NULL,
`FORMAT_NAME` varchar(10) DEFAULT NULL,
`FORMAT_VERSION` varchar(10) DEFAULT NULL,
`INDEX` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`INDEX`)
) ENGINE=MyISAM AUTO_INCREMENT=960831 DEFAULT CHARSET=utf8
The only unique field is the PriKey, 'index'.
Unique records can be established by looking at DROID_V,Sig_V,SPEED.NAME and PUID
Of the ¬900,000 rows, I have about 10,000 dups that are either a single duplicate of a record, or have upto 6 repetitions of the record.
Row examples: As Is
5;"v37";"slow";"10266";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/7";"image/tiff";"Tagged Ima";"3";"191977"
5;"v37";"slow";"10268";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/8";"image/tiff";"Tagged Ima";"4";"191978"
5;"v37";"slow";"10269";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/9";"image/tiff";"Tagged Ima";"5";"191979"
5;"v37";"slow";"10270";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/10";"image/tiff";"Tagged Ima";"6";"191980"
5;"v37";"slow";"12766";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/7";"image/tiff";"Tagged Ima";"3";"193977"
5;"v37";"slow";"12768";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/8";"image/tiff";"Tagged Ima";"4";"193978"
5;"v37";"slow";"12769";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/9";"image/tiff";"Tagged Ima";"5";"193979"
5;"v37";"slow";"12770";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/10";"image/tiff";"Tagged Ima";"6";"193980"
Row Example: As It should be
5;"v37";"slow";"10266";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/7";"image/tiff";"Tagged Ima";"3";"191977"
5;"v37";"slow";"10268";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/8";"image/tiff";"Tagged Ima";"4";"191978"
5;"v37";"slow";"10269";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/9";"image/tiff";"Tagged Ima";"5";"191979"
5;"v37";"slow";"10270";;"file:";"V1-FL425817.tif";"V1-FL425817.tif";"BINARY_SIG";"MultipleIdenti";"20603284";"FILE";"tif";"2008-11-03";;;;"fmt/10";"image/tiff";"Tagged Ima";"6";"191980"
Please note, you can see from the index column at the end that I have cut out some other rows - I have only idenitified a very small set of repeating rows. Please let me know if you need any more 'noise' from the rest of the DB
Thanks.
I figured out a fix - using the count function, I was using a COUNT(*) that just returned everything in the table, by using a COUNT (distinct NAME) function I am able to weed out the dup rows that fit the dup critera (as set out by the field selection in a WHERE clause)
Example:
SELECT `PUID`,`DROID_V`,`SIG_V`,`SPEED`, COUNT(distinct NAME) as Hit FROM sourcelist, main_small WHERE sourcelist.SourcePUID = 'MyVariableHere' AND main_small.NAME = sourcelist.SourceFileName
GROUP BY `PUID`,`DROID_V`,`SIG_V`,`SPEED` ORDER BY `DROID_V` ASC, `SIG_V` ASC, `SPEED`;

Getting MySQL to use an index/key, 1 column in where and 2 in order by

How do I get MySQL to use a key/index with the following table structure and query?
-- the table
CREATE TABLE `country` (
`id` int(11) NOT NULL auto_increment,
`expiry_date` datetime NOT NULL,
`name` varchar(50) collate utf8_unicode_ci NOT NULL,
`symbol` varchar(5) collate utf8_unicode_ci NOT NULL,
`exchange_rate` decimal(11,5) NOT NULL default '1.00000',
`code` char(3) collate utf8_unicode_ci NOT NULL,
`currency_code` varchar(3) collate utf8_unicode_ci NOT NULL,
`display_order` smallint(6) unsigned NOT NULL default '0',
PRIMARY KEY (`id`),
KEY `code` (`code`),
KEY `currency_code` (`currency_code`),
KEY `display_order` (`expiry_date`,`name`,`display_order`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
-- the query
SELECT `country`.*
FROM `country`
WHERE `country`.`expiry_date` = 0
ORDER BY `country`.`display_order` ASC, `country`.`name` ASC;
I'm trying to get it to use a key because the query with 180 in the result takes 0.0013s and is by far the slowest query on the page (3x longer than the next slowest). From my understanding, the query should use the display_order index/key.
Change it to:
CREATE TABLE `country` (
`id` int(11) NOT NULL auto_increment,
`expiry_date` datetime NOT NULL,
`name` varchar(50) collate utf8_unicode_ci NOT NULL,
`symbol` varchar(5) collate utf8_unicode_ci NOT NULL,
`exchange_rate` decimal(11,5) NOT NULL default '1.00000',
`code` char(3) collate utf8_unicode_ci NOT NULL,
`currency_code` varchar(3) collate utf8_unicode_ci NOT NULL,
`display_order` smallint(6) unsigned NOT NULL default '0',
PRIMARY KEY (`id`),
KEY `code` (`code`),
KEY `currency_code` (`currency_code`),
KEY `expiry` (`expiry_date`,`name`,`display_order`) <<- renamed key for clarity
/* always name compound keys for their left-most parts*/
KEY `name` (`name`) <<-- new key here
KEY `display` (`display_order`) <<--new key here
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
-- the query
SELECT `country`.*
FROM `country`
WHERE `country`.`expiry_date` = 0
ORDER BY `country`.`display_order` ASC, `country`.`name` ASC;
Compound indexes are tricky
MySQL did not use the index on name in the compound index, because name was in the middle and MySQL only uses parts of an index if that part is the left-most part of a compound index.
The same goes for the index on field display order. The compound index that has display_order in it uses that field as it's right-most part, and therefore will not sort.
Solution
Make a separate index for field name,
and a separate index for field display_order.
Sometimes MySQL does not use an index, even if one is available. One circumstance under which this occurs is when the optimizer estimates that using the index would require MySQL to access a very large percentage of the rows in the table. (In this case, a table scan is likely to be much faster because it requires fewer seeks.) However, if such a query uses LIMIT to retrieve only some of the rows, MySQL uses an index anyway, because it can much more quickly find the few rows to return in the result.
Also if a large percentage of rows have the same value for a field (> 40% (IIRC)) then MySQL will not use the index.
See: http://dev.mysql.com/doc/refman/5.1/en/mysql-indexes.html
See: http://dev.mysql.com/doc/refman/5.1/en/index-hints.html
On how to force indexes as per FractalizeR suggestion.
Make sure to time your select after forcing the index
On such a simple query MySQL seems unlikely to be wrong, and your select time of 0.0013 seconds suggests that there are few rows in the table.
Indexes don't work as you'd expect when there are few rows in a table, because of the percentage rule stated above.
Note that in this case forcing the index would not have worked, because you cannot force MySQL to use the rightmost part of a compound index. It just cannot do that.
If you think MySQL chooses indexes unwisely and you are sure of that, use FORCE INDEX index hint: http://dev.mysql.com/doc/refman/5.1/en/index-hints.html
Your query has an ORDER BY on columns {display_order}+{name}, while
your index named "display_order" is in fact defined on columns {expiry_date}+{name}+{display_order}.
The order of columns in the index does matter. You can benefit an index if you need sorting of filtering on columns that are the beginning of the index.
This become obvious if you keep in mind that index are pre-sorted information.
If you want to benefit an index on {display_order}+{name} then you need an index that begins with {display_order}+{name}. For example {display_order}+{name} or {display_order}+{name}+{expiry_date}.
So in order to optimize your query, you have to change your index in the table, or your SORT clause in the query.
last thing you can do is, use "FORCE INDEX" as mentionten by fractalizeR