mysql query using the wrong indexes - mysql

Question

I have some optimization problems with some of my queries in a mysql database. After I build my application I am trying to optimize using mysqltuner and explain, to find non indexed queries. This is a query that is running often and reports that is not using the index :
SELECT count(*) AS rangedandselling
FROM
( SELECT DISTINCT `store_formats`.`Store Name`
FROM (`eds_sales`
JOIN `store_formats`
ON (`eds_sales`.`Store Nbr` = `store_formats`.`Store Nbr`)
)
WHERE `eds_sales`.`Prime Item Nbr` = '4'
AND `eds_sales`.`Date` BETWEEN CAST('2016-07-14' AS DATETIME)
AND CAST('2016-07-21' AS DATETIME)
AND `store_formats`.`Format Name` IN ('format1','format2')
AND `store_formats`.`Store Name` IN (
SELECT DISTINCT `store_formats`.`Store Name`
FROM (`eds_stock`
JOIN `store_formats`
ON (`eds_stock`.`Store Nbr` = `store_formats`.`Store Nbr`)
)
WHERE `eds_stock`.`Prime Item Nbr` = '4'
AND `eds_stock`.`Date` BETWEEN CAST('2016-07-14' AS DATETIME)
AND CAST('2016-07-21' AS DATETIME)
AND `store_formats`.`Format Name` IN ('format1','format2')
AND `eds_stock`.`Curr Traited Store/Item Comb.` = '1' )
) t
This is the explain output : https://tools.mariadb.org/ea/pyb3h
Although I have indexed the columns involved in the joins and lookups, it looks like it is picking another index. this other index is called uniqness, and is composed of 6 different columns in the source columns that I use for inserts (the combination of those columns is the only thing that makes a row unique, hence the name I gave.). I then made sure I have indexes for the other columns and I can see them in the explain. I am not sure why this happens, can someone help?
Any ideas on optimizing this query?
Here is the explain for those that the link above does not work :
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+---+---+---+---+---+---+---+---+---+---+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 167048 | |
| 2 | DERIVED | eds_sales | ref | uniqness,Prime Item Nbr,Store Nbr | uniqness | 4 | const | 23864 | Using where; Using index; Using temporary |
| 2 | DERIVED | store_formats | ref | Store Nbr,Store Name,Format Name | Store Nbr | 5 | equidata.eds_sales.Store Nbr | 1 | Using where |
| 2 | DERIVED | <subquery3> | eq_ref | distinct_key | distinct_key | 84 | func | 1 | Distinct |
| 3 | MATERIALIZED | store_formats | ALL | Store Nbr,Store Name,Format Name | NULL | NULL | NULL | 634 | Using where; Distinct |
| 3 | MATERIALIZED | eds_stock | ref | uniqness,Prime Item Nbr,Store Nbr | uniqness | 8 | const,equidata.store_formats.Store Nbr | 7 | Using where; Distinct |
+---+---+---+---+---+---+---+---+---+---+
I am also posting the related tables structure :
--
-- Table structure for table `eds_sales`
--
CREATE TABLE `eds_sales` (
`id` int(12) NOT NULL,
`Prime Item Nbr` int(12) NOT NULL,
`Prime Item Desc` varchar(255) NOT NULL,
`Prime Size Desc` varchar(255) NOT NULL,
`Variety` varchar(255) NOT NULL,
`WHPK Qty` int(5) NOT NULL,
`SUPPK Qty` int(5) NOT NULL,
`Depot Nbr` int(5) NOT NULL,
`Depot Name` varchar(255) NOT NULL,
`Store Nbr` int(5) NOT NULL,
`Store Name` varchar(255) NOT NULL,
`EPOS Quantity` int(5) NOT NULL,
`EPOS Sales` float(4,2) NOT NULL,
`Date` date NOT NULL,
`Client` varchar(255) NOT NULL,
`Retailer` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
ALTER TABLE `eds_sales`
ADD PRIMARY KEY (`id`),
ADD UNIQUE KEY `uniqness` (`Prime Item Nbr`,`Prime Item Desc`,`Prime Size Desc`,`Variety`,`WHPK Qty`,`SUPPK Qty`,`Depot Nbr`,`Depot Name`,`Store Nbr`,`Store Name`,`Date`,`Client`) USING BTREE,
ADD KEY `Prime Item Nbr` (`Prime Item Nbr`),
ADD KEY `Store Nbr` (`Store Nbr`);
Table structure for table eds_stock
CREATE TABLE `eds_stock` (
`Prime Item Nbr` int(12) NOT NULL,
`Prime Item Desc` varchar(255) NOT NULL,
`Prime Size Desc` varchar(255) NOT NULL,
`Variety` varchar(255) NOT NULL,
`Curr Valid Store/Item Comb.` int(12) NOT NULL,
`Curr Traited Store/Item Comb.` int(12) NOT NULL,
`Store Nbr` int(12) NOT NULL,
`Store Name` varchar(255) NOT NULL,
`Curr Str On Hand Qty` int(12) NOT NULL,
`Curr Str In Transit Qty` int(12) NOT NULL,
`Curr Str On Order Qty` int(12) NOT NULL,
`Curr Str In Depot Qty` int(12) NOT NULL,
`Curr Instock %` int(12) NOT NULL,
`Max Shelf Qty` int(12) NOT NULL,
`On Hand Qty` int(12) NOT NULL,
`Date` date NOT NULL,
`Client` varchar(255) NOT NULL,
`Retailer` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
ALTER TABLE `eds_stock`
ADD UNIQUE KEY `uniqness` (`Prime Item Nbr`,`Store Nbr`,`Date`,`Client`,`Retailer`),
ADD KEY `Prime Item Nbr` (`Prime Item Nbr`),
ADD KEY `Store Nbr` (`Store Nbr`),
ADD KEY `Curr Valid Store/Item Comb.` (`Curr Valid Store/Item Comb.`);
Table structure for table store_formats
CREATE TABLE `store_formats` (
`id` int(12) NOT NULL,
`Store Nbr` int(4) DEFAULT NULL,
`Store Name` varchar(27) DEFAULT NULL,
`City` varchar(19) DEFAULT NULL,
`Post Code` varchar(9) DEFAULT NULL,
`Region #` int(2) DEFAULT NULL,
`Region Name` varchar(10) DEFAULT NULL,
`Distr #` int(3) DEFAULT NULL,
`Dist Name` varchar(26) DEFAULT NULL,
`Square Footage` varchar(7) DEFAULT NULL,
`Format` int(1) DEFAULT NULL,
`Format Name` varchar(23) DEFAULT NULL,
`Store Type` varchar(20) DEFAULT NULL,
`TV Region` varchar(12) DEFAULT NULL,
`Pharmacy` varchar(3) DEFAULT NULL,
`Optician` varchar(3) DEFAULT NULL,
`Home Shopping` varchar(3) DEFAULT NULL,
`Retailer` varchar(15) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
ALTER TABLE `store_formats`
ADD PRIMARY KEY (`id`),
ADD KEY `Store Nbr` (`Store Nbr`),
ADD KEY `Store Name` (`Store Name`),
ADD KEY `Format Name` (`Format Name`);

Answer 1

CAST('2016-07-14' AS DATETIME) -- the CAST is not needed; '2016-07-14' works fine. (Especially since you are comparing against a DATE.)
IN ( SELECT ... ) is inefficient. Change to a JOIN.
On eds_stock, instead of
INDEX(`Prime Item Nbr`)
have these two:
INDEX(`Prime Item Nbr`, `Date`)
INDEX(`Prime Item Nbr`, `Curr Traited Store/Item Comb.`, `Date`)
INT is always a 4-byte number, even if you say int(2). Consider switching to TINYINT UNSIGNED (and other sizes of INT).
float(4,2) -- Do not use (m,n); it causes an extra rounding and my cause undesired truncation. Either use DECIMAL(4,2) (for money), or plain FLOAT.
Bug?? Did you really want 8 days, not just a week in
AND `Date` BETWEEN CAST('2016-07-14' AS DATETIME) AND CAST('2016-07-21' AS DATETIME)
I like this pattern:
AND `Date` >= '2016-07-14'
AND `Date` < '2016-07-14' + INTERVAL 1 WEEK
Instead of two selects
SELECT count(*) AS rangedandselling
FROM ( SELECT DISTINCT `store_formats`.`Store Name` ...
One select will probably work (and be faster):
SELECT COUNT(DISTINCT `store_formats`.`Store Name`) AS rangedandselling ...
Once you have cleaned up most of that, we can get back to your question about 'wrong index', if there is still an issue. (Please start a new Question if you need further help.)