Index Columns and Order - mysql

Question

If I have a select statement like the statement below, what order and what columns should be included in an index?
SELECT MIN(BenchmarkID),
MIN(BenchmarkDateTime),
Currency1,
Currency2,
BenchmarkType
FROM Benchmark
INNER JOIN MyCurrencyPairs ON Currency1 = Pair1
AND Currency2 = Pair2
WHERE BenchmarkDateTime > IN_BeginningTime
GROUP BY Currency1, Currency2, BenchmarkType;
Items to note:
The Benchmark table will have billions of rows
The MyCurrencyPairs table is a local table that will have less than 10 records
IN_BeginningTime is a input parameter
Columns Currency1 and Currency2 are VARCHARs
Columns BenchmarkID and BenchmarkType are INTs
Column BenchmarkDateTime is a datetime (hopefully that was obvious)
I've created an index with Currency1, Currency2, BenchmarkType, BenchmarkDateTime, and BenchmarkID but I wasn't getting the speed I was wanting. Could I create a better index?
Edit #1: Someone requested the explain results below. Let me know if anything else is needed
Edit #2: Someone requested the DDL (I'm assuming this is the create statement) for the two tables:
(this benchmark table exists in the database)
CREATE TABLE `benchmark` (
`SequenceNumber` INT(11) NOT NULL,
`BenchmarkType` TINYINT(3) UNSIGNED NOT NULL,
`BenchmarkDateTime` DATETIME NOT NULL,
`Identifier` CHAR(6) NOT NULL,
`Currency1` CHAR(3) NULL DEFAULT NULL,
`Currency2` CHAR(3) NULL DEFAULT NULL,
`AvgBMBid` DECIMAL(18,9) NOT NULL,
`AvgBMOffer` DECIMAL(18,9) NOT NULL,
`AvgBMMid` DECIMAL(18,9) NOT NULL,
`MedianBMBid` DECIMAL(18,9) NOT NULL,
`MedianBMOffer` DECIMAL(18,9) NOT NULL,
`OpenBMBid` DECIMAL(18,9) NOT NULL,
`ClosingBMBid` DECIMAL(18,9) NOT NULL,
`ClosingBMOffer` DECIMAL(18,9) NOT NULL,
`ClosingBMMid` DECIMAL(18,9) NOT NULL,
`LowBMBid` DECIMAL(18,9) NOT NULL,
`HighBMOffer` DECIMAL(18,9) NOT NULL,
`BMRange` DECIMAL(18,9) NOT NULL,
`BenchmarkId` INT(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`BenchmarkId`),
INDEX `NextBenchmarkIndex01` (`Currency1`, `Currency2`, `BenchmarkType`),
INDEX `NextBenchmarkIndex02` (`BenchmarkDateTime`, `Currency1`, `Currency2`, `BenchmarkType`, `BenchmarkId`),
INDEX `BenchmarkOptimization` (`BenchmarkType`, `BenchmarkDateTime`, `Currency1`, `Currency2`)
)
(I'm creating the MyCurrencyPairs table in my routine)
CREATE TEMPORARY TABLE MyCurrencyPairs
(
Pair1 VARCHAR(50),
Pair2 VARCHAR(50)
) ENGINE=memory;
CREATE INDEX IDX_MyCurrencyPairs ON MyCurrencyPairs (Pair1, Pair2);

Answer 1

BenchMarkDateTime should be the first column in your index.
The rule is, if you use only a part of a composite index, the used part should be the leading part.
Secondly, the Group By should match an index.
Your performance would be better if some how you can make your query use "=" instead of ">" which is a range check query.

Answer 2

The main problem is that MySQL can't directly use the index to handle the aggregation. This is due to the join with MyCurrencyPairs and the fact that you're asking for MIN(BenchmarkId) while also having the range condition on BenchmarkDateTime. These two need to be eliminated to get a better execution plan.
Let's have a look at the required indexes and the resulting query first:
ALTER TABLE benchmark
ADD KEY `IDX1` (
`Currency1`,
`Currency2`,
`BenchmarkType`,
`BenchmarkDateTime`
),
ADD KEY `IDX2` (
`Currency1`,
`Currency2`,
`BenchmarkType`,
`BenchmarkId`,
`BenchmarkDateTime`
);
SELECT
(
SELECT
BenchmarkId
FROM
benchmark FORCE KEY (IDX2)
WHERE
Currency1 = ob.Currency1 AND
Currency2 = ob.Currency2 AND
BenchmarkType = ob.BenchmarkType
AND BenchmarkDateTime > IN_BeginningTime
ORDER BY
Currency1, Currency2, BenchmarkType, BenchmarkId
LIMIT 1
) AS BenchmarkId
ob.*
FROM
(
SELECT
MIN(BenchmarkDateTime),
Currency1,
Currency2,
BenchmarkType
FROM
benchmark
WHERE
BenchmarkDateTime > IN_BeginningTime
GROUP BY
Currency1, Currency2, BenchmarkType
) AS ob
INNER JOIN
MyCurrencyPairs ON Currency1 = Pair1 AND Currency2 = Pair2;
The first change is that the GROUP BY part happens in its own subquery. This means that it generates all combinations of Currency1, Currency2, BenchmarkType, even those that don't appear in MyCurrencyPairs, but unless there are lots of combinations, the fact that MySQL can now use an index to perform the operation should make this faster. This subquery uses IDX1 without requiring a temporary table or a filesort.
The second change is the isolation of the MIN(BenchmarkId) part into its own subquery. The sorting in that subquery can be handled using IDX2, so no sorting is required here either. The FORCE KEY (IDX2) hint and that even the "fixed-value" columns Currency1, Currency2 and BenchmarkType appear in the ORDER-part is required to make the MySQL optimizer do the right thing. Again, this is a trade-off. If the final result set is large the subqueries might turn out to be a loss, but I presume that there aren't that many rows.
Explaining that query gives the following query plan (uninteresting columns dropped for readability):
+----+--------------------+-----------------+-------+---------+------+---------------------------------------+
| id | select_type | table | type | key_len | rows | Extra |
+----+--------------------+-----------------+-------+---------+------+---------------------------------------+
| 1 | PRIMARY | <derived3> | ALL | NULL | 1809 | |
| 1 | PRIMARY | MyCurrencyPairs | ref | 106 | 2 | Using where |
| 3 | DERIVED | benchmark | range | 17 | 1225 | Using where; Using index for group-by |
| 2 | DEPENDENT SUBQUERY | benchmark | ref | 9 | 520 | Using where; Using index |
+----+--------------------+-----------------+-------+---------+------+---------------------------------------+
We see that all the interesting parts are properly covered by indexes, and we require neither temporary tables nor filesorts.
Timings on my test data show this version to be about 20 times as fast (1.07s vs. 0.05s), but I have only about 1.2million rows in my benchmark table and the data distribution is likely way off, so YMMV.