MYSQL indexing with partitioning - mysql

Question

We are currently evaluating mysql for one of our use case related to analytics.
The table schema is some what like this
CREATE TABLE IF NOT EXISTS `analytics`(
`date` DATE,
`dimension1` BIGINT UNSIGNED,
`dimension2` BIGINT UNSIGNED,
`metrics1` BIGINT UNSIGNED,
`metrics2` BIGINT UNSIGNED,
INDEX `baseindex` (`dimension1`,`dt`)
);
Since most query would be around dimension 1 and date we felt that a combined index would be our best case to optimize query lookup
With this table schema in mind an explain query returns the following results
explain
select dimension2,dimension1
from analytics
where dimension1=1123 and dt between '2016-01-01' and '2016-01-30';
The following query returns the
+----+-------------+-----------+------+---------------+-----------+---------+-------------+------+-----------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+------+---------------+-----------+---------+-------------+------+-----------------------+
| 1 | SIMPLE | analytics | ref | baseindex | baseindex | 13 | const,const | 1 | Using index condition |
+----+-------------+-----------+------+---------------+-----------+---------+-------------+------+-----------------------+
This look good so far as we are getting indication that the indexes are being fired up.
However we though if we can optimize this a bit further, since most of our lookups will be for the current month or month based lookup we felt a date partitioning will further improve the performance.
The table was later modified to add partitions by month
ALTER TABLE analytics
PARTITION BY RANGE( TO_DAYS(`dt`))(
PARTITION JAN2016 VALUES LESS THAN (TO_DAYS('2016-02-01')),
PARTITION FEB2016 VALUES LESS THAN (TO_DAYS('2016-03-01')),
PARTITION MAR2016 VALUES LESS THAN (TO_DAYS('2016-04-01')),
PARTITION APR2016 VALUES LESS THAN (TO_DAYS('2016-05-01')),
PARTITION MAY2016 VALUES LESS THAN (TO_DAYS('2016-06-01')),
PARTITION JUN2016 VALUES LESS THAN (TO_DAYS('2016-07-01')),
PARTITION JUL2016 VALUES LESS THAN (TO_DAYS('2016-08-01')),
PARTITION AUG2016 VALUES LESS THAN (TO_DAYS('2016-09-01')),
PARTITION SEPT2016 VALUES LESS THAN (TO_DAYS('2016-10-01')),
PARTITION OCT2016 VALUES LESS THAN (TO_DAYS('2016-11-01')),
PARTITION NOV2016 VALUES LESS THAN (TO_DAYS('2016-12-01')),
PARTITION DEC2016 VALUES LESS THAN (TO_DAYS('2017-01-01'))
);
With the partition in place, the same query now returns the following results
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+-------+---------------+-----------+---------+------+------+-------------+
| 1 | SIMPLE | analytics | range | baseindex | baseindex | 13 | NULL | 1 | Using where |
+----+-------------+-----------+-------+---------------+-----------+---------+------+------+-------------+
Now the "Extra" column show that its switching to where instead of using index condition.
We have not noticed any performance boost or degradation, so curios to know how does adding a partition changes the value inside the extra column

Answer 1

This is too long for a comment.
MySQL partitions both the data and the indexes. So, the result of your query is that the query is accessing a smaller index which refers to fewer data pages.
Why don't you see a performance boost? Well, looking up rows in a smaller index is negligible savings (although there might be some savings for the first query from a cold start because the index has to be loaded into memory).
I am guessing that the data you are looking for is relatively small -- say, the records come from a handful of data pages. Well, fetching a handful of data pages from a partition is pretty much the same thing as fetching a handful of data pages from the full table.
Does this mean that partitioning is useless? Not at all. For one thing, the partitioned data and index is much smaller than the overall table. So, you have a savings in memory on the server side -- and this can be a big win on a busy server.
In general, though, partitions really shine when you have queries that don't fully use indexes. The smaller data sizes in each partition often make such queries more efficient.

Answer 2

Use NOT NULL (wherever appropriate).
Don't use BIGINT (8 bytes) unless you really need huge numbers. Dimension ids can usually fit in SMALLINT UNSIGNED (0..64K, 2 bytes) or MEDIUMINT UNSIGNED. (0..16M, 3 bytes).
Yes, INDEX(dim1, dt) is optimal for that one SELECT.
No, PARTITIONing will not help for that SELECT.
PARTITION BY RANGE(TO_DAYS(..)) is excellent if you intend to delete old data. But there is rarely any other benefit.
Use InnoDB.
Explicitly specify the PRIMARY KEY. It will be important in the discussion below.
When working with huge databases, it is a good idea to "count the disk hits". So, let's analyze your query.
INDEX(dim1, dt) with WHERE dim1 = a AND dt BETWEEN x and y will
If partitioned, prune down to the partition(s) representing x..y.
Drill down in the Index's BTree to [a,x]. With partitioning the BTree might be 1 level shallower, but that savings is lost to the pruning of step 1.
Scan forward until [a,y]. If only one partition is involved, this scan hits exactly the same number of blocks whether partitioned or not. If multiple partitions are needed, then there is some extra overhead.
For each row, use the PRIMARY KEY to reach over into the data to get dim2. Again, virtually the same amount of effort. Without the Engine and PRIMARY KEY, I cannot finish discussion this #4.
If (dim1, dim2, dt) is unique, make it the PK. In this case, INDEX(dim1, dt) is actually dim1, dt, dim2 since the PK is included in every secondary index. That says that #4 really involves a 'covering' index. That is, the no extra work to reach for dim2 (zero disk hits).
If, on the other hand, you did SELECT metric..., then #4 does have the effort mentioned.