MySQL query optimization with compound index and persistent column - mysql

Question

The following query is being run on MariaDB 10.0.28, taking ~17 seconds, I'm looking to speed it up substantially.
select series_id,delivery_date,delivery_he,forecast_date,forecast_he,value
from forecast where forecast_he=8
AND series_id in (12142594,20735627,632287496,1146453088,1206342447,1154376340,2095084238,2445233529,2495523920,2541234725,2904312523,3564421486)
AND delivery_date >= '2016-07-13'
AND delivery_date < '2018-06-27'
and DATEDIFF(delivery_date,forecast_date)=1
The first attempt to speed it up was to create a persistent column as (datediff(delivery_date,forecast_date)), rebuild the index using the persistent column, and modify the query, replacing the datediff calc with forecast_delivery_delta=1
> describe forecast;
+-------------------------+------------------+------+-----+---------+------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------------+------------------+------+-----+---------+------------+
| series_id | int(10) unsigned | NO | PRI | 0 | |
| delivery_date | date | NO | PRI | NULL | |
| delivery_he | int(11) | NO | PRI | NULL | |
| forecast_date | date | NO | PRI | NULL | |
| forecast_he | int(11) | NO | PRI | NULL | |
| value | float | NO | | NULL | |
| forecast_delivery_delta | tinyint(4) | YES | | NULL | PERSISTENT |
+-------------------------+------------------+------+-----+---------+------------+
> show index from forecast;
+----------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| forecast | 0 | PRIMARY | 1 | series_id | A | 35081 | NULL | NULL | | BTREE | | |
| forecast | 0 | PRIMARY | 2 | delivery_date | A | 130472 | NULL | NULL | | BTREE | | |
| forecast | 0 | PRIMARY | 3 | delivery_he | A | 1290223 | NULL | NULL | | BTREE | | |
| forecast | 0 | PRIMARY | 4 | forecast_date | A | 2322401 | NULL | NULL | | BTREE | | |
| forecast | 0 | PRIMARY | 5 | forecast_he | A | 23224016 | NULL | NULL | | BTREE | | |
| forecast | 1 | he_series_delta_date | 1 | forecast_he | A | 29812 | NULL | NULL | | BTREE | | |
| forecast | 1 | he_series_delta_date | 2 | series_id | A | 74198 | NULL | NULL | | BTREE | | |
| forecast | 1 | he_series_delta_date | 3 | delivery_date | A | 774133 | NULL | NULL | | BTREE | | |
+----------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
This seems to have taken ~2 seconds off the runtime, but I'm wondering if there are better ways to speed this up substantially. I looked into adjusting the buffer size but it seemed not to be wildly misconfigured.
>show variables like '%innodb_buffer_pool_size%';
+-------------------------+-----------+
| Variable_name | Value |
+-------------------------+-----------+
| innodb_buffer_pool_size | 134217728 |
+-------------------------+-----------+
Total table size:
+----------+------------+
| Table | Size in MB |
+----------+------------+
| forecast | 1547.00 |
+----------+------------+
EXPLAIN:
+------+-------------+----------+-------+------------------------------+----------------------+---------+------+--------+-----------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+----------+-------+------------------------------+----------------------+---------+------+--------+-----------------------+
| 1 | SIMPLE | forecast | range | PRIMARY,he_series_delta_date | he_series_delta_date | 11 | NULL | 832016 | Using index condition |
+------+-------------+----------+-------+------------------------------+----------------------+---------+------+--------+-----------------------+

Answer 1

If you are going to say
AND forecast_delivery_delta=1
then the optimal index is one starting with the two = columns:
(forecast_he, forecast_delivery_delta, -- in either order
series_id, -- an IN might work ok next
delivery_date) -- finally a range
It is generally useless to put a column (delivery_date) tested via a range anywhere other than last.
But note, that index will not work very well if you say forecast_delivery_delta <= 2. Now it is a "range", and nothing after it in the index will be used for filtering. Still it may be worth it to have a small number of different indexes, just in case you turn = into a range or vice versa.
And increase innodb_buffer_pool_size to be about 70% of RAM (assuming you have over 4GB of RAM).