MySQL Queries with a range on the Join (speed issue) - mysql

Question

I'm trying to run the following query:
SELECT formatted_journeys.*, MAX(speed)
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id
however, it seems to be extremely slow. Here's the explain for that query.
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
| 1 | SIMPLE | formatted_journeys | ALL | vehicle,start_g_log,end_g_log | NULL | NULL | NULL | 824 | 100.00 | Using temporary; Using filesort |
| 1 | SIMPLE | g_log | ref | PRIMARY,Dupes | Dupes | 4 | motrak.formatted_journeys.vehicle | 1985 | 100.00 | Using where |
+----+-------------+--------------------+------+-------------------------------+-------+---------+-----------------------------------+------+----------+---------------------------------+
2 rows in set, 1 warning (0.02 sec)
The tables are as follows:
formatted_journeys:
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| start_g_log | int(11) | YES | MUL | NULL | |
| end_g_log | int(11) | YES | MUL | NULL | |
| start_latitude | decimal(18,12) | YES | | NULL | |
| start_longitude | decimal(18,12) | YES | | NULL | |
| end_latitude | decimal(18,12) | YES | | NULL | |
| end_longitude | decimal(18,12) | YES | | NULL | |
| start_location | text | YES | | NULL | |
| end_location | text | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| start_date | datetime | YES | | NULL | |
| end_date | datetime | YES | | NULL | |
| vehicle | int(11) | YES | MUL | NULL | |
| private | bit(1) | NO | | b'0' | |
+-----------------+------------------+------+-----+---------+----------------+
Indices:
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| formatted_journeys | 0 | PRIMARY | 1 | id | A | 830 | NULL | NULL | | BTREE | |
| formatted_journeys | 1 | vehicle | 1 | vehicle | A | 4 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | start_g_log | 1 | start_g_log | A | 830 | NULL | NULL | YES | BTREE | |
| formatted_journeys | 1 | end_g_log | 1 | end_g_log | A | 830 | NULL | NULL | YES | BTREE | |
+--------------------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
tracker.g_log:
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| g_logid | int(10) unsigned | NO | PRI | NULL | auto_increment |
| imeiid | int(10) unsigned | NO | MUL | NULL | |
| latitude | decimal(18,12) | YES | | NULL | |
| longitude | decimal(18,12) | YES | | NULL | |
| speed | int(4) | YES | MUL | NULL | |
| bearing | int(4) | YES | | NULL | |
| distance | decimal(10,5) | YES | | NULL | |
| eventcode | int(10) | YES | | NULL | |
| status | int(10) | YES | | NULL | |
| date | datetime | YES | | NULL | |
+-----------+------------------+------+-----+---------+----------------+
Indices:
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| g_log | 0 | PRIMARY | 1 | g_logid | A | 31760 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 1 | imeiid | A | 16 | NULL | NULL | | BTREE | |
| g_log | 0 | Dupes | 2 | date | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 0 | Dupes | 3 | eventcode | A | 31760 | NULL | NULL | YES | BTREE | |
| g_log | 1 | speed | 1 | speed | A | 423 | NULL | NULL | YES | BTREE | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Now I know that the filesort isn't a good thing but how would I get rid of it?

Answer 1

You will hardly get rid of Using temporary; Using filesort, but the query might be faster.
First, try to rewrite the query as:
SELECT *
FROM (
SELECT id, MAX(speed) as max_speed
FROM formatted_journeys
JOIN tracker.g_log
ON imeiid = vehicle
AND g_logid BETWEEN start_g_log AND end_g_log
GROUP BY id) as maxspeeds
JOIN formatted_journeys USING (id);
Then, you can try to force the query to use a coverage index, though that is not easy for the query.
First try: add a composite index (vehicle, start_g_log, end_g_log) and look in explain whether it is used (you should see "Using index" there)