MySQL group by query with subselect optimization - mysql

Question

I have the following tables in MySQL:
CREATE TABLE `events` (
`pv_name` varchar(60) COLLATE utf8mb4_unicode_ci NOT NULL,
`time_stamp` bigint(20) unsigned NOT NULL,
`event_type` varchar(40) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin,
`value_type` varchar(40) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`value_count` bigint(20) DEFAULT NULL,
`alarm_status` varchar(40) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`alarm_severity` varchar(40) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`pv_name`,`time_stamp`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=COMPRESSED;
CREATE TEMPORARY TABLE `matching_pv_names` (
`pv_name` varchar(60) NOT NULL,
PRIMARY KEY (`pv_name`)
) ENGINE=Memory DEFAULT CHARSET=latin1;
The matching_pv_names table holds a subset of the unique events.pv_name values.
The following query runs using the 'loose index scan' optimization:
SELECT events.pv_name, MAX(events.time_stamp) AS time_stamp
FROM events
WHERE events.time_stamp <= time_stamp_in
GROUP BY events.pv_name;
Is it possible to improve the time of this query by restricting the events.pv_name values to those in the matching_pv_names table without losing the 'loose index scan' optimization?

Answer 1

Try one of the below queries to limit your output to matching values found in matching_pv_names.
Query 1:
SELECT e.pv_name, MAX(e.time_stamp) AS time_stamp
FROM events e
INNER JOIN matching_pv_names pv ON e.pv_name = pv.pv_name
WHERE e.time_stamp <= time_stamp_in
GROUP BY e.pv_name;
Query 2:
SELECT e.pv_name, MAX(e.time_stamp) AS time_stamp
FROM events e
WHERE e.time_stamp <= time_stamp_in
AND EXISTS ( select 1 from matching_pv_names pv WHERE e.pv_name = pv.pv_name )
GROUP BY e.pv_name;
Let me quote manual here, since I think it applies to your case (bold emphasis mine):
If the WHERE clause contains range predicates (...), a loose index scan looks up the first key of each group
that satisfies the range conditions, and again reads the least
possible number of keys. This is possible under the following
conditions:
The query is over a single table.
Knowing this, I believe Query 1 would not be able to use a loose index scan, but probably second query could do that. If that is still not the case, you could also give a try for third method proposed which uses a derived table.
Query 3:
SELECT e.*
FROM (
SELECT e.pv_name, MAX(e.time_stamp) AS time_stamp
FROM events e
WHERE e.time_stamp <= time_stamp_in
GROUP BY e.pv_name
) e
INNER JOIN matching_pv_names pv ON e.pv_name = pv.pv_name;

Answer 2

Your query is very efficient. You can 'prove' it by doing this:
FLUSH STATUS;
SELECT ...;
SHOW SESSION STATUS LIKE 'Handler%';
Most numbers refer to "rows touched", either in the index or in the data. You will see very low numbers. If the biggest one is about the number of rows returned, that is very good. (I tried a similar query and got about 2x; I don't know why.)
With that few rows touched then either
Outputting the rows will overwhelm the run time. So, who cares about the efficiency; or
You were I/O-bound because of leapfrogging through the index (actually, the table in your case). Run it a second time; it will be fast because of caching.
The only way to speed up leapfrogging is to somehow move the desired rows next to each other. That seems unreasonable for this query.
As for playing games with another table -- Maybe. Will the JOIN significantly decrease the number of events to look at? Then Maybe. Otherwise, I say "a very efficient query is not going to get faster by adding complexity".