MySQL subquery with group by in left join - optimisation - mysql

Question

MySQL seems to be unable to optimise a select with a GROUP BY subquery and ends up in long execution times. There must be a known optimisation for such common scenario.
Let's assume that we're trying to return all orders from the database, with a flag indicating if it is the first order for the customer.
CREATE TABLE orders (order int, customer int, date date);
Retrieving the first orders by customer is superfast.
SELECT customer, min(order) as first_order FROM orders GROUP BY customer;
However, it becomes very slow once we join this with the full order set using a subquery
SELECT order, first_order FROM orders LEFT JOIN (
SELECT customer, min(order) as first_order FROM orders GROUP BY customer
) AS first_orders ON orders.order=first_orders.first_order;
I hope there is a simple trick that we're missing, because otherwise it would be about 1000x faster to do
CREATE TEMPORARY TABLE tmp_first_order AS
SELECT customer, min(order) as first_order FROM orders GROUP BY customer;
CREATE INDEX tmp_boost ON tmp_first_order (first_order)
SELECT order, first_order FROM orders LEFT JOIN tmp_first_order
ON orders.order=tmp_first_order.first_order;
EDIT:
Inspired by #ruakh proposed option 3, there is indeed a less ugly workaround using INNER JOIN and UNION, which has acceptable performance yet does not require temporary tables. However, it is a bit specific to our case and I am wondering if a more generic optimisation exists.
SELECT order, "YES" as first FROM orders INNER JOIN (
SELECT min(order) as first_order FROM orders GROUP BY customer
) AS first_orders_1 ON orders.order=first_orders_1.first_order
UNION
SELECT order, "NO" as first FROM orders INNER JOIN (
SELECT customer, min(order) as first_order FROM orders GROUP BY customer
) AS first_orders_2 ON first_orders_2.customer = orders.customer
AND orders.order > first_orders_2.first_order;

Answer 1

Here are a few things you might try:
Removing customer from the subquery's field-list, since it's not doing anything anyway:
SELECT order,
first_order
FROM orders
LEFT
JOIN ( SELECT MIN(order) AS first_order
FROM orders
GROUP
BY customer
) AS first_orders
ON orders.order = first_orders.first_order
;
Conversely, adding customer to the ON clause, so it actually does something for you:
SELECT order,
first_order
FROM orders
LEFT
JOIN ( SELECT customer,
MIN(order) AS first_order
FROM orders
GROUP
BY customer
) AS first_orders
ON orders.customer = first_orders.customer
AND orders.order = first_orders.first_order
;
Same as previous, but using an INNER JOIN instead of a LEFT JOIN, and converting your original ON clause into a CASE expression:
SELECT order,
CASE WHEN first_order = order THEN first_order END AS first_order
FROM orders
INNER
JOIN ( SELECT customer,
MIN(order) AS first_order
FROM orders
GROUP
BY customer
) AS first_orders
ON orders.customer = first_orders.customer
;
Replacing the whole JOIN approach with an uncorrelated IN-subquery in a CASE expression:
SELECT order,
CASE WHEN order IN
( SELECT MIN(order)
FROM orders
GROUP
BY customer
)
THEN order
END AS first_order
FROM orders
;
Replacing the whole JOIN approach with a correlated EXISTS-subquery in a CASE expression:
SELECT order,
CASE WHEN NOT EXISTS
( SELECT 1
FROM orders AS o2
WHERE o2.customer = o1.customer
AND o2.order < o1.order
)
THEN order
END AS first_order
FROM orders AS o1
;
(It's very likely that some of the above will actually perform worse, but I think they're all worth trying.)

Answer 2

I would expect this to be faster when using a variable instead of the LEFT JOIN:
SELECT
`order`,
If(#previous_customer<>(#previous_customer:=`customer`),
`order`,
NULL
) AS first_order
FROM orders
JOIN ( SELECT #previous_customer := -1 ) x
ORDER BY customer, `order`;
That's what my example on SQL Fiddle returns:
CUSTOMER ORDER FIRST_ORDER
1 1 1
1 2 (null)
1 3 (null)
2 4 4
2 5 (null)
3 6 6
4 7 7