Mysql optimized query and index for exclusion - mysql

Question

Mysql optimized query and index with exclusion
In the case of a select on a high volume table with a select criteria excluding results, what are the possible alternatives?
for example with the following table:
+----+---+---+----+----+
| id | A | B | C | D |
+----+---+---+----+----+
| 1 | a | b | c | d |
| 2 | a | b | c | d |
| 3 | a | b | c1 | d1 |
| 4 | a | b | c2 | d |
| 5 | a | b | c | d2 |
| 6 | a | b | c | d2 |
+----+---+---+----+----+
I would like to select all the tuples (C,D) where A=a and B=b and (C!=c or D!=d)
SELECT C,D FROM my_table WHERE A=a AND B=b AND (C!=c OR D!=d) GROUP BY C,D;
expected result:
(c1,d1)
(c2,d)
(c,d2)
I tried to add an index like that: CREATE INDEX idx_my_index ON my_table(A, B, C, D); but response times are still very long
NB: I'm using MariadDB 10.3
The explain:
+----+-------------+-----------+-------+----------------+---------------+---------+-------------+-----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+-------+----------------+---------------+---------+-------------+-----------+--------------------------+
| 1 | SIMPLE | my_table | ref | idx_my_index | idx_my_index | 6 | const,const | 12055772 | Using where; Using index |
+----+-------------+-----------+-------+----------------+---------------+---------+-------------+-----------+--------------------------+
Is there some improvement to add on my index, on mariadb config or another select to do that?
Specific solution: If we use this query as a subquery we can use the FirstMatch strategy to avoid the full scan of the table. this is described into https://mariadb.com/kb/en/firstmatch-strategy/
SELECT * FROM my__second_table tbis
WHERE (tbis.C, tbis.D)
IN (SELECT C,D FROM my_table WHERE A=a AND B=b AND (C!=c OR D!=d));

Answer 1

Your index is optimal. Discussion:
INDEX(A, B, -- see Note 1
C, D) -- see note 2
Note 1: A,B can be in either order. These will be used for filtering on = to find possible rows. Confirmed by "const,const".
Note 2: C,D can be in either order. != does not work well for filtering, hence these come after A and B. They are included here to make the index "covering". Confirmed by "Using index".
"response times are still very long" -- 12M rows in the table? How many rows before the GROUP BY? How many rows in the result? (These might give us clues into where to go next.)
"Alternative". Probably SELECT DISTINCT ... instead of SELECT ... GROUP BY ... would run at the same speed. (But you could try it. Also, the EXPLAIN might be the same`; the result should be the same.)
Please provide SHOW CREATE TABLE; it might give some more clues, such as NULL/NOT NULL and Engine type. (I don't hold out much hope here.)
Please provide EXPLAIN FORMAT=JSON SELECGT ... -- This might give some more insight. Also: Turn on the Optimizer Trace.