How to partition MySQL table with two indexes - mysql

Question

I have a table game_log with fields id, game_id and several varchar fields.
id is primary key and game_id is non-unique key.
There're two frequent queries:
SELECT * FROM game_log ORDER BY id DESC LIMIT 20
SELECT * FROM game_log WHERE game_id = <value> ORDER BY id DESC
The table is huge (6.1GB and 32M rows). InnoDB. Rows in it are being added randomly (one per query). Also, some games are being deleted.
I need to reduce disk IO and imrpove responsiveness.
Should I use key or range partitioning? If range, then by id or by game_id? Is there any theory?

Answer 1

Use partitioning by range.
If you partition by key, both of your example queries have to touch every partition.
The theory is that partitioning by KEY is like partitioning by hash, in that consecutive values of the primary key are bound to be stored in separate partitions. By querying a range of id values, you spoil the partition pruning.
Demo:
CREATE TABLE `game_log` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`game_id` int(11) NOT NULL DEFAULT '0',
`xyz` varchar(15) DEFAULT NULL,
PRIMARY KEY (`id`,`game_id`)
)
PARTITION BY KEY ()
PARTITIONS 13;
INSERT INTO game_log (game_id) VALUES (1), (2), (3), (4), (5), (6);
EXPLAIN PARTITIONS SELECT * FROM game_log ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12
EXPLAIN PARTITIONS SELECT * FROM game_log WHERE game_id = 4 ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12
Whereas if you partition by range on game_id, you can get partition pruning to help you at least when you query for a specific game_id. But your query for any game_id order by id desc is still bound to touch every partition.
CREATE TABLE `game_log` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`game_id` int(11) NOT NULL DEFAULT '0',
`xyz` varchar(15) DEFAULT NULL,
PRIMARY KEY (`id`,`game_id`)
)
PARTITION BY RANGE (game_id)
(PARTITION p1 VALUES LESS THAN (3),
PARTITION p2 VALUES LESS THAN (6),
PARTITION p3 VALUES LESS THAN MAXVALUE);
INSERT INTO game_log (game_id) VALUES (1), (2), (3), (4), (5), (6);
EXPLAIN PARTITIONS SELECT * FROM game_log ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p1,p2,p3
EXPLAIN PARTITIONS SELECT * FROM game_log WHERE game_id = 4 ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p2