partition mysql table on a none primary key column - mysql

I have a table:
+----+---------+----------+
| id | user_id | comment |
+----+---------+----------+
Where column type is:
id (bigint not null primary key autoincrement)
user_id (bigint not null)
comment (text)
How can I partition this table on user_id by range? I tried to partition this table by range in PHPMyAdmin but doesn't allow me because user_id isn't a primary key. If I have many 10 billion users and each has an infinite amount of comments this table will be very large. I want to partition it like:
partition 1 (user_id<500)
+----+---------+----------+
| id | user_id | comment |
+----+---------+----------+
partition 2 (user_id<1000)
+----+---------+----------+
| id | user_id | comment |
+----+---------+----------+
And so on.

Ensure you have satisfied the criteria of when to use partitioning. This is a rather rare case and needs to map closely to your queries. A 500 user range seems tiny. MySQL can handle large tables without partitioning so don't assume its necessary.
The form is:
CREATE TABLE tbl (
id bigint unsigned AUTO_INCREMENT NOT NULL,
user_id bigint unsigned NOT NULL,
COMMENT TEXT NOT NULL,
PRIMARY KEY (user_id, id),
key(id))
PARTITION BY RANGE (user_id) (
PARTITION p0 VALUES LESS THAN (500),
PARTITION p1 VALUES LESS THAN (1000),
PARTITION p2 VALUES LESS THAN (2000),
PARTITION p3 VALUES LESS THAN (3000)
);
ref: fiddle

Yes, since user_id is not part of the table primary key(s) or unique keys you can't create partitions solely for the user_id on your table as the DOCs states very clearly
every unique key on the table must use every column in the table's partitioning expression
So for your case what you can do is to add a unique key on your table for both id and user_id
alter table myTable add unique key uk_id_userid (id, user_id);
And then add the range partition for only user_id column as such:
alter table myTable partition by range (user_id) (
PARTITION p0 VALUES LESS THAN (10),
PARTITION p1 VALUES LESS THAN (20),
PARTITION p2 VALUES LESS THAN (30),
PARTITION p3 VALUES LESS THAN (40)
);
Note Since you already have a table with values in order to define your partition ranges you need to wrap around all existing values for your user_id column in your partitions. That is if you have a user_id of 1000 you can not define your last partition as PARTITION p3 VALUES LESS THAN (1000) that will fail. You will need one more partition i.e.: PARTITION p3 VALUES LESS THAN (2000) or PARTITION p3 VALUES LESS THAN MAXVALUE
See it working here: http://sqlfiddle.com/#!9/8ca7ed
Full working example:
create table myTable (
id bigint not null auto_increment,
user_id bigint not null,
comment text,
key (id)
) engine=InnoDb;
insert into myTable (user_id, comment) values
(1, 'bla'), (1, 'ble'), (1, 'bli'), (1, 'blo'),
(12, 'bla'), (12, 'ble'), (12, 'bli'), (12, 'blo'),
(23, 'bla'), (23, 'ble'), (23, 'bli'), (23, 'blo'),
(34, 'bla'), (34, 'ble'), (34, 'bli'), (34, 'blo');
alter table myTable add unique key uk_id_userid (id, user_id);
alter table myTable partition by range (user_id) (
PARTITION p0 VALUES LESS THAN (10),
PARTITION p1 VALUES LESS THAN (20),
PARTITION p2 VALUES LESS THAN (30),
PARTITION p3 VALUES LESS THAN (40)
);

Related

Moving MANY partitions from one table to another -Mysql

I have a table with many partitions by date
I want to move some of the oldest partitions to another table.
I succeed moving the oldest partition by following the manual
but when I try to move more partitions I get: Error Code: 1737. Found a row that does not match the partition
So, I deleted the oldest partition and move the next partition -but then the rows from the first partitions returned to the original table (I did NOT see any documentation for record that go back...)
How can I move the three first partitions to another table?
THANKS
CREATE TABLE `TestPartA` (
`Name` VARCHAR(50) NOT NULL,
`Time` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
`Slot` INT(11) NOT NULL DEFAULT '-1',
`Text` VARCHAR(50) NOT NULL,
PRIMARY KEY (`Name`, `Text`, `Time`),
INDEX `ClusterTimeIdx` (`Name`, `Time`, `Slot`),
INDEX `Time` (`Time`)
)
PARTITION BY RANGE (TO_DAYS(TIME))
(PARTITION p20190407 VALUES LESS THAN (TO_DAYS('2019-04-07')) ,
PARTITION p20190421 VALUES LESS THAN (TO_DAYS('2019-04-21')) ,
PARTITION p20190428 VALUES LESS THAN (TO_DAYS('2019-04-28')),
PARTITION p20190505 VALUES LESS THAN (TO_DAYS('2019-05-05'))) ;
CREATE TABLE TestPartB LIKE TestPartA;
ALTER TABLE TestPartB REMOVE PARTITIONING;
insert into TestPartA values ('A','2019-04-02',1,'W1');
insert into TestPartA values ('A','2019-04-04',1,'W1');
insert into TestPartA values ('A','2019-04-08',1,'W1');
insert into TestPartA values ('A','2019-04-20',1,'W1');
insert into TestPartA values ('A','2019-05-01',1,'W1');
SELECT PARTITION_NAME, TABLE_ROWS FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_NAME = 'TestPartA';
-- move the first partition
ALTER TABLE TestPartA EXCHANGE PARTITION p20190407 WITH TABLE TestPartB; -- Works GREAT
select * from TestPartA;
select * from TestPartB;
SELECT PARTITION_NAME, TABLE_ROWS FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_NAME = 'TestPartA'; -- this is not working any more - but according to documentation it happens sometimes
--move the second partition
ALTER TABLE TestPartA EXCHANGE PARTITION p20190421 WITH TABLE TestPartB; -- FAILED
ALTER TABLE TestPartA drop PARTITION p20190407;
ALTER TABLE TestPartA EXCHANGE PARTITION p20190421 WITH TABLE TestPartB; -- Succeed but the rows from the first partitions returned to table A
select * from TestPartA;
select * from TestPartB;

How to create partition mysql 8

I am trying to create a table with partition with following way
create table transaction_demo
(
trx_id bigint not null primary key,
created_datetime datetime
) partition by range (YEAR(created_datetime)) (
partition p1 values less than (2019),
partition p2 values less than (2020),
partition p3 values less than (2021)
);
But it is not working. I am getting following error
[2019-08-02 15:08:43] [HY000][1503] A PRIMARY KEY must include all columns in the table's partitioning function
[2019-08-02 15:08:43] [HY000][1503] A PRIMARY KEY must include all columns in the table's partitioning function
How to fix this?
Then define primary key for each column which will be part of the partition.
CREATE TABLE transaction_demo (
trx_id int NOT NULL,
created_datetime datetime NOT NULL,
other_column_1 int NULL,
other_column_2 varchar(222) NULL,
PRIMARY KEY(trx_id,created_datetime)
)
PARTITION BY RANGE( YEAR(created_datetime) )(
PARTITION p1 VALUES LESS THAN (2019),
PARTITION p2 VALUES LESS THAN (2020),
PARTITION p3 VALUES LESS THAN (2021)
);

Mysql - how to keep unique constraint while partitioning by RANGE (timestamp)?

I have one table, want to partition by RANGE (created_at timestamp), so can delete old data easily (by drop partition).
CREATE TABLE `orders` (
`order_id` NVARCHAR(64) NOT NULL,
`amount` INTEGER NOT NULL,
`created_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
`modified_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
KEY `order_id` (`order_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
ALTER TABLE dropship.orders
PARTITION BY RANGE (UNIX_TIMESTAMP(created_at)) (
PARTITION p0 VALUES LESS THAN ( UNIX_TIMESTAMP('2019-03-01 00:00:00') ),
PARTITION p1 VALUES LESS THAN ( UNIX_TIMESTAMP('2019-04-01 00:00:00') ),
PARTITION p2 VALUES LESS THAN ( UNIX_TIMESTAMP('2019-05-01 00:00:00') ),
PARTITION p3 VALUES LESS THAN ( UNIX_TIMESTAMP('2019-06-01 00:00:00') ),
PARTITION p4 VALUES LESS THAN ( UNIX_TIMESTAMP('2019-07-01 00:00:00') ),
PARTITION p5 VALUES LESS THAN (MAXVALUE)
);
This table only has two usages: get by order_id, or update by order_id.
select * from orders where order_id = '123';
update orders set amount = 10 where order_id = '123';
Due to the limitation of Mysql partitioning, I cannot add an unique key for order_id since will use created_at field for partitioning.
All columns used in the table's partitioning expression must be part of every unique key that the table may have, including any primary key.
Question:
Any way to make order_id unique in this table please?
I have thought about partitioning by order_id, but it's hard to delete old data in that way.
Any suggestion is welcome. (For example may be you have better design for this table).
BEGIN;
SELECT 1 FROM orders WHERE order_id = 234 FOR UPDATE;
if row exists, you have a dup error.
INSERT INTO orders ... order_id = 234;
COMMIT;
But, as Raymond points out, you may as well drop PARTITIONing and make the column the PRIMARY KEY. This would make all the stated operations slightly faster.

How do I create a partition for every day of the current year in mysql table

Actually I want to store data last three month from current date rest of record deleted. data having millions of record
you can use given query.
CREATE TABLE TABLE_NAME1 (
id BIGINT(20) NOT NULL AUTO_INCREMENT,
Ctime DATETIME DEFAULT NULL,
KEY id (id)
) ENGINE=INNODB AUTO_INCREMENT=286802795 DEFAULT CHARSET=utf8
PARTITION BY RANGE( TO_DAYS(Ctime) ) (
PARTITION p1 VALUES LESS THAN (TO_DAYS('2011-04-02')),
PARTITION p2 VALUES LESS THAN (TO_DAYS('2011-04-03')),
PARTITION p3 VALUES LESS THAN (TO_DAYS('2011-04-04')),
PARTITION p4 VALUES LESS THAN (TO_DAYS('2011-04-05')),
PARTITION p5 VALUES LESS THAN (TO_DAYS('2011-04-06')),
PARTITION p6 VALUES LESS THAN (TO_DAYS('2011-04-07'))
);

How to partition MySQL table with two indexes

I have a table game_log with fields id, game_id and several varchar fields.
id is primary key and game_id is non-unique key.
There're two frequent queries:
SELECT * FROM game_log ORDER BY id DESC LIMIT 20
SELECT * FROM game_log WHERE game_id = <value> ORDER BY id DESC
The table is huge (6.1GB and 32M rows). InnoDB. Rows in it are being added randomly (one per query). Also, some games are being deleted.
I need to reduce disk IO and imrpove responsiveness.
Should I use key or range partitioning? If range, then by id or by game_id? Is there any theory?
Use partitioning by range.
If you partition by key, both of your example queries have to touch every partition.
The theory is that partitioning by KEY is like partitioning by hash, in that consecutive values of the primary key are bound to be stored in separate partitions. By querying a range of id values, you spoil the partition pruning.
Demo:
CREATE TABLE `game_log` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`game_id` int(11) NOT NULL DEFAULT '0',
`xyz` varchar(15) DEFAULT NULL,
PRIMARY KEY (`id`,`game_id`)
)
PARTITION BY KEY ()
PARTITIONS 13;
INSERT INTO game_log (game_id) VALUES (1), (2), (3), (4), (5), (6);
EXPLAIN PARTITIONS SELECT * FROM game_log ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12
EXPLAIN PARTITIONS SELECT * FROM game_log WHERE game_id = 4 ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12
Whereas if you partition by range on game_id, you can get partition pruning to help you at least when you query for a specific game_id. But your query for any game_id order by id desc is still bound to touch every partition.
CREATE TABLE `game_log` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`game_id` int(11) NOT NULL DEFAULT '0',
`xyz` varchar(15) DEFAULT NULL,
PRIMARY KEY (`id`,`game_id`)
)
PARTITION BY RANGE (game_id)
(PARTITION p1 VALUES LESS THAN (3),
PARTITION p2 VALUES LESS THAN (6),
PARTITION p3 VALUES LESS THAN MAXVALUE);
INSERT INTO game_log (game_id) VALUES (1), (2), (3), (4), (5), (6);
EXPLAIN PARTITIONS SELECT * FROM game_log ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p1,p2,p3
EXPLAIN PARTITIONS SELECT * FROM game_log WHERE game_id = 4 ORDER BY id DESC LIMIT 3\G
id: 1
select_type: SIMPLE
table: game_log
partitions: p2