I need to partition my MySQL table using LIST COLUMNS partition. I already have the table with data so I need to alter the table to create the partition.
Here is what I am doing.
Sample schema:
CREATE TABLE mytable (
id bigint(20) primary key auto_increment,
.....
status varchar(10)
);
Alter table script that I am using:
ALTER TABLE mytable PARTITION BY LIST COLUMNS (status)
(
PARTITION p1 VALUES IN (NULL),
PARTITION p2 VALUES IN ('SUCCESS'),
PARTITION p3 VALUES IN ('FAILED')
);
Error: A PRIMARY KEY must include all columns in the table's partitioning function
However, when I just tried to create a table without id column as below:
CREATE TABLE mytable (
status varchar(10)
);
And then try to create the partition using the same above alter script, it worked. So, am I missing anything on the table with id?
SQLFiddle:
Working: http://sqlfiddle.com/#!9/c8a3d1
Not Working: http://sqlfiddle.com/#!9/d06a1
Ok, I think I found the solution. I had to change the primary key. The primary key should include both the id and the status column for partitioning based on the status.
Related
summary
in mysql,
I want add uniqueness to existing column, without drop table, but it spits error because of existing auto-increment / primary key / partition key.
describe
I have table like below. and already have some rows.
CREATE TABLE my_table (
`id` INT NOT NULL AUTO_INCREMENT,
`event_tx_id` varchar(64) NOT NULL,
`amount` int(10),
PRIMARY KEY(id)
)
ENGINE=InnoDB
PARTITION BY KEY(id)
and I changed my mind, I want add constraint to column event_tx_id also unique.
I tried
when I try alter table like this (add unique),
ALTER TABLE my_table MONDIFY COLUMN event_tx_id varchar(64) UNIQUE;
it spits error that
Error Code: 1503. A UNIQUE INDEX must include all columns in the table's partitioning function 0.078 sec
so I tried partition by key, first, then
ALTER TABLE my_table PARTITION BY KEY(id, event_tx_id);
then spits
ALTER TABLE my_table PARTITION BY KEY(id, event_tx_id); Error Code: 1503. A PRIMARY KEY must include all columns in the table's partitioning function
tries above needs changing primary key. so when I try to change primary key (drop -> add, because there isn't modify method)
ALTER TABLE my_table DROP PRIMARY KEY Error Code: 1075. Incorrect table definition; there can be only one auto column and it must be defined as a key
it complains about YOU CANNOT CHANGE(DROP) PRIMARY KEY THAT HAS AUTO-INCREMENT..
is there any way to handle this .. ?
thanx.
Uniqueness constraints do not work on Partitioned tables, except where the partition key is part of the constraint.
Partitioning rarely provides any performance benefit, or any other benefit. What were you hoping to achieve?
Is there a way to create a unique index across tables in a MySQL database?
By unique, I mean,
table A ids=1,2,5,7...
table B ids=3,4,6,8...
I think, it is not possible by directly creating a constraint. But there are some solutions in order for you to get unique IDs.
One suggestion would be is by using TRIGGER. Create a BEFORE INSERT trigger on both tables which checks the ID for existence during INSERT or UPDATE.
Second, is by creating a third table which contains UNIQUE numbers and the other tables: TableA and TableB will then be reference on the third one.
Like JW says, this would probably work well with using a third table. In MySQL, you can use identity fields to make this easier. A very simple example would be using these tables (simple versions of the tables with just names without knowing any further info):
CREATE TABLE a
(
`id` int,
`name` varchar(100),
PRIMARY KEY(`id`)
)
ENGINE = INNODB;
CREATE TABLE b
(
`id` int,
`name` varchar(100),
PRIMARY KEY(`id`)
)
ENGINE = INNODB;
CREATE TABLE c
(
`id` int auto_increment,
`intable` varchar(10) not null,
PRIMARY KEY(`id`)
)
ENGINE = INNODB;
Then, when you want to insert a value on either table, do (a sample inserting 'Joe' into a):
INSERT INTO c (`intable`) VALUES ('a');
INSERT INTO a (`id`, `name`)
SELECT LAST_INSERT_ID() AS id, 'Joe' AS name;
The only reason for the intable entry in c is so you know which table it was created for. Any sort of value to insert into c can be used instead.
I'm using MySQL 5.5, and I have an existing table in production that stores customer transactions. A simplified version of the table is:
CREATE TABLE transactions (
id INT NOT NULL AUTO_INCREMENT,
description CHAR(100),
posted DATE,
PRIMARY KEY (id)
) ENGINE=MyISAM
We are exploring the idea of using partitioning on the transaction date to make reports that use date filtering execute faster. The following attempt fails because of restrictions on primary keys and partitions explained in MySQL Partitioning Keys documentation.
mysql> CREATE TABLE transactions (
-> id INT NOT NULL AUTO_INCREMENT,
-> description CHAR(100),
-> posted DATE,
-> PRIMARY KEY (id)
-> ) ENGINE=MyISAM
-> PARTITION BY HASH(MONTH(posted)) PARTITIONS 12;
ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
A possible workaround is as follows:
CREATE TABLE transactions (
id INT NOT NULL AUTO_INCREMENT,
description CHAR(100),
posted DATE,
PRIMARY KEY (id, posted)
) ENGINE=MyISAM
PARTITION BY HASH(MONTH(posted)) PARTITIONS 12;
Another workaround would be:
CREATE TABLE transactions (
id INT NOT NULL AUTO_INCREMENT,
description CHAR(100),
posted DATE,
KEY (id)
) ENGINE=MyISAM
PARTITION BY HASH(MONTH(posted)) PARTITIONS 12;
In both workarounds the database would not stop the situation of multiple records with the same id, but different posted dates. Is there any way to use partitioning on the posted field and maintain the original unique constraints?
I've been facing this same "problem", and one workaround that I found for that was splitting my table in two, resulting in something like this in your case:
CREATE TABLE transactions (
id INT NOT NULL AUTO_INCREMENT,
description CHAR(100),
PRIMARY KEY (id)
) ENGINE=MyISAM;
And the other:
CREATE TABLE transactions_date (
id INT NOT NULL,
posted DATE
) ENGINE=MyISAM
PARTITION BY HASH(MONTH(posted)) PARTITIONS 12;
The obvious problem is that you have to add some extra logic to the application, like the need to fetch both tables to retrieve all the data when using SELECT statements. You could probably use triggers to help you with the tasks related to INSERT, UPDATE and DELETE.
Just a note: the only functions that can benefit from the use of partition pruning in DATE or DATETIME columns are: YEAR(), TO_DAYS() and TO_SECONDS() (this last one is only available since MySQL 5.5).
I have found a great answer for when inserting a new record, ignore if the data already exists.
1) Create a UNIQUE INDEX on the columns.
2) INSERT IGNORE INTO ...
But my problem is that one of the columns is a VARCHAR(2000)**, and MySQL has a 1000-character limit to indexes.
The columns are: id (int), type (varchar 35), data (varchar 2000)
So is there a way to make sure I'm not adding the same data twice with a single query? Or do I need to do a select first to check for existence and if false, perform the insert?
Thanks.
** This is not design, I'm just moving data around so no chance of making this column smaller.
Given the table design you mentioned:
CREATE TABLE mydb.mytable
(
id int not null auto_increment,
type varchar(35),
data varchar(2000),
primary key (id)
);
Your best chance would be the following:
CREATE TABLE mydb.mytable_new LIKE mydb.mytable;
ALTER TABLE mydb.mytable_new ADD COLUMN data_hash CHAR(40);
ALTER TABLE mydb.mytable_new ADD UNIQUE INDEX (data_hash);
INSERT INTO mydb.mytable_new (id,type,data,data_hash)
SELECT id,type,data,UPPER(SHA(data)) FROM mydb.mytable;
ALTER TABLE mydb.mytable RENAME mydb.mytable_old;
ALTER TABLE mydb.mytable_new RENAME mydb.mytable;
DROP TABLE mydb.mytable_old;
Once you add this new column and index, table should now look like this:
CREATE TABLE mydb.mytable
(
id int not null auto_increment,
type varchar(35),
data varchar(2000),
data_hash char(40),
primary key (id),
unique key data_hash (data_hash)
);
Simply perform your operations as follows:
INSERTs
INSERT INTO mydb.mytable (type,data,data_hash)
VALUES ('somtype','newdata',UPPER(SHA('newdata')));
INSERTs should fail on data_hash is you attempt a duplicate key insertion
SELECTs
SELECT * FROM mydb.mytable
WHERE data_hash = UPPER(SHA('data_I_am_searching_for'));
Give it a Try !!!
Hashes have collisions so unless you don’t care about that, this is not a foolproof solution
I am setting up a table in mysql of engine type merge in mysql and was wondering if i have to have all my tables created previously that i want to merge. For example:
CREATE TABLE t1 (
a INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
message CHAR(20)) ENGINE=MyISAM;
CREATE TABLE t2 (
a INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
message CHAR(20)) ENGINE=MyISAM;
INSERT INTO t1 (message) VALUES ('Testing'),('table'),('t1');
INSERT INTO t2 (message) VALUES ('Testing'),('table'),('t2');
CREATE TABLE total (
a INT NOT NULL AUTO_INCREMENT,
message CHAR(20), INDEX(a))
ENGINE=MERGE UNION=(t1,t2) INSERT_METHOD=LAST;
Now if i have code that automatically created a t3 table i would have to modify the merge table to add this to the union? Would i use an ALTER query for that?
note: i am not using MySQL partitions because i have a mysql version 5.0.
Now if i have code that automatically created a t3 table i would have to modify the merge table to add this to the union? Would i use an ALTER query for that?
From the documentation:
To remap a MERGE table to a different collection of MyISAM tables, you can use one of the following methods:
DROP the MERGE table and re-create it.
Use ALTER TABLE tbl_name UNION=(...) to change the list of underlying tables.
Beginning with MySQL 5.0.60, it is also possible to use ALTER TABLE ... UNION=() (that is, with an empty UNION clause) to remove all of the underlying tables.