Sorry, I don't know English, but I need help :(
I'm using partitioning by LIST COLUMNS by ALTER TABLE statement
My table :
table member_list:
id int,
name varchar(255),
company varchar(255),
cell_phone varchar(20)
It's haven't key
I have more than 900.000 records in the current. After inserting, I tried partitioning table by LIST COLUMNS :
alter table member_list
partition by list columns(company)(
partition p1 values in ('Lavasoft','Cakewalk','Lycos'),
partition p2 values in ('Adobe','Vivoo','Apple Systems','Sibelius'),
partition p3 values in ('Finale','Borland','Macromedia','FPT'),
partition p4 values in ('Chami','Yahoo','Google','Altavista')
)
After runned :
#1526 - Table has no partition for value from column_list
MySQL returned me this error, I can not find support from Oracle page. I hope you will help me. Thanks
#1526 - Table has no partition for value from column_list
The error message is telling you that there is a value in your data in one of the columns you have chosen for partitioning that is not accounted for in your defined partitions.
In this case, there is something in the "company" field that cannot be placed into any of the partitions. For instance, on some record, company="Blackberry." MySQL cannot put this row into any of your partitions.
LIST partitioning allow only Integer values. If you want to use columns with varchar partitioning use HASH or KEY PARTITIONS. Besides partition can only be used on columns that have primary or unique attribute.
Related
I'm trying to partition my very large MySQL table called companyScores (60million rows and 50 columns).
Basically, the table features companies (with the column varchar "company_idx" with unique IDs going from 0 to 10,000 companies) and their respective timestamp (with the column "timestamp") and scores "Scores" (with the column "Scores").
I'd like to include around 500 companies into each partition.
Please let me know if the following would do the job?
ALTER TABLE `companyScores`
PARTITION BY RANGE( company_idx ) (
PARTITION p0 VALUES LESS THAN (500),
PARTITION p1 VALUES LESS THAN (1000),
PARTITION p2 VALUES LESS THAN (1500),
PARTITION p3 VALUES LESS THAN (2000),
and so on...
);
Would the above work?
Also, can we easily insert new values into this database once it has been partitioned?
Would the above work?
No. For several reasons.
If company_idx is a varchar, you need to use RANGE COLUMNS. The RANGE partitioning only works on integers. If you try to use RANGE partitioning on a varchar, you get this error:
ERROR 1659 (HY000): Field 'company_idx' is of a not allowed type for this type of partitioning
Assuming you correct that, you have another problem:
Your partition clauses use integer values, not quoted string values. Those are different types, and the partitioning engine won't use them for defining partitions. If you try, you'll this this error:
ERROR 1654 (HY000): Partition column values of incorrect type
Assuming you correct that by quoting the numbers, you have another problem:
You list the partition for 500 before the string 1000, but the string '500' should come after the string '1000' lexically. RANGE or RANGE COLUMNS partitions must be declared in increasing order. If you try to do it in the order you have, you'll get this error:
ERROR 1493 (HY000): VALUES LESS THAN value must be strictly increasing for each partition
Assuming you correct the order, it works, but it might not do what you want:
CREATE TABLE `companyScores` (
`company_idx` varchar(10) NOT NULL,
PRIMARY KEY (`company_idx`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
/*!50500 PARTITION BY RANGE COLUMNS(company_idx)
(PARTITION p1 VALUES LESS THAN ('1000') ENGINE = InnoDB,
PARTITION p2 VALUES LESS THAN ('1500') ENGINE = InnoDB,
PARTITION p3 VALUES LESS THAN ('2000') ENGINE = InnoDB,
PARTITION p0 VALUES LESS THAN ('500') ENGINE = InnoDB) */
Now another question you asked:
Also, can we easily insert new values into this database once it has been partitioned?
If you insert a new value that isn't covered by the partitions you defined, you'll get this error:
mysql> insert into companyScores set company_idx = '700';
ERROR 1526 (HY000): Table has no partition for value from column_list
Why is that? You have a partition for company_idx less than 1000 right?
No. You have a partition for company_idx less than the string '1000'. You tried to insert the string '700', which is lexically greater than '500', as well as all the other partitions. Therefore it's beyond any of the partitions defined.
You could solve all of the above problems if you change your customer_idx to an integer column.
So im trying to migrate an old table to a new while maintaining compatibility, so my first guess would be to just map everything over by from example_A to example_B while inserting an old date.
Anyways, my question is there anything else I should do in order to maintain backward compatibility such using relations?
Thank you!
CREATE TABLE example_A
(
id INT
price NUMERIC
primary key (id)
)
CREATE TABLE example_B
(
id INT
price NUMERIC
date DATE
primary key (id, date)
)
Most queries should work compatibly, they'll just ignore the new column. Here are some potential problems:
Queries that use SELECT *. If the query involves a JOIN with another table that has a date column, the date column in the result will be ambiguous as to whether it refers to this table or the other table.
If you join with a table that has a date column, and refer to that column without a table prefix, it will become a syntax error because of the ambiguous reference.
INSERT statements that don't list specific columns will break, because the number of values will no longer match the number of columns.
You should also specify a default value for the new column, so that INSERT queries that don't fill it in will work.
I'm new to this with partitions. Didn't knew it existed but came aware when I tried to make our new 'url_hash' column unique in a table in our database. And got the error message:
A UNIQUE INDEX must include all columns in the table's partitioning function
This is a database created by another person that I don't know and who are not involved in the project anymore.
I have tried to read mysql documentation and read on forums about Partition. What it is and how it works. Understand the purpose, to "divide" a table in to several "parts" so it becomes faster to retrieve relevant data. A common example is to partition in to years intervals. But most examples shows an manual method. Where you decide for example less than three specific years. For example:
PARTITION BY RANGE ( YEAR(separated) ) (
PARTITION p0 VALUES LESS THAN (1991),
PARTITION p1 VALUES LESS THAN (1996),
PARTITION p2 VALUES LESS THAN (2001),
PARTITION p3 VALUES LESS THAN MAXVALUE
);
But in our table, the partitions are created this way:
PARTITION BY HASH ( `feeditemsID` + YEAR(`feeddate`))
PARTITIONS 3;
What does that mean? How does our partition work?
feeditemsID is the unique ID for every row in our table.
When you use hash partitioning, the partition that contains each record is determined by calculating a hash code from the expression feaditemsID + YEAR(feeddate), and then finding the modulus of this code by the number of partitions. So if the hash code for a row is 123, it calculates 123 % 3, which is 0, so the record goes into partition 0.
This is explained inthe MySQL documentation.
As stated there,
Note
If a table to be partitioned has a UNIQUE key, then any columns supplied as arguments to the HASH user function or to the KEY's column_list must be part of that key.
In your case, the table's primary key needs to be:
PRIMARY KEY (feeditemsID, feeddate)
Assuming feeditemsID is already unique (presumably it's an auto-increment column), adding feeddate to the primary is redundant as far as keeping the data unique is concerned, but it's needed to satisfy the partitioning requirement. Putting feeditemsID first in the composite key will allow it to be used by itself to optimize table lookup.
This requirement is probably because each partition has its own index. When inserting/updating a row and checking for uniqueness, it only checks the index of the partition where that row will be stored. So when it finds the partition using the hash function, it needs to be sure that this partition will uniquely contain the indexed columns.
For more information see
Partitioning Keys, Primary Keys, and Unique Keys
I am creating a table which will store around 100million rows in MySQL 5.6 using InnoDB storage engine. This table will have a foreign key that will link to another table with around 5 million rows.
Current Table Structure:
`pid`: [Foreign key from another table]
`price`: [decimal(9,2)]
`date`: [date field]
and every pid should have only one record for a date
What is the best way to create indexes on this table?
Option #1: Create Primary index on two fields pid and date
Option #2: Add another column id with AUTO_INCREMENT and primary index and create a unique index on column pid and date
Or any other option?
Only select query i will be using on this table is:
SELECT pid,price,date FROM table WHERE pid = 123
Based on what you said (100M; the only query is...; InnoDB; etc):
PRIMARY KEY(pid, date);
and no other indexes
Some notes:
Since it is InnoDB, all the rest of the fields are "clustered" with the PK, so a lookup by pid is acts as if price were part of the PK. Also WHERE pid=123 ORDER BY date would be very efficient.
No need for INDEX(pid, date, price)
Adding an AUTO_INCREMENT gains nothing (except a hint of ordering). If you needed ordering, then an index starting with date might be best.
Extra indexes slow down inserts. Especially UNIQUE ones.
Either method is fine. I prefer having synthetic primary keys (that is, the auto-incremented version with the additional unique index). I find that this is useful for several reasons:
You can have a foreign key relationship to the table.
You have an indicator of the order of insertion.
You can change requirements, so if some pids allows two values per day or only one per week, then the table can support them.
That said, there is additional overhead for such a column. This overhead adds space and a small amount of time when you are accessing the data. You have a pretty large table, so you might want to avoid this additional effort.
I would try with an index that attempts to cover the query, in the hope that MySQL has to access to the index only in order to get the result set.
ALTER TABLE `table` ADD INDEX `pid_date_price` (`pid` , `date`, `price`);
or
ALTER TABLE `table` ADD INDEX `pid_price_date` (`pid` , `price`, `date`);
Choose the first one if you think you may need to select applying conditions over pid and date in the future, or the second one if you think the conditions will be most probable over pid and price.
This way, the index has all the data the query needs (pid, price and date) and its indexing on the right column (pid)
By the way, always use EXPLAIN to see if the query planner will really use the whole index (take a look at the key and keylen outputs)
Trying to implement a partition strategy for a MySQL 5.5 (InnoDB) table and I am not sure my understanding is right or if I need to change the syntax in creating the partition.
Table "Apple" has 10 mill rows...Columns "A" to "H"
PK is columns "A", "B" and "C"
Column "A" is a char column and can identify groups of 2 million rows.
I thought column "A" would be a nice candidate to try and implement a partition around since
I select and delete by this column and could really just truncate the partition when the data is no longer needed.
I issued this command:
ALTER TABLE Apple
PARTITION BY KEY (A);
After looking at the partition info using this command:
SELECT PARTITION_NAME, TABLE_ROWS FROM
INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_NAME = 'Apple';
I see all the data is on partition p0
I am wrong in thinking that MySQL was going to break out the partitions in groups of 2 million automagically?
Did I need to specify the number of partitions in the Alter command?
I was hoping this would create groups of 2 million rows in a partition and then create a new partition as new data comes in with a unique value for column "A".
Sorry if this was too wordy.
Thanks - JeffSpicoli
Yes, you need to specify the number of partitions (I assume the default was to create 1 partition). Partition by KEY uses internal hashing function http://dev.mysql.com/doc/refman/5.1/en/partitioning-key.html , so the partition is not selected based on the value of column, but on hash computed from it. Hashing functions return the same result for same input, so yes, all rows having the same value will be in the same partition.
But maybe you want to partition by RANGE if you want to be able to DROP PARTITION (because if partitioned by KEY, you only know that the rows are spaced evenly in the partitions, but you many different values end up in the same partition).