I have a table employees which i need to partition based on the month of timestamp. I have tried some types buts not working.
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated TIMESTAMP NOT NULL DEFAULT '9999-12-31',
job_code INT,
store_id INT
)
PARTITION BY RANGE ( date_format(separated,'%m') ) (
PARTITION p0 VALUES LESS THAN (01),
PARTITION p1 VALUES LESS THAN (02),
PARTITION p2 VALUES LESS THAN (03),
----
PARTITION p4 VALUES LESS THAN (04)
);
when am creating this table it is showing that this partition function is not allowed
if am using MONTH(seperated) it works for DATE type but it returns invalid default value for seperated
How we do it in our project (may be not the right way, but works for us) - (MySQL 5.1.60)
PARTITION p201 VALUES LESS THAN (TO_DAYS('2012-01-01')),
PARTITION p202 VALUES LESS THAN (TO_DAYS('2012-02-01')),
PARTITION p203 VALUES LESS THAN (TO_DAYS('2012-03-01')),
PARTITION p204 VALUES LESS THAN (TO_DAYS('2012-04-01')),
PARTITION p205 VALUES LESS THAN (TO_DAYS('2012-05-01')),
PARTITION p206 VALUES LESS THAN (TO_DAYS('2012-06-01')),
PARTITION p207 VALUES LESS THAN (TO_DAYS('2012-07-01')),
PARTITION p208 VALUES LESS THAN (TO_DAYS('2012-08-01')),
PARTITION p209 VALUES LESS THAN (TO_DAYS('2012-09-01')),
PARTITION p210 VALUES LESS THAN (TO_DAYS('2012-10-01')),
PARTITION p211 VALUES LESS THAN (TO_DAYS('2012-11-01')),
PARTITION p212 VALUES LESS THAN (TO_DAYS('2012-12-01'))
That is partitioning for Complete year of 2012
Related
I have a large table that I have partitioned by month
CREATE TABLE `log` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`logdate` date NOT NULL,
...
,
PRIMARY KEY (`id`,`logdate`)
) PARTITION BY RANGE (month(`logdate`))
(PARTITION part0 VALUES LESS THAN (2),
PARTITION part1 VALUES LESS THAN (3),
PARTITION part2 VALUES LESS THAN (4),
PARTITION part3 VALUES LESS THAN (5),
PARTITION part4 VALUES LESS THAN (6),
PARTITION part5 VALUES LESS THAN (7),
PARTITION part6 VALUES LESS THAN (8),
PARTITION part7 VALUES LESS THAN (9),
PARTITION part8 VALUES LESS THAN (10),
PARTITION part9 VALUES LESS THAN (11),
PARTITION part10 VALUES LESS THAN (12),
PARTITION part11 VALUES LESS THAN MAXVALUE);
I have inserted 3 months of data and can see that the rows have been put into their respective partitions.
When I query specifying the partition, the correct data is returned and the explain shows that it is selecting from the correct partition
select logdate, sum(total) from log partition(part10)
where logdate between '2020-11-01' and '2020-11-30' group by 1 order by 1 desc;
However when not specifying the partition no partition pruning is occurring for the below.
select logdate, sum(total) from log
where logdate between '2020-11-01' and '2020-11-30' group by 1 order by 1 desc;
select logdate, sum(total) from log
where month(logdate) = 11 group by 1 order by 1 desc;
According to mysql-8 documentation https://dev.mysql.com/doc/refman/8.0/en/partitioning-range.html
Partitioning schemes based on time intervals. If you wish to implement a partitioning scheme based on ranges or intervals of time in MySQL 8.0, you have two options:
Partition the table by RANGE, and for the partitioning expression, employ a function operating on a DATE, TIME, or DATETIME column and returning an integer value - as shown here in my code
Partition the table by RANGE COLUMNS, using a DATE or DATETIME column as the partitioning column.
What am I missing?
According to 24.6.3 Partitioning Limitations Relating to Functions:
In MySQL 8.0, partition pruning is supported for the TO_DAYS(), TO_SECONDS(), YEAR(), and UNIX_TIMESTAMP() functions. See Section 24.4, “Partition Pruning”, for more information.
Since you are partitioning by MONTH(), partition pruning won't work.
I have so far only found the following very unelegant solution to creating a TABLE with a PARTITION on a BINARY column in MySQL (not part of primary key, so I can't use PARTITION BY KEY):
CREATE TABLE IF NOT EXISTS TestPartition (
uuid BINARY(16) NOT NULL
)
PARTITION BY RANGE COLUMNS(uuid) (
PARTITION p0 VALUES LESS THAN (X'10000000000000000000000000000000'),
PARTITION p1 VALUES LESS THAN (X'20000000000000000000000000000000'),
PARTITION p2 VALUES LESS THAN (X'30000000000000000000000000000000'),
PARTITION p3 VALUES LESS THAN (X'40000000000000000000000000000000'),
PARTITION p4 VALUES LESS THAN (X'50000000000000000000000000000000'),
PARTITION p5 VALUES LESS THAN (X'60000000000000000000000000000000'),
PARTITION p6 VALUES LESS THAN (X'70000000000000000000000000000000'),
PARTITION p7 VALUES LESS THAN (X'80000000000000000000000000000000'),
PARTITION p8 VALUES LESS THAN (X'90000000000000000000000000000000'),
PARTITION p9 VALUES LESS THAN (X'A0000000000000000000000000000000'),
PARTITION pA VALUES LESS THAN (X'B0000000000000000000000000000000'),
PARTITION pB VALUES LESS THAN (X'C0000000000000000000000000000000'),
PARTITION pC VALUES LESS THAN (X'D0000000000000000000000000000000'),
PARTITION pD VALUES LESS THAN (X'E0000000000000000000000000000000'),
PARTITION pE VALUES LESS THAN (X'F0000000000000000000000000000000'),
PARTITION pF VALUES LESS THAN (MAXVALUE)
);
Is there a better way to do this? This becomes unpleasant if I want more partitions.
MySQL throwing error while creating partitions on table.
Error Code : 1486
Constant, random or timezone-dependent expressions in (sub)partitioning function are not allowed.
I have tried following query :
alter table test.tbl1
partition by range(unix_timestamp(sys_time))
(
PARTITION p20151001 VALUES LESS THAN (unix_timestamp('2015-10-01')),
PARTITION p20151101 VALUES LESS THAN (unix_timestamp('2015-11-01')),
PARTITION p20151201 VALUES LESS THAN (unix_timestamp('2015-12-01')),
PARTITION p20160101 VALUES LESS THAN (unix_timestamp('2016-01-01')),
PARTITION p20160201 VALUES LESS THAN (unix_timestamp('2016-02-01')),
PARTITION p20160301 VALUES LESS THAN (unix_timestamp('2016-03-01'))
);
How can I round this problem.
Thanks in Advance
Reading here it may be possible that you are using MYSQL 5.1:
https://dev.mysql.com/tech-resources/articles/mysql_55_partitioning.html
Another pain point in MySQL 5.1 is the handling of date columns. You
can't use them directly, but you need to convert such columns using
either YEAR or TO_DAYS
If your column sys_time is a DATETIME, you dont need to specify the timestamp in order to partition it, you just need to do TO_DAYS, since you're not doing it by year:
alter table test.tbl1
partition by range (TO_DAYS(sys_time))
(
PARTITION p20151001 VALUES LESS THAN (TO_DAYS('2015-10-01')),
PARTITION p20151101 VALUES LESS THAN (TO_DAYS('2015-11-01')),
PARTITION p20151201 VALUES LESS THAN (TO_DAYS('2015-12-01')),
PARTITION p20160101 VALUES LESS THAN (TO_DAYS('2016-01-01')),
PARTITION p20160201 VALUES LESS THAN (TO_DAYS('2016-02-01')),
PARTITION p20160301 VALUES LESS THAN (TO_DAYS('2016-03-01'))
);
if sys_time is a TIMESTAMP then you dont need to convert your timestamp to a timestamp, I have taken that out of the range parameter:
alter table test.tbl1
partition by range(sys_time)
(
PARTITION p20151001 VALUES LESS THAN (unix_timestamp('2015-10-01')),
PARTITION p20151101 VALUES LESS THAN (unix_timestamp('2015-11-01')),
PARTITION p20151201 VALUES LESS THAN (unix_timestamp('2015-12-01')),
PARTITION p20160101 VALUES LESS THAN (unix_timestamp('2016-01-01')),
PARTITION p20160201 VALUES LESS THAN (unix_timestamp('2016-02-01')),
PARTITION p20160301 VALUES LESS THAN (unix_timestamp('2016-03-01'))
);
My DB design includes multiple MYISAM tables with measurements collected online,
Each row record contains auto-incremented id, some data and an integer representing unixtime.
I am designing an aging mechanism, and i am interested to use MySQL partitioning to partition each such table based on unixtime dynamically.
Say that i am interested that each partition will represent single month of data, last partition should represent 2 months, if records arrive for the next not represented month, the partition that represented 2 months should be reorganized to represent single month, and new partition should be created representing 2 month (1 taken from the last partition and 1 for future measurements),
Additionally, when a new partition is created i am interested that the oldest partition will be dropped.
What type of partitioning i should use (my unixtime is not a unique key, and how would i use unixtime for partitioning purposes)?
How would i design the partitioning to be fully dynamical based on new records added to the tables?
UPDATE 12.12.12
I have found and interesting link to similar approach to what i have described your-magical-range-partitioning-maintenance-query.
Partitioning does not need to be based solely on a unique key. However if unique key is present, then it should be included in columns used to partition the table on. To partition table on UNIXTIME column do:
ALTER TABLE MyTable
PARTITION BY RANGE COLUMNS (UNIX_TIMESTAMP(datetime_column))
(
PARTITION p01 VALUES LESS THAN (2),
PARTITION p02 VALUES LESS THAN (3),
PARTITION p03 VALUES LESS THAN (4),
PARTITION p04 VALUES LESS THAN (MAXVALUE));
Or you can partition on datetime column stright away in MySQL 5.5+ :
ALTER TABLE MyTable
PARTITION BY RANGE COLUMNS (datetime_column)
(
PARTITION p01 VALUES LESS THAN ('2013-01-01'),
PARTITION p02 VALUES LESS THAN ('2013-02-01'),
PARTITION p03 VALUES LESS THAN ('2013-03-01'),
PARTITION p04 VALUES LESS THAN (MAXVALUE));
Fully automated version (it would keep every month in its own partition, 5 months of data held):
ALTER TABLE MyTable
PARTITION BY RANGE COLUMNS (YEAR(datetime_column)*100 + MONTH(datetime_column))
(
PARTITION p201301 VALUES LESS THAN (201301),
PARTITION p201302 VALUES LESS THAN (201302),
PARTITION p201303 VALUES LESS THAN (201303),
PARTITION p201304 VALUES LESS THAN (201304),
PARTITION p201305 VALUES LESS THAN (201305),
PARTITION p_MAXVALUE VALUES LESS THAN (MAXVALUE));
DECLARE #Min_Part int
DECLARE #Last_Part int
DECLARE #SQL varchar (1000)
If (select count (distinct MONTH(datetime_column)) from MyTable) > 5 THEN
BEGIN
select #Min_Part = (select min(year(datetime_column)*100 + month(datetime_column)) from MyTable),
#Last_Part = (select max(year(datetime_column)*100 + month(datetime_column)) from MyTable)
set #SQL = 'Alter table MyTable REORGANIZE PARTITION p_MAXVALUE (into partition p' +TO_CHAR (#Last_Part) + 'values less than (' + TO_CHAR (#Last_Part) + ')'
call common_schema.eval (#sql)
set #SQL = 'Alter table MyTable DROP PARTITION p' + TO_CHAR (#Min_Part)
call common_schema.eval (#sql)
END
P.S. Apologies if SQL is not exactly correct - cannot parse it right now.
when partitioning tables in MySQL using a list, how can I generate a partition for remaining items?
E.G:
CREATE TABLE tbl
(
ID integer
)
PARTITION BY LIST (ID)
(
PARTITION P1 values in (1),
PARTITION P2 values in (2),
PARTITION P3 values in (3),
PARTITION Pother values in (<all remaining values of ID>)
);
In Oracle, I use values in (default), but that does work in MySQL.
Unlike range partitioning, there is no "catch-all" in list partitioning. To quote the documentation:
Unlike the case with RANGE partitioning, there is no “catch-all” such
as MAXVALUE; all expected values for the partitioning expression
should be covered in PARTITION ... VALUES IN (...) clauses. An INSERT
statement containing an unmatched partitioning column value fails with
an error...
Unfortunately, I believe, you're not able to combine a list and a range partition either. I'm not entirely certain why you would want to use a list partition in this particular instance; wouldn't a range partition work just a well?
CREATE TABLE tbl
(
ID integer
)
PARTITION BY RANGE (ID)
(
PARTITION P1 values less than (2),
PARTITION P2 values less than (3),
PARTITION P3 values less than (4),
PARTITION Pother values less than maxvalue
);
I'm assuming that this is just an example. Partitioning each key in a primary key is a little pointless.