As said, I'm doing and practicing a partition data in MySQL.
As I want to clear all the data, the data still there and cannot delete in one time.
This is the partition table that I created.
CREATE TABLE events
(
event_id INT unsigned NOT NULL AUTO_INCREMENT,
inserted_date DATETIME DEFAULT CURRENT_TIMESTAMP,
last_updated_date DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
data JSON NOT NULL,
KEY (event_id)
)
default charset utf8
PARTITION BY RANGE(to_days(inserted_date))
(
PARTITION p20210301 VALUES LESS THAN (to_days('2021-04-01')),
PARTITION p20210401 VALUES LESS THAN (to_days('2021-05-01')),
PARTITION p20210501 VALUES LESS THAN (to_days('2021-06-01')),
PARTITION p20210601 VALUES LESS THAN (to_days('2021-07-01')),
PARTITION p20210701 VALUES LESS THAN (to_days('2021-08-01')),
PARTITION p20210801 VALUES LESS THAN (to_days('2021-09-01')),
PARTITION p20210901 VALUES LESS THAN (to_days('2021-10-01')),
PARTITION p20211001 VALUES LESS THAN (to_days('2021-11-01')),
PARTITION p20211101 VALUES LESS THAN (to_days('2021-12-01')),
PARTITION p20211201 VALUES LESS THAN (to_days('2022-01-01')),
PARTITION p20220101 VALUES LESS THAN (to_days('2022-02-01')),
PARTITION p20220201 VALUES LESS THAN (to_days('2022-03-01')),
PARTITION p20220301 VALUES LESS THAN (to_days('2022-04-01')),
PARTITION p20220401 VALUES LESS THAN (to_days('2022-05-01')),
PARTITION p20220501 VALUES LESS THAN (to_days('2022-06-01')),
PARTITION future VALUES LESS THAN (MAXVALUE)
);
First question is everytime I select all data, different rows of data come out.
SELECT
*
FROM
plover_audit_log.events
Second I can't truncate the table. Everytime I truncate the table, the data still there.
truncate table plover_audit_log.events;
Third even I try to delete the data, it cannot delete completely. The data still there after delete.
delete from plover_audit_log.events;
I'm curious on is my MySQL software problem? Or partition table make this happen?
Because I have a backup table without partition is work well with Truncate, Delete and Select script.
data was running by one procedure, which found in processlist. So first need to kill that pid from processlist. like kill pid;
then execute truncate and check if data is still persist
Partitioned tables require a slightly different instruction to perform a TRUNCATE
To remove data from all partitions
ALTER plover_audit_log.events TRUNCATE PARTITION ALL;
To remove all data from a specific partition
ALTER plover_audit_log.events TRUNCATE PARTITION p20210301;
Reference https://dev.mysql.com/doc/refman/8.0/en/alter-table-partition-operations.html
Related
I have created main partition 20170621 and 24 sub partitions
20170621_0 .. 20170621_23
Now I would like to delete the main partition. But I get an error.
alter table VAL90W02 drop PARTITION `20180621`
#1508 - Cannot remove all partitions, use DROP TABLE instead.
I can´t drop sub-partitions either. So, how do I drop the partition?
(from Comment)
create table mytable (
id int(11) NOT NULL AUTO_INCREMENT,
...,
x_date datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (id, x_date)
) ENGINE = MYISAM
PARTITION BY RANGE (day(x_date))
SUBPARTITION BY HASH (hour(x_date))
( PARTITION 20180621 VALUES LESS THAN (24)
( SUBPARTITION 20180621_0 ENGINE = MyISAM,
SUBPARTITION 20180621_1 ENGINE = MyISAM, ...)
), ...;
Irritatingly, when deleting the last partition of a partitioned table, you have to use
ALTER TABLE VAL90W02 REMOVE PARTITIONING;
instead.
This is a misleading error thrown by MySQL (I'm using 5.7 Aurora, not sure which versions this affects).
Arguably, it's a failure of MySQL to handle the edge case on the part of the ALTER TABLE DROP PARTITION command.
What's wrong with this MySQL query?
Result is [Err] 1654 - Partition column values of incorrect type
DROP TABLE IF EXISTS part;
CREATE TABLE `part` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`cnt` varchar(255) DEFAULT NULL,
`created` datetime NOT NULL,
PRIMARY KEY (`id`, `created`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
PARTITION BY RANGE COLUMNS (created)(
PARTITION p_2015_01 VALUES LESS THAN ('2015-01-30') ENGINE=InnoDB,
PARTITION p_2015_02 VALUES LESS THAN ('2015-02-30') ENGINE=InnoDB,
PARTITION p_2015_03 VALUES LESS THAN ('2015-03-30') ENGINE=InnoDB,
PARTITION p_catchall VALUES LESS THAN (MAXVALUE) ENGINE=InnoDB
);
If it matters, the version is 5.5
It took quite a long time to see the obvious:
The date '2015-02-30' does not exist.
Presumably it is converted to NULL or something, therefore the message about 'incorrect type'.
Hopefully it helps somebody someday.
If you're using the online-schema-change tool from the Percona toolkit to reorganize partitions, you can also get this error for using single quotes around an alter containing single quotes. Silly, but I just ran into it.
root#host:~# pt-online-schema-change --execute --no-drop-old-table --progress='time,1' --recursion-method=none --alter='REORGANIZE PARTITION pMAXVALUE INTO ( PARTITION p20161206 VALUES LESS THAN ('2016-12-06'),PARTITION pMAXVALUE VALUES LESS THAN (MAXVALUE))' D=somedb,t=sometable
vs
root#host:~# pt-online-schema-change --execute --no-drop-old-table --progress='time,1' --recursion-method=none --alter="REORGANIZE PARTITION pMAXVALUE INTO ( PARTITION p20161206 VALUES LESS THAN ('2016-12-06'),PARTITION pMAXVALUE VALUES LESS THAN (MAXVALUE))" D=somedb,t=sometable
An example of a variant that may lead to a similar error [#HY000 Partition column values of incorrect type]:
create table if not exists tbl (
col varchar(10))
partition by list columns (col)(
partition blablablabla values in ('blablablabla')
);
If length('blablablabla') > to the size specified in the column col (varchar(10)) we will get this error. In my case, increasing the max length of the characters of that specific column solve the issue.
I am using MySQL 5.6 Server. I had created a table with HASH partitiong but some how I am unable to use specific partitions in my query.
Table Structure
CREATE TABLE `testtable` (
`id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`purchased` DATE DEFAULT NULL,
KEY `ìd` (`id`),
KEY `Purchased` (`purchased`)
) ENGINE=INNODB
/*!50100 PARTITION BY RANGE ( YEAR(purchased))
SUBPARTITION BY HASH ( dayofyear(purchased))
SUBPARTITIONS 366
(PARTITION p0 VALUES LESS THAN (2015) ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (2016) ENGINE = InnoDB) */
My Query
EXPLAIN PARTITIONS
SELECT *
FROM testtable
WHERE purchased BETWEEN '2014-12-29' AND '2014-12-31';
Check SQL FIDDLE Page
My EXPLAIN plan tells me that server is using all partitions instead of specific partitions.
How can I write a query so that server scans specific partitions?
And also want to know what is the problem with my current query and why it is not working?
Thanks in advance...
True. HASH partitioning is essentially useless.
Other things to note...
Having more than about 50 partitions leads to certain inefficiencies.
If you will be purging "old" rows, then consider BY RANGE and have a month in each partition. Then do the purging via DROP PARTITION. More details, including sample code: http://mysql.rjweb.org/doc.php/partitionmaint
I've a 30M rows table and I want to partition it by dates.
mysql > SHOW CREATE TABLE `parameters`
CREATE TABLE `parameters` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`add_time` datetime DEFAULT NULL,
...(etc)
) ENGINE=MyISAM AUTO_INCREMENT=28929477 DEFAULT CHARSET=utf8
Table stores data for last 5 years and rows count increases dramatically. I want partition it by years(2009, 2010, 2011, 2012, 2013).
ALTER TABLE parameters DROP PRIMARY KEY, ADD INDEX(id);
ALTER TABLE parameters PARTITION BY RANGE (TO_DAYS(id)) (
PARTITION y2009 VALUES LESS THAN (TO_DAYS('2010-01-01')),
PARTITION y2010 VALUES LESS THAN (TO_DAYS('2011-01-01')),
PARTITION y2011 VALUES LESS THAN (TO_DAYS('2012-03-01')),
PARTITION y2012 VALUES LESS THAN (TO_DAYS('2013-01-01')),
PARTITION y2013 VALUES LESS THAN MAXVALUE
);
Everyting works on dev-server, but there is a problem on production-server.
The problem: almost all of the rows moved to the first partition(y2009). But data is uniformly distributed by years. Physically there is large y2009.myd file in DATA folder and others partitions have much less size.
Also I tried to reorganize first partition in order to exclude Null dates:
alter table raw
reorganize partition y2012 into (
PARTITION y0 VALUES LESS THAN (0),
PARTITION y2012 VALUES LESS THAN (TO_DAYS('2013-01-01')),
);
P.S.: production and dev servers have same version of MySQL 5.1.37
You need to use date column in RANGE not id for partition.
I have changed TO_DAYS(id) to TO_DAYS(add_time)
Try below:
ALTER TABLE parameters PARTITION BY RANGE (TO_DAYS(add_time)) (
PARTITION y0 VALUES LESS THAN (TO_DAYS('2009-01-01')),
PARTITION y2009 VALUES LESS THAN (TO_DAYS('2010-01-01')),
PARTITION y2010 VALUES LESS THAN (TO_DAYS('2011-01-01')),
PARTITION y2011 VALUES LESS THAN (TO_DAYS('2012-03-01')),
PARTITION y2012 VALUES LESS THAN (TO_DAYS('2013-01-01')),
PARTITION y2013 VALUES LESS THAN MAXVALUE
);
I have a log table that gets processed every night. Processing will be done on data that was logged yesterday. Once the processing is complete I want to delete the data for that day. At the same time, I have new data coming into the table for the current day. I partitioned the table based on day of week. My hope was that I could delete data and insert data at the same time without contention. There could be as many as 3 million rows of data a day being processed. I have searched for information but haven't found anything to confirm my assumption.
I don't want to have the hassles of writing a job that adds partitions and drop partitions as I have seen in other examples. I was hoping to implement a solution using seven partions. eg.
CREATE TABLE `professional_scoring_log` (
`professional_id` int(11) NOT NULL,
`score_date` date NOT NULL,
`scoring_category_attribute_id` int(11) NOT NULL,
`displayable_score` decimal(7,3) NOT NULL,
`created_at` datetime NOT NULL,
PRIMARY KEY (`professional_id`,`score_date`,`scoring_category_attribute_id`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8
/*!50100 PARTITION BY RANGE (DAYOFWEEK(`score_date`))
(PARTITION Sun VALUES LESS THAN (2) ENGINE = InnoDB,
PARTITION Mon VALUES LESS THAN (3) ENGINE = InnoDB,
PARTITION Tue VALUES LESS THAN (4) ENGINE = InnoDB,
PARTITION Wed VALUES LESS THAN (5) ENGINE = InnoDB,
PARTITION Thu VALUES LESS THAN (6) ENGINE = InnoDB,
PARTITION Fri VALUES LESS THAN (7) ENGINE = InnoDB,
PARTITION Sat VALUES LESS THAN (8) ENGINE = InnoDB) */
When my job that processes yesterday's data is complete, it would delete all records where score_date = current_date-1. At any one time, I am likely only going to have data in one or two partitions, depending on time of day.
Are there any holes in my assumptions?
Charlie, I don't see any holes in your logic/assumptions.
I guess my one comment would be why not use the drop/add partition syntax? It has to be more efficient than DELETE FROM .. Where ..; and it's just two calls - no big deal -- store "prototype" statements and substitute for "Sun" and "2" as required for each day of the week -- I often use sprintf for doing just that
ALTER TABLE `professional_scoring_log` DROP PARTITION Sun;
ALTER TABLE `professional_scoring_log` ADD PARTITION (
PARTITION Sun VALUES LESS THAN (2)
);