Duplicate entry for key 'PRIMARY' on AUTO_INCREMENT field - mysql

Question

I have a MySQL table with a PRIMARY KEY AUTO_INCREMENT column. I started running an event that runs ALTER TABLE update_log REORGANIZE PARTITION. Suddenly my clients (which INSERT) started getting errors java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '146255393' for key 'PRIMARY'.
I noticed that the duplicate entry was the same as the AUTO_INCREMENT=… value returned by SHOW CREATE TABLE update_log;:
CREATE TABLE `update_log` (
`update_id` bigint(20) NOT NULL AUTO_INCREMENT,
`org_id` int(11) NOT NULL,
`inventory_id` binary(16) NOT NULL,
`item_url` text NOT NULL,
`item_update_count` bigint(20) NOT NULL,
PRIMARY KEY (`update_id`)
) ENGINE=InnoDB AUTO_INCREMENT=146255393 DEFAULT CHARSET=utf8mb4
/*!50100 PARTITION BY RANGE (update_id)
(PARTITION initial VALUES LESS THAN (1000000) ENGINE = InnoDB,
PARTITION `1000000-1999999` VALUES LESS THAN (2000000) ENGINE = InnoDB,
PARTITION `2000000-2999999` VALUES LESS THAN (3000000) ENGINE = InnoDB,
PARTITION `3000000-3999999` VALUES LESS THAN (4000000) ENGINE = InnoDB,
PARTITION `4000000-4999999` VALUES LESS THAN (5000000) ENGINE = InnoDB,
PARTITION `5000000-5999999` VALUES LESS THAN (6000000) ENGINE = InnoDB,
PARTITION `6000000-6999999` VALUES LESS THAN (7000000) ENGINE = InnoDB,
PARTITION `7000000-7999999` VALUES LESS THAN (8000000) ENGINE = InnoDB,
PARTITION `8000000-8999999` VALUES LESS THAN (9000000) ENGINE = InnoDB,
PARTITION `9000000-9999999` VALUES LESS THAN (10000000) ENGINE = InnoDB,
PARTITION `10000000-10999999` VALUES LESS THAN (11000000) ENGINE = InnoDB,
PARTITION `11000000-11999999` VALUES LESS THAN (12000000) ENGINE = InnoDB,
PARTITION `12000000-12999999` VALUES LESS THAN (13000000) ENGINE = InnoDB,
PARTITION `13000000-13999999` VALUES LESS THAN (14000000) ENGINE = InnoDB,
PARTITION `14000000-14999999` VALUES LESS THAN (15000000) ENGINE = InnoDB,
PARTITION `15000000-15999999` VALUES LESS THAN (16000000) ENGINE = InnoDB,
PARTITION `16000000-16999999` VALUES LESS THAN (17000000) ENGINE = InnoDB,
PARTITION `17000000-17999999` VALUES LESS THAN (18000000) ENGINE = InnoDB,
PARTITION `18000000-18999999` VALUES LESS THAN (19000000) ENGINE = InnoDB,
PARTITION `19000000-19999999` VALUES LESS THAN (20000000) ENGINE = InnoDB,
PARTITION `20000000-20999999` VALUES LESS THAN (21000000) ENGINE = InnoDB,
PARTITION `21000000-21999999` VALUES LESS THAN (22000000) ENGINE = InnoDB,
PARTITION `22000000-22999999` VALUES LESS THAN (23000000) ENGINE = InnoDB,
PARTITION `23000000-23999999` VALUES LESS THAN (24000000) ENGINE = InnoDB,
PARTITION `24000000-24999999` VALUES LESS THAN (25000000) ENGINE = InnoDB,
PARTITION `25000000-25999999` VALUES LESS THAN (26000000) ENGINE = InnoDB,
PARTITION `26000000-26999999` VALUES LESS THAN (27000000) ENGINE = InnoDB,
PARTITION `27000000-27999999` VALUES LESS THAN (28000000) ENGINE = InnoDB,
PARTITION `28000000-28999999` VALUES LESS THAN (29000000) ENGINE = InnoDB,
PARTITION `29000000-29999999` VALUES LESS THAN (30000000) ENGINE = InnoDB,
PARTITION `30000000-30999999` VALUES LESS THAN (31000000) ENGINE = InnoDB,
PARTITION `31000000-31999999` VALUES LESS THAN (32000000) ENGINE = InnoDB,
PARTITION future VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */;
So I manually altered the AUTO_INCREMENT value to be one greater than the maximum value in the table: ALTER TABLE update_log AUTO_INCREMENT=146262544;
Then the same error happened again (java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '146255393' for key 'PRIMARY'), and I again tried to increase AUTO_INCREMENT by hand to a much higher value: ALTER TABLE update_log AUTO_INCREMENT=146300000; Then the error happened again (java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '146311957' for key 'PRIMARY').
Edit (to answer question from #RaymondNijland): All the inserting clients get the same error at the same position. The insertions to the table are made using the slick library which compile into:
insert into `update_log` (`org_id`,`inventory_id`,`item_url`,`item_update_count`)
select s37.`org_id`, s38.`inventory_id`, s38.`item_url`, s38.`item_update_count`
from `org` s37, `inventory_version` s38
where (s37.`org_code` = 'myorg') and ((s38.`org_id` = s37.`org_id`) and (s38.`inventory_id` = ?))
By the way, also I noticed in SHOW TABLE STATUS\G that Rows does not match COUNT(*):
mysql> SHOW TABLE STATUS\G
…
*************************** 6. row ***************************
Name: update_log
Engine: InnoDB
Version: 10
Row_format: Dynamic
Rows: 0
Avg_row_length: 0
Data_length: 1425408
Max_data_length: 0
Index_length: 0
Data_free: 158334976
Auto_increment: 146311958
Create_time: 2019-08-23 22:01:31
Update_time: 2019-08-23 22:01:31
Check_time: NULL
Collation: utf8mb4_general_ci
Checksum: NULL
Create_options: partitioned
Comment:
…
mysql> SELECT MAX(update_id), COUNT(*) FROM update_log\G
*************************** 1. row ***************************
MAX(update_id): 146312275
COUNT(*): 255167
What’s going on? Am I doomed to manually increment my auto_increment primary key forever?