mysql sometimes removes and old index when adding a new index? - mysql

Question

TL;DR
We've seen on a virtual machine, that adding a UNIQUE index causes an older non-UNIQUE index to be removed automatically.
I cannot reproduce this on any other machines (yet?), and if I do:
mysql -e 'create db2'
mysqldump db1 | mysql db2
before adding the new index to db1, and then try adding the index on db2 instead, the old index is not removed from db2. Funky!
We have a snapshot of the machine with db1, and can reproduce it there in the existing db1 database...
Anybody have any idea what is going on? We have scripts to automate adding/removing indexes, and that started failing because on that one machine, the old index didn't exist any longer.
When a dump/restore cycle causes the problem to go away, that makes it hard understand, reproduce and reduce the problem to a simple example.
Details
The table in question looked like this before adding the new index (from SHOW CREATE TABLE). Note the monitoredTableRowID KEY:
CREATE TABLE `cfgAttributeInstances` (
`ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
`attributeID` int(10) unsigned DEFAULT NULL,
`nodeID` int(10) unsigned DEFAULT NULL,
`groupID` int(10) unsigned DEFAULT NULL,
`statisticID` int(10) unsigned DEFAULT NULL,
`nodeStatisticID` int(10) unsigned DEFAULT NULL,
`serviceID` int(10) unsigned DEFAULT NULL,
`nodeServiceID` int(10) unsigned DEFAULT NULL,
`nodeComponentID` int(10) unsigned DEFAULT NULL,
`syslogFilterID` int(10) unsigned DEFAULT NULL,
`value` varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY (`ID`),
UNIQUE KEY `nodeID` (`nodeID`,`attributeID`),
UNIQUE KEY `groupID` (`groupID`,`attributeID`),
UNIQUE KEY `statisticID` (`statisticID`,`attributeID`),
UNIQUE KEY `nodeStatisticID` (`nodeStatisticID`,`attributeID`),
UNIQUE KEY `serviceID` (`serviceID`,`attributeID`),
UNIQUE KEY `nodeServiceID` (`nodeServiceID`,`attributeID`),
KEY `attributeID` (`attributeID`),
KEY `monitoredTableRowID` (`nodeComponentID`),
KEY `syslogFilterID` (`syslogFilterID`),
CONSTRAINT `cfgAttributeInstances_ibfk_1` FOREIGN KEY (`attributeID`) REFERENCES `cfgAttributes` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_2` FOREIGN KEY (`nodeID`) REFERENCES `cfgNodes` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_3` FOREIGN KEY (`groupID`) REFERENCES `cfgGroups` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_4` FOREIGN KEY (`statisticID`) REFERENCES `cfgStatistics` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_5` FOREIGN KEY (`nodeStatisticID`) REFERENCES `cfgNodeStatistics` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_6` FOREIGN KEY (`serviceID`) REFERENCES `cfgServices` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_7` FOREIGN KEY (`nodeServiceID`) REFERENCES `cfgNodeServices` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_8` FOREIGN KEY (`nodeComponentID`) REFERENCES `cfgNodeComponents` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_9` FOREIGN KEY (`syslogFilterID`) REFERENCES `cfgSyslogFilters` (`ID`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='CapMon Attribute instances';
I added this index:
ALTER TABLE cfgAttributeInstances ADD
UNIQUE new_nodeComponentID (nodeComponentID, attributeID)
It looked like this after adding that index:
CREATE TABLE `cfgAttributeInstances` (
`ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
`attributeID` int(10) unsigned DEFAULT NULL,
`nodeID` int(10) unsigned DEFAULT NULL,
`groupID` int(10) unsigned DEFAULT NULL,
`statisticID` int(10) unsigned DEFAULT NULL,
`nodeStatisticID` int(10) unsigned DEFAULT NULL,
`serviceID` int(10) unsigned DEFAULT NULL,
`nodeServiceID` int(10) unsigned DEFAULT NULL,
`nodeComponentID` int(10) unsigned DEFAULT NULL,
`syslogFilterID` int(10) unsigned DEFAULT NULL,
`value` varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY (`ID`),
UNIQUE KEY `nodeID` (`nodeID`,`attributeID`),
UNIQUE KEY `groupID` (`groupID`,`attributeID`),
UNIQUE KEY `statisticID` (`statisticID`,`attributeID`),
UNIQUE KEY `nodeStatisticID` (`nodeStatisticID`,`attributeID`),
UNIQUE KEY `serviceID` (`serviceID`,`attributeID`),
UNIQUE KEY `nodeServiceID` (`nodeServiceID`,`attributeID`),
UNIQUE KEY `new_nodeComponentID` (`nodeComponentID`,`attributeID`),
KEY `attributeID` (`attributeID`),
KEY `syslogFilterID` (`syslogFilterID`),
CONSTRAINT `cfgAttributeInstances_ibfk_1` FOREIGN KEY (`attributeID`) REFERENCES `cfgAttributes` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_2` FOREIGN KEY (`nodeID`) REFERENCES `cfgNodes` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_3` FOREIGN KEY (`groupID`) REFERENCES `cfgGroups` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_4` FOREIGN KEY (`statisticID`) REFERENCES `cfgStatistics` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_5` FOREIGN KEY (`nodeStatisticID`) REFERENCES `cfgNodeStatistics` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_6` FOREIGN KEY (`serviceID`) REFERENCES `cfgServices` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_7` FOREIGN KEY (`nodeServiceID`) REFERENCES `cfgNodeServices` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_8` FOREIGN KEY (`nodeComponentID`) REFERENCES `cfgNodeComponents` (`ID`) ON DELETE CASCADE,
CONSTRAINT `cfgAttributeInstances_ibfk_9` FOREIGN KEY (`syslogFilterID`) REFERENCES `cfgSyslogFilters` (`ID`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='CapMon Attribute instances';
But hey! Where did the monitoredTableRowID index go?
Again, I cannot reproduce this. If I do a mysqldump > dump.db/mysql < dump.db cycle, the monitoredTableRowID doesn't disappear...
Any ideas on what is going on?

Answer 1

The composite key with nodeComponentID causes the non-composite key with nodeComponentID to be redundant within the context of the foreign key reference it is used. MySQL automatically drops the redundant implicit index. The behavior is dependant on column index ordering, as the dropped index will always be the first column within the defined composite key. [sic]
The issue(s) are specifically caused by the automatic implicit index created when adding a foreign key CONSTRAINT, and the index(es) being explicitly defined in SHOW CREATE TABLE which is also used by mysqldump.
The index_name value is ignored if there is already an explicitly
defined index on the child table that can support the foreign key.
Otherwise, MySQL implicitly creates a foreign key index... [sic]
MySQL requires indexes on foreign keys and referenced keys so that
foreign key checks can be fast and not require a table scan. In the
referencing table, there must be an index where the foreign key
columns are listed as the first columns in the same order. Such an
index is created on the referencing table automatically if it does not
exist. This index might be silently dropped later, if you create
another index that can be used to enforce the foreign key constraint.
index_name, if given, is used as described previously. [sic]
To reproduce the issue
Implicit Key Example db-fiddle
Note - INDEX FK_BAR_FOO (foo_id) is not explicitly defined.
DROP TABLE IF EXISTS `BAR`;
DROP TABLE IF EXISTS `FOO`;
CREATE TABLE `FOO` (
`id` INT(11) NOT NULL,
PRIMARY KEY (`id`)
)
ENGINE=InnoDB
;
CREATE TABLE `BAR` (
`id` INT(11) NOT NULL,
`foo_id` INT(11) NOT NULL,
`b` INT(11) NOT NULL,
PRIMARY KEY (`id`),
CONSTRAINT `FK_BAR_FOO` FOREIGN KEY (`foo_id`) REFERENCES `FOO` (`id`) ON DELETE CASCADE
)
ENGINE=InnoDB
;
SHOW CREATE TABLE `BAR`;
Result
INDEX FK_BAR_FOO (foo_id) is explicitly defined in SHOW CREATE TABLE, but is implicitly created by MySQL.
CREATE TABLE `BAR` (
`id` INT(11) NOT NULL,
`foo_id` INT(11) NOT NULL,
`b` INT(11) NOT NULL,
PRIMARY KEY (`id`),
INDEX `FK_BAR_FOO` (`foo_id`),
CONSTRAINT `FK_BAR_FOO` FOREIGN KEY (`foo_id`) REFERENCES `FOO` (`id`) ON DELETE CASCADE
)
ENGINE=InnoDB
;
Composite Key Example: db-fiddle
ALTER TABLE `BAR`
ADD UNIQUE INDEX `foo_id_b` (`foo_id`, `b`);
SHOW CREATE TABLE `BAR`;
Result
foo_id index is removed due to being redundant.
Since foo_id is the first column within the composite key, the implicit foo_id index FK_BAR_FOO is removed.
CREATE TABLE `BAR` (
`id` INT(11) NOT NULL,
`foo_id` INT(11) NOT NULL,
`b` INT(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `foo_id_b` (`foo_id`, `b`),
CONSTRAINT `FK_BAR_FOO` FOREIGN KEY (`foo_id`) REFERENCES `FOO` (`id`) ON DELETE CASCADE
)
ENGINE=InnoDB
;
Explicit Key Example: db-fiddle
MySQL ignores the redundant indexes, if the table was created with an explicit index that satisfies the foreign key CONSTRAINT.
DROP TABLE IF EXISTS `BAR`;
DROP TABLE IF EXISTS `FOO`;
CREATE TABLE `FOO` (
`id` INT(11) NOT NULL,
PRIMARY KEY (`id`)
)
ENGINE=InnoDB
;
CREATE TABLE `BAR` (
`id` INT(11) NOT NULL,
`foo_id` INT(11) NOT NULL,
`b` INT(11) NOT NULL,
PRIMARY KEY (`id`),
INDEX `FK_BAR_FOO` (`foo_id`),
CONSTRAINT `FK_BAR_FOO` FOREIGN KEY (`foo_id`) REFERENCES `FOO` (`id`) ON DELETE CASCADE
)
ENGINE=InnoDB
;
ALTER TABLE `BAR`
ADD UNIQUE INDEX `foo_id_b` (`foo_id`, `b`);
SHOW CREATE TABLE `BAR`;
Result
Composite-key index for foo_id, b and index on foo_id are created.
CREATE TABLE `BAR` (
`id` INT(11) NOT NULL,
`foo_id` INT(11) NOT NULL,
`b` INT(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `foo_id_b` (`foo_id`, `b`),
INDEX `FK_BAR_FOO` (`foo_id`),
CONSTRAINT `FK_BAR_FOO` FOREIGN KEY (`foo_id`) REFERENCES `FOO` (`id`) ON DELETE CASCADE
)
ENGINE=InnoDB
;
Solution
In your scripts, verify the index (does not) exist before attempting to drop/add the index.
IF NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_NAME = 'cfgAttributeInstances'
AND INDEX_NAME = 'new_nodeComponentID'
AND INDEX_SCHEMA = 'DbName')
THEN
ALTER TABLE `cfgAttributeInstances`
ADD UNIQUE INDEX `new_nodeComponentID` (`nodeComponentID`,`attributeID`);
END IF;
IF EXISTS (
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_NAME = 'cfgAttributeInstances'
AND INDEX_NAME = 'nodeComponentID'
AND INDEX_SCHEMA = 'DbName')
THEN
ALTER TABLE `cfgAttributeInstances`
DROP INDEX `nodeComponentID`;
END IF;