I'm running the following query... the issue I'm running into is that it saves the id field properly with rows 0-27 but then it tries to duplicate row 0. Any thoughts on why this is trying to duplicate row 0?
INSERT INTO
deal (`id`, `site_id`)
VALUES (NULL, 1)
ON DUPLICATE KEY UPDATE `id` = NULL,`site_id` = 1
From the manual:
You can also explicitly assign NULL or 0 to the column to generate sequence numbers.
So, effectivly, entering (0,'whatever') is seen as ('the next autoincrement id','whatever'). No duplicate key issues as far as MySQL is concerned. You can bypass this by using NO_AUTO_VALUE_ON_ZERO. Note that using 0 as an identifier is not recommended practice.
A PRIMARY KEY can't be NULL in MySQL, so your UPDATE id = NULL is casted to UPDATE id = 0.
See also: http://www.xaprb.com/blog/2009/09/12/the-difference-between-a-unique-index-and-primary-key-in-mysql/
Related
I have a table with schema like this:
CREATE TABLE `things` (
`thing_id` INT(11) NOT NULL AUTO_INCREMENT,
`name` VARCHAR(32) NOT NULL DEFAULT '',
PRIMARY KEY (`thing_id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8mb4;
And some sample data:
INSERT INTO things VALUES (NULL, 'thing 1'), (NULL, 'thing 2');
In my application, sometimes I want to bump the primary key of a row to be the highest in the table. For example, I want the thing_id of thing 1 changed to 3 (the table's current autoincrement value). Previously the table was MyISAM and this was accomplished as follows:
UPDATE things t
JOIN (SELECT MAX(thing_id) + 1 AS max_id FROM things) v
SET t.thing_id = v.max_id
WHERE thing_id = 1;
That part still works. However, now with InnoDB, the next insert fails since doing that update leaves the table's autoincrement value still pointing to the same thing. So now if I do an insert now like this:
INSERT INTO things VALUES (NULL, 'thing 3');
I will get an error like:
Duplicate entry '3' for key 'PRIMARY'
This was not an issue with MyISAM. How can I accomplish the same thing using InnoDB without running into this error? My guess is that there's a better way to change the thing_id which will keep the table's autoincrement value intact, but that's where I'm stumped.
And lastly, here's a dbfiddle:
https://www.db-fiddle.com/f/enJPVkwNN6hocjquw38BHD/0
Reference: Innodb Auto Increment Initialization
If you specify an AUTO_INCREMENT column for an InnoDB table, the table handle in the InnoDB data dictionary contains a special counter called the auto-increment counter that is used in assigning new values for the column. This counter is stored only in main memory, not on disk.
Update:
You can reset this counter using below query, works for InnoDB storage engine in MySQL 5.7
ALTER TABLE things AUTO_INCREMENT = 1;
Executing this is resetting auto-increment counter to Max + 1 value.
I have no idea how to fix this, I've dumped and recreated the database (per an answer somewhere else), tried manually inserting 128 ... and tried just deleting 127 and tried again. :/ Can't figure it out.
Copied this from the query I just tried:
INSERT INTO `bestofthebest2`.`Topics` (`topicid`, `category`, `topic`)
VALUES ('128', '', ''), (NULL , '', '')
MySQL said: Documentation
#1062 - Duplicate entry '127' for key 'topicid'
Is the data type of topicid a TINYINT by chance? The maximum value a signed tinyint can hold is 127. Additionally, MySQL has a rather odd (in my opinion) overflow behavior in that it simply rounds down overflows rather than erroring. What's happening is that it's trying to increment to 128, but that overflows, so it gets changed to 127. Since 127 already exists, it errors.
This can be easily recreated:
CREATE TABLE test ( id TINYINT AUTO_INCREMENT PRIMARY KEY );
INSERT INTO test (id) VALUES (128);
-- A select will show you a row with id = 127
INSERT INTO test (id) VALUES (128);
-- ERROR 1062 (23000): Duplicate entry '127' for key 'PRIMARY'
The simplest fix is to use a bigger data type. A signed integer will afford you 2^31-1 values, so unless you plan on having more than 2 billionish topics, it should work well:
ALTER TABLE bestofthebest2 CHANGE topicid topicid INT NOT NULL;
First:
SELECT MAX(topicid)+1 FROM `bestofthebest2`.`Topics`;
Then with the result as $RESULT:
ALTER TABLE `bestofthebest2`.`Topics` AUTO_INCREMENT = $RESULT;
I have Table called url_info and the structure of the table is:
url_info:
url_id ( auto_increment, primary key )
url ( unique,varchar(500) )
When I insert into table like this:
INSERT INTO url_info(url) VALUES('Tom');
INSERT INTO url_info(url) VALUES('Jerry');
The output is:
1 Tom
2 Jerry
When I insert like this
INSERT INTO url_info(url) VALUES('Tom');
INSERT INTO url_info(url) VALUES('Tom');
INSERT INTO url_info(url) VALUES('Jerry');
The output is
1 Tom
3 Jerry
The auto-increment id is incremented when I try to insert to duplicate entry. I have also tried Insert Ignore
How to prevent it from incrementing when I try to insert a duplicate entry?
It's probably worth creating a stored procedure to insert what you want into the table. But, in the stored procedure check what items you have already in the table. If these match what you're trying to insert, then the query should not even attempt the insert.
Ie. The procedure needs to contain something like this:
IF NOT EXISTS(SELECT TOP 1 url_id FROM url_info WHERE url = 'Tom')
INSERT INTO url_info(url) VALUES('Tom')
So, in your stored procedure, it would look like this (assuming the arguments/variables have been declared)
IF NOT EXISTS(SELECT TOP 1 url_id FROM url_info WHERE url = #newUrl)
INSERT INTO url_info(url) VALUES(#newUrl)
This is expected behaviour in InnoDB. The reason is that they want to let go of the auto_increment lock as fast as possible to improve concurrency. Unfortunately this means they increment the AUTO_INCREMENT value before resolving any constraints, such as UNIQUE.
You can read more about the idea in the manual on AUTO_INCREMENT Handling in InnoDB, but the manual is also unfortunately buggy and doesn't tell why your simple insert will give non-consecutive values.
If this is a real problem for you and you really need consecutive numbers, consider setting the innodb_autoinc_lock_mode option to 0 in your server, but this is not recommended as it will have severe effects on your database (you cannot do any inserts concurrently).
Auto_increment is performed updated by the engine. This is done before hand of checking a value is unique or not. And we can't roll back the operation to get back to former value of auto_increment.
Hence NO to start from where you last read on auto_increment.
And it is not an issue in loosing some intermediate values on auto_increment field.
The MAX value you can store into a SIGNED INT field is 2^31-1 equal to 2,147,483,647. If you read it loud, it sounds 2 billion+.
And I don't think it is small and won't suite your requirement.
CREATE TABLE `url_info` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`url` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=4 ;
When I execute:
INSERT INTO url_info(url) VALUES('Tom');
INSERT INTO url_info(url) VALUES('Tom');
INSERT INTO url_info(url) VALUES('Jerry');
I get:
Make sure you ID column is UNIQUE too.
As the manual says:
A UNIQUE index creates a constraint such that all values in the index
must be distinct. An error occurs if you try to add a new row with a
key value that matches an existing row. This constraint does not apply
to NULL values except for the BDB storage engine. For other engines, a
UNIQUE index permits multiple NULL values for columns that can contain
NULL. If you specify a prefix value for a column in a UNIQUE index,
the column values must be unique within the prefix.
I have to update a table with the following structure:
CREATE TABLE `eav_entity_attribute` (
`entity_attribute_id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Entity Attribute Id',
`entity_type_id` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Entity Type Id',
`attribute_set_id` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Attribute Set Id',
`attribute_group_id` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Attribute Group Id',
`attribute_id` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Attribute Id',
`sort_order` smallint(6) NOT NULL DEFAULT '0' COMMENT 'Sort Order',
PRIMARY KEY (`entity_attribute_id`),
UNIQUE KEY `UNQ_EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_SET_ID_ATTRIBUTE_ID` (`attribute_set_id`,`attribute_id`),
UNIQUE KEY `UNQ_EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_GROUP_ID_ATTRIBUTE_ID` (`attribute_group_id`,`attribute_id`),
KEY `IDX_EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_SET_ID_SORT_ORDER` (`attribute_set_id`,`sort_order`),
KEY `IDX_EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_ID` (`attribute_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Eav Entity Attributes'
Above table contains a single row:
INSERT INTO `eav_entity_attribute`
(`entity_attribute_id`, `entity_type_id`, `attribute_set_id`, `attribute_group_id`, `attribute_id`, `sort_order`)
VALUES
(32758, 4, 224, 3423, 5171, 12)
I'm running an automatic import procedure, which will read an external source of data and write into this table.
This import runs multiple times and, therefore, sometimes the same data is imported several times. In such case, the procedure simply overwrites the old data with the new one, even when the new one is identical to the old. The condition where the same data exists is handled with an ON DUPLICATE KEY UPDATE clause. This works almost perfectly, except on this specific table.
On this table, when the procedure attempts an UPDATE, I receive a "Duplicate key" message, which I can't explain. I debugged the code, and this is the query that fails (extracted from the INSERT..ON DUPLICATE KEY):
UPDATE eav_entity_attribute
SET
`attribute_group_id` = 3423
,`attribute_id` = 5171
,`attribute_set_id` = 223
,`entity_type_id` = 4
,`sort_order` = 320
WHERE
(`attribute_group_id` = 3423) AND
(`attribute_id` = 5171)
The error is the following:
Error Code: 1062. Duplicate entry '3423-5171' for key 'UNQ_EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_GROUP_ID_ATTRIBUTE_ID'
I know that the pair 3423-5171 already exists, but the UPDATE would replace these values with themselves, not create a new entry. I'm quite confused about the cause of this issue, any suggestion would be very welcome. Thanks.
Update - New finding
I got some sort of "inspiration" and I made an experiment. I removed the Unique constraint involving on (attribute_set_id,attribute_id) (note, this is not the one in the error) and I ran the INSERT..ON DUPLICATE query. It worked perfectly.
Mine is a conjecture, but this is what I think: the data I'm trying to write to the table clashes with two constraints:
UNIQUE(attribute_set_id,attribute_id)
UNIQUE(attribute_group_id,attribute_id)
The INSERT fails, presumably because of the duplication error raised by the first constraint. This triggers the UPDATE, which uses the first constraint as the implicit WHERE clause. My speculation is that, in such case, the first constraint is somehow ignored, but the UPDATE trips over the second, which didn't get involved earlier.
This still doesn't seem, to me, a valid reason for an UPDATE which replaces something with itself to raise a duplicate entry error, but it may shed some light on the logic behind it.
Second Update
I found out that the table I was testing against actually contains a lot of rows (I forgot to disable the filtered view) resulting from the successful import of other data. However, the "duplicate candidate" is still unique in the set.
I confirm what posted in the comments, when the table contains only that rows, the INSERT..ON DUPLICATE works, as well as the UPDATE alone. Now I'm wondering why does the table get messed up when there is more data in it, since we are still talking about a single unique row being updated with the same data.
Third Update - Found the root cause
I finally found out the reason why the UPDATE fails, now I have to find out how do I get in such condition.
The clue was my conjecture in the first update. Simply, I have two very similar rows (please note I'm using different values as I started from a clean database).
row,entity_attribute_id,entity_type_id,attribute_set_id,attribute_group_id,attribute_id,sort_order
1,16919, 4, 120, 1746, 80, 1
2,16649, 4, 119, 1744, 80, 210
Here's what happens:
The INSERT attempts to insert a row with the following values: 120, 4, 1744, 80, 54.
This triggers the "duplicate key", since the values 120, 80 are a duplicate for the fields attribute_set_id, attribute_id (row 1).
MySQL then tries the UPDATE, which becomes as follows:
UPDATE table
entity_type_id = 4
,attribute_group_id = 1744
,sort_order = 54
WHERE
(attribute_set_id = 120) AND (attribute_id = 80)
This time, the UPDATE fails because the values 1744,80 are violate the constraint on the pair attribute_group_id, attribute_id, found in row 2.
In summary
The INSERT fails because row 1 has the same values for the key attribute_set_id, attribute_id.
The UPDATE fails because row 2 has the same values for the key attribute_group_id, attribute_id.
Solution
I will have to review the whole import procedure, as, in theory, none of such duplicates should arise. MySQL is doing its job fine, it's the database that is complicated.
Thanks for all the suggestions.
Try not to update key values within UPDATE clause of INSERT ... ON DUPLICATE KEY UPDATE. It is strange to ask MySQL to change key values if a record with these key values already exist, so, unexpected behavoiur of MySQL is not surprising.
In my MySQL table I've created an ID column which I'm hoping to auto-increment in order for it to be the primary key.
I've created my table:
CREATE TABLE `test` (
`id` INT( 11 ) NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`name` VARCHAR( 50 ) NOT NULL ,
`date_modified` DATETIME NOT NULL ,
UNIQUE (
`name`
)
) TYPE = INNODB;
then Inserted my records:
INSERT INTO `test` ( `id` , `name` , `date_modified` )
VALUES (
NULL , 'TIM', '2011-11-16 12:36:30'
), (
NULL , 'FRED', '2011-11-16 12:36:30'
);
I'm expecting that my ID's for the above are 1 and 2 (respectively). And so far this is true.
However when I do something like this:
insert into test (name) values ('FRED')
on duplicate key update date_modified=now();
then insert a new record, I'm expecting it to be 3, however now I'm shown an ID of 4; skipping the place spot for 3.
Normally this wouldn't be an issue but I'm using millions of records which have thousands of updates every day.. and I don't really want to even have to think about running out of ID's simply because I'm skipping a ton of numbers..
Anyclue to why this is happening?
MySQL version: 5.1.44
Thank you
My guess is that the INSERT itself kicks off the code that generates the next ID number. When the duplicate key is detected, and ON DUPLICATE KEY UPDATE is executed, the ID number is abandoned. (No SQL dbms guarantees that automatic sequences will be without gaps, AFAIK.)
MySQL docs say
In general, you should try to avoid using an ON DUPLICATE KEY UPDATE
clause on tables with multiple unique indexes.
That page also says
If a table contains an AUTO_INCREMENT column and INSERT ... ON
DUPLICATE KEY UPDATE inserts or updates a row, the LAST_INSERT_ID()
function returns the AUTO_INCREMENT value.
which stops far short of describing the internal behavior I guessed at above.
Can't test here; will try later.
Is it possible to change your key to unsigned bigint - 18,446,744,073,709,551,615 is a lot of records - thus delaying the running out of ID's
Found this in mysql manual http://dev.mysql.com/doc/refman/5.1/en/example-auto-increment.html
Use a large enough integer data type for the AUTO_INCREMENT column to hold the
maximum sequence value you will need. When the column reaches the upper limit of
the data type, the next attempt to generate a sequence number fails. For example,
if you use TINYINT, the maximum permissible sequence number is 127.
For TINYINT UNSIGNED, the maximum is 255.
More reading here http://dev.mysql.com/doc/refman/5.6/en/information-functions.html#function_last-insert-id it could be inferred that the insert to a transactional table is a rollback so the manual says "LAST_INSERT_ID() is not restored to that before the transaction"
What about for a possible solution to use a table to generate the ID's and then insert into your main table as the PK using LAST_INSERT_ID();
From the manual:
Create a table to hold the sequence counter and initialize it:
mysql> CREATE TABLE sequence (id INT NOT NULL);
mysql> INSERT INTO sequence VALUES (0);
Use the table to generate sequence numbers like this:
mysql> UPDATE sequence SET id=LAST_INSERT_ID(id+1);
mysql> SELECT LAST_INSERT_ID();
The UPDATE statement increments the sequence counter and causes the next call to
LAST_INSERT_ID() to return the updated value. The SELECT statement retrieves that
value. The mysql_insert_id() C API function can also be used to get the value.
See Section 20.9.3.37, “mysql_insert_id()”.
It's really a bug how you can see here: http://bugs.mysql.com/bug.php?id=26316
But, apparently, they fixed it on 5.1.47 and it was declared as INNODB plugin problem.
A duplicate, but same problem, you can see here too: http://bugs.mysql.com/bug.php?id=53791 referenced to the first page mentioned here in this answer.