Related
I have table structure like this
when I insert row to the table I'm using this query:
INSERT INTO table_blah ( material_item, ... hidden ) VALUES ( data, ... data ) ON DUPLICATE KEY UPDATE id = id, material_item = data, ... hidden = data;
when I first insert data without triggering the ON DUPLICATE KEY the id increments fine:
but when the ON DUPLICATE KEY triggers and i INSERT A NEW ROW the id looks odd to me:
How can I keep the auto increment, increment properly even when it triggers ON DUPLICATE KEY?
This behavior is documented (paragraph in parentheses):
If you specify ON DUPLICATE KEY UPDATE, and a row is inserted that
would cause a duplicate value in a UNIQUE index or PRIMARY KEY, MySQL
performs an UPDATE of the old row. For example, if column a is
declared as UNIQUE and contains the value 1, the following two
statements have similar effect:
INSERT INTO table (a,b,c) VALUES (1,2,3) ON DUPLICATE KEY UPDATE c=c+1;
UPDATE table SET c=c+1 WHERE a=1;
(The effects are not identical for
an InnoDB table where a is an auto-increment column. With an
auto-increment column, an INSERT statement increases the
auto-increment value but UPDATE does not.)
Here is a simple explanation. MySQL attempts to do the insert first. This is when the id gets auto incremented. Once incremented, it stays. Then the duplicate is detected and the update happens. But the value gets missed.
You should not depend on auto_increment having no gaps. If that is a requirement, the overhead on the updates and inserts is much larger. Essentially, you need to put a lock on the entire table, and renumber everything that needs to be renumbered, typically using a trigger. A better solution is to calculate incremental values on output.
This question is a fairly old one, but I answer it maybe it helps someone, to solve the auto-incrementing problem use the following code before insert/on duplicate update part and execute them all together:
SET #NEW_AI = (SELECT MAX(`the_id`)+1 FROM `table_blah`);
SET #ALTER_SQL = CONCAT('ALTER TABLE `table_blah` AUTO_INCREMENT =', #NEW_AI);
PREPARE NEWSQL FROM #ALTER_SQL;
EXECUTE NEWSQL;
together and in one statement it should be something like below:
SET #NEW_AI = (SELECT MAX(`the_id`)+1 FROM `table_blah`);
SET #ALTER_SQL = CONCAT('ALTER TABLE `table_blah` AUTO_INCREMENT =', #NEW_AI);
PREPARE NEWSQL FROM #ALTER_SQL;
EXECUTE NEWSQL;
INSERT INTO `table_blah` (`the_col`) VALUES("the_value")
ON DUPLICATE KEY UPDATE `the_col` = "the_value";
I had the same frustration of gaps in the auto increment but I found a way to avoid it.
In terms of previouslly discussed "overheads". When I first wrote my DB query code, it did so many separate queries that it took 5 hours. Once I put on
"ON DUPLICATE KEY UPDATE"
it got it down to about 50 seconds. Amazing! Anyway the way I solved it was by using 2 queries. Which doulbles the time it takes to 2 minutes, which is still fine.
First I did an sql query for writing all the data (updates and inserts), but I included "IGNORE" in the first query, so this just bypasses the updates and only inserts the new stuff. So assuming your auto_increment previously has no gaps then it will still have no gaps because its only new records. I believe it is updates that cause the gaps. So for inserts:
"INSERT IGNORE INTO mytablename(stuff,stuff2) VALUES "
Next I did the "ON DUPLICATE KEY UPDATE" variation of that sql query. It will keep the ID's in tact because all the records being updated have ID's already. The only thing it breaks is the auto_increment value, which gets incremented when a new record is added (or updated). So the solution is to patch this auto_increment value back to what it was before, once you have applied the updates.
To patch the auto increment value use this sql in your php:
"ALTER TABLE mytablename AUTO_INCREMENT = " . ($TableCount + 1);
This works because when you do the updates you are not increasing the amount of records. Therefore we can use the tablecount to know what the next ID should be. You set $TableCount to the table count, then we add 1 and that's the next auto increment number.
This is cheap and dirty but it seems to work. Could be bad using this while something else is writing to the db though.
Change database engine from InnoDB to MyIsam will resolve your issue.
I often deal with this by creating a temporary table, recording in the temporary table whether the record is new or not, doing an UPDATE only on the rows that are not new, and doing an INSERT with the new rows. Here's a complete example:
## THE SETUP
# This is the table we're trying to insert into
DROP TABLE IF EXISTS items;
CREATE TABLE items (
id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(100) UNIQUE,
price INT
);
# Put a few rows into the table
INSERT INTO items (name, price) VALUES
("Bike", 200),
("Basketball", 10),
("Fishing rod", 25)
;
## THE INSERT/UPDATE
# Create a temporary table to help with the update
DROP TEMPORARY TABLE IF EXISTS itemUpdates;
CREATE TEMPORARY TABLE itemUpdates (
name VARCHAR(100) UNIQUE,
price INT,
isNew BOOLEAN DEFAULT(true)
);
# Change the price of the Bike and Basketball and add a new Tent item
INSERT INTO itemUpdates (name, price) VALUES
("Bike", 150),
("Basketball", 8),
("Tent", 100)
;
# For items that already exist, set isNew false
UPDATE itemUpdates
JOIN items
ON items.name = itemUpdates.name
SET isNew = false;
# UPDATE the already-existing items
UPDATE items
JOIN itemUpdates
ON items.name = itemUpdates.name
SET items.price = itemUpdates.price
WHERE itemUpdates.isNew = false;
# INSERT the new items
INSERT INTO items (name, price)
SELECT name, price
FROM itemUpdates
WHERE itemUpdates.isNew = true;
# Check the results
SELECT * FROM items;
# Results:
# ID | Name | Price
# 1 | Bike | 150
# 2 | Basketball | 8
# 3 | Fishing rod | 25
# 4 | Tent | 100
The INSERT IGNORE INTO approach is simpler, but it ignores any error, which isn't what I want. And I agree that this is strange behavior on the part of MySQL but it's what we've got to work with.
I just thought I'd add, as i was trying to find an answer to my problem.
I could not stop the duplicate warning and found it was because I had it set it to TINYINT which only allows 127 entries, changing to SMALL/MED/BIGINT allows for many more
I don't think this is a problem with MySQL 5.6. See this example.
ON DUPLICATE KEY UPDATE id=LAST_INSERT_ID(id)
Adding less of a direct answer and more of a fix to the end results.
If you don't use your autoincrement as an identification field within your application (and you really shouldn't be. A UUID or something of that nature is better practice), and of course, if you don't have multi-billions of lines, you can reset your autoincrement field fairly easily.
SET SQL_SAFE_UPDATES = 0;
SET #num := 0;
UPDATE my_table SET id = #num := (#num+1);
ALTER TABLE my_table AUTO_INCREMENT =1;
I kinda hate that this is a thing when doing an INSERT UPDATE in MySQL.
This is not my code. I got it some somewhere on SO but it was so long ago...
Additional note, this is not really an answer to this issue. Its more to help fix an out-of-control autoincrement field.
INSERT INTO table_blah ( material_item, ... hidden ) VALUES ( data, ... data ) ON DUPLICATE KEY UPDATE material_item = data, ... hidden = data
Yes remove the ID=ID as it will automaticly add where PRIMARY KEY = PRIMARY KEY...
I am trying to set up a script to generate a particular set of test data into my database, at the beginning of which I want to clear the tables concerned without dropping constraints (because the test data is not the appropriate place to be rebuilding constraints) and reset the AUTO_INCREMENT for each table since setting up the test data is much, much simpler if I can hard-code many of the IDs.
For example, I have two statements like this (there's a pair for nearly every table):
DELETE FROM AppointmentAttr
ALTER TABLE AppointmentAttr AUTO_INCREMENT = 1
and while the records are deleted, the auto-increment value is not reverting to 1, even though all the documentation and SO answers I can find indicate that this should work.
If I do the same statement in MySQL Workbench it also does not revert it.
This is on an INNODB database.
What am I missing?
(Note: I cannot use TRUNCATE due to the presence of constraints).
MySQL does not permit you to decrease the AUTO_INCREMENT value, as specified here:
http://dev.mysql.com/doc/refman/5.6/en/alter-table.html
You cannot reset the counter to a value less than or equal to the value that is currently in use. For both InnoDB and MyISAM, if the value is less than or equal to the maximum value currently in the AUTO_INCREMENT column, the value is reset to the current maximum AUTO_INCREMENT column value plus one.
Even with your constraints, I would try one of the following:
Explicitly insert your identities for your test data. MySQL doesn't have problems with this, unlike some other database engines
Delete and recreate your identity column (or just change it from being an identity), if the constraints aren't on it itself.
Not use an Identity column and use another method (such as a procedure or outside code) to control your Identity. This is really a last resort and I wouldn't generally recommend it...
Note from OP: It was (1) that was what I needed.
From what I can see about the alter table statement.
You can reset auto increment value by using the ALTER TABLE statement. The syntax of the ALTER TABLE statement to reset auto increment value is as follows:
ALTER TABLE table_name AUTO_INCREMENT = value;
You specify the table name after the ALTER TABLE clause and the value which we want to reset to in the expression AUTO_INCREMENT = value.
Notice that the value must be greater than or equal to the current maximum value of the auto-increment column.
Which is where your problem lies I suspect. So basically you are left with a couple of options as follows:
TRUNCATE TABLE: which according to our discussion is not a option
DROP and RECREATE the table: A long and painful experience
ALTER auto number column: I have not tried this but you could theoretically alter the primary key column from auto number to a int and then make it a auto number again. Something like:
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT NOT NULL;
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT AUTONUMBER NOT NULL;
Hope these help a little.
Can you not drop the relevant, auto increment column and recreate it? Example follows:
;;assuming your column is called id and your table is tbl
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD COLUMN id BIGINT UNSIGNED DEFAULT 1 PRIMARY KEY FIRST;
This should work, but I don't use MySQL, just going off the docs. If you need further help, leave a comment and I'll do my best to help out.
I'm sure this has been long answered but when i need to truncate and can't I just do a set foreign_key_checks = 0 then run my truncate and then set foreign_key_checks = 1.
I've run into this problem when I've deleted middle rows from my table.
My answer would be to INSERT NEW DATA TO NOT EXISTING ID.
I expect that my answer still be usefull even if it's PHP not MYSQL.
First fetch your data.
if found not existing row Insert values and exit;
else if not found in whole loop then do insertion for default value;
$rez = mysql_query("SELECT * FROM users");
$exists = 1;
while($row = mysql_fetch_array($rez)){
if ( $exists != $row{'id'} ){
echo "found not existing id: ".$exists;
INSERT INTO users VALUES($exists,.. ,..);
exit;
} $exists += 1;
}
INSERT INTO users VALUES(NULL,.. ,..); ##auto_inc column converts NULL to latest
I HOPE it will help somehow.
In non-problematic circumstances you can do
ALTER TABLE tbl AUTO_INCREMENT = 0;
which brings auto_increment value to the lowest allowed at the time.
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD id INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (id); in your phpMyAdmin
ALTER TABLE table_name AUTO_INCREMENT = value;
This worked for me, I had to set it to the last record in my database while going through the operations panel never worked for me.
This worked for me hope it helps.
SET #autoid = 0; UPDATE users set id = #autoid := (#autoid+1); ALTER TABLE users AUTO_INCREMENT = 1;
I am using mysql, and am looking at a strange behavior.
Scenario :
I have a table having table_id as primary key, which is set to auto-increment.
table_id more_columns
1 some value
2 others
Now if i delete row 2, and insert one more row, the table_id becomes 3 (Expected is 2)
table_id more_columns
1 some value
3 recent
Why is it so? Here I am loosing some ids (I know they are not important). Please put some lights on this behavior
In auto-increment field If a row is deleted, the auto_increment column of that row will not be re-assigned.
Please see here for more information.
For reasons why auto-increment doesn't use deleted values you can refer here(mentioned in comments by #AaronBlenkush).
The auto_increment value is a counter stored internally for each table. The counter is only increased, never decreased.
Every syntactically correct INSERT statement fired against the table increments this counter, even when it is rolled back and also when you define an insert value for the primary key.
A MySQL auto_increment column maintains a number internally, and will always increment it, even after deletions. If you need to fill in an empty space, you have to handle it yourself in PHP, rather than use the auto_increment keyword in the table definition.
Rolling back to fill in empty row ids can cause all sorts of difficulty if you have foreign key relationships to maintain, and it really isn't advised.
The auto_increment can be reset using a SQL statement, but this is not advised because it will cause duplicate key errors.
-- Doing this will cause problems!
ALTER table AUTO_INCREMENT=12345;
EDIT
To enforce your foreign key relationships as described in the comments, you should add to your table definition:
FOREIGN KEY (friendid) REFERENCES registration_table (id) ON DELETE SET NULL;
Fill in the correct table and column names. Now, when a user is deleted from the registration, their friend association is nulled. If you need to reassociate with a different user, that has to be handled with PHP. mysql_insert_id() is no longer helpful.
If you need to find the highest numbered id still in the database after deletion to associate with friends, use the following.
SELECT MAX(id) FROM registration_table;
After delete write this query
ALTER TABLE tablename AUTO_INCREMENT = 1
I am trying to set up a script to generate a particular set of test data into my database, at the beginning of which I want to clear the tables concerned without dropping constraints (because the test data is not the appropriate place to be rebuilding constraints) and reset the AUTO_INCREMENT for each table since setting up the test data is much, much simpler if I can hard-code many of the IDs.
For example, I have two statements like this (there's a pair for nearly every table):
DELETE FROM AppointmentAttr
ALTER TABLE AppointmentAttr AUTO_INCREMENT = 1
and while the records are deleted, the auto-increment value is not reverting to 1, even though all the documentation and SO answers I can find indicate that this should work.
If I do the same statement in MySQL Workbench it also does not revert it.
This is on an INNODB database.
What am I missing?
(Note: I cannot use TRUNCATE due to the presence of constraints).
MySQL does not permit you to decrease the AUTO_INCREMENT value, as specified here:
http://dev.mysql.com/doc/refman/5.6/en/alter-table.html
You cannot reset the counter to a value less than or equal to the value that is currently in use. For both InnoDB and MyISAM, if the value is less than or equal to the maximum value currently in the AUTO_INCREMENT column, the value is reset to the current maximum AUTO_INCREMENT column value plus one.
Even with your constraints, I would try one of the following:
Explicitly insert your identities for your test data. MySQL doesn't have problems with this, unlike some other database engines
Delete and recreate your identity column (or just change it from being an identity), if the constraints aren't on it itself.
Not use an Identity column and use another method (such as a procedure or outside code) to control your Identity. This is really a last resort and I wouldn't generally recommend it...
Note from OP: It was (1) that was what I needed.
From what I can see about the alter table statement.
You can reset auto increment value by using the ALTER TABLE statement. The syntax of the ALTER TABLE statement to reset auto increment value is as follows:
ALTER TABLE table_name AUTO_INCREMENT = value;
You specify the table name after the ALTER TABLE clause and the value which we want to reset to in the expression AUTO_INCREMENT = value.
Notice that the value must be greater than or equal to the current maximum value of the auto-increment column.
Which is where your problem lies I suspect. So basically you are left with a couple of options as follows:
TRUNCATE TABLE: which according to our discussion is not a option
DROP and RECREATE the table: A long and painful experience
ALTER auto number column: I have not tried this but you could theoretically alter the primary key column from auto number to a int and then make it a auto number again. Something like:
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT NOT NULL;
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT AUTONUMBER NOT NULL;
Hope these help a little.
Can you not drop the relevant, auto increment column and recreate it? Example follows:
;;assuming your column is called id and your table is tbl
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD COLUMN id BIGINT UNSIGNED DEFAULT 1 PRIMARY KEY FIRST;
This should work, but I don't use MySQL, just going off the docs. If you need further help, leave a comment and I'll do my best to help out.
I'm sure this has been long answered but when i need to truncate and can't I just do a set foreign_key_checks = 0 then run my truncate and then set foreign_key_checks = 1.
I've run into this problem when I've deleted middle rows from my table.
My answer would be to INSERT NEW DATA TO NOT EXISTING ID.
I expect that my answer still be usefull even if it's PHP not MYSQL.
First fetch your data.
if found not existing row Insert values and exit;
else if not found in whole loop then do insertion for default value;
$rez = mysql_query("SELECT * FROM users");
$exists = 1;
while($row = mysql_fetch_array($rez)){
if ( $exists != $row{'id'} ){
echo "found not existing id: ".$exists;
INSERT INTO users VALUES($exists,.. ,..);
exit;
} $exists += 1;
}
INSERT INTO users VALUES(NULL,.. ,..); ##auto_inc column converts NULL to latest
I HOPE it will help somehow.
In non-problematic circumstances you can do
ALTER TABLE tbl AUTO_INCREMENT = 0;
which brings auto_increment value to the lowest allowed at the time.
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD id INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (id); in your phpMyAdmin
ALTER TABLE table_name AUTO_INCREMENT = value;
This worked for me, I had to set it to the last record in my database while going through the operations panel never worked for me.
This worked for me hope it helps.
SET #autoid = 0; UPDATE users set id = #autoid := (#autoid+1); ALTER TABLE users AUTO_INCREMENT = 1;
Currently every time I add an entry to my database, the auto increment value increments by 1, as it should. However, it is only at a count of 47. So, if I add a new entry, it will be 48, and then another it will be 49 etc.
I want to change what the current Auto Increment counter is at. I.e. I want to change it from 47 to say, 10000, so that the next value entered, will be 10001. How do I do that?
You can use ALTER TABLE to set the value of an AUTO_INCREMENT column ; quoting that page :
To change the value of the
AUTO_INCREMENT counter to be used for
new rows, do this:
ALTER TABLE t2 AUTO_INCREMENT = value;
There is also a note saying that :
You cannot reset the counter to a
value less than or equal to any that
have already been used.
For MyISAM, if
the value is less than or equal to the
maximum value currently in the
AUTO_INCREMENT column, the value is
reset to the current maximum plus one.
For InnoDB, if the value is less than
the current maximum value in the
column, no error occurs and the
current sequence value is not changed.
See manual for ALTER TABLE - this should do it:
ALTER TABLE [tablename] AUTO_INCREMENT = [number]
you can get that done by executing the following statement
ALTER TABLE t2 AUTO_INCREMENT = 10000;
So next Auto Increment key will start from the 10001.
I hope this will solve the problem
You can also set it with the table creation statement as follows;
CREATE TABLE mytable (
id int NOT NULL AUTO_INCREMENT,
...
PRIMARY KEY (ID)
)AUTO_INCREMENT=10000;
Hope it helps someone.