We are running an import of an existing product table into a new table of our own. The import script we've written runs perfectly and inserts the right amount of rows (6000 or so). However, after the import the next auto incremented primary key/id is 1500 entries (or so) above the number of rows in the table.
We can't understand why MySQL is doing this and we'd like it to stop. I've done the usual searches online looking for help but I am drawing a blank - any ideas?
Let's take this simple table for example:
CREATE TABLE `products` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`name` VARCHAR(64) NULL DEFAULT NULL,
PRIMARY KEY (`id`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM;
And import file that look like:
"id","name"
1,"product 1"
2,"product 2"
5,"product 3"
102,"product 4"
Then you are importing data to both columns, so auto incrementing mechanism does not work.
After importing all rows, autoincrement value for table is set to MAX(id)+1 [103 in this case] to ensure next autoincremented id is unique. If it was equal to number of rows inserted, then next autincrement value would be 5 and would colide with row #3.
If you want to have clean start and last id equal to number of rows you have to either get rid of "id" column from .csv file, or create table without AUTO_INCREMENT for id, import data and run this simple sql:
SET #i=0;
UPDATE `products` SET id=#i:=#i+1 ORDER BY id;
ALTER TABLE `products` CHANGE COLUMN `id` `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT FIRST;
First query sets auxiliary variable, that will be incremented before updating the record.
Second one updates record to have id equal to row number.
Third will change id column to be autoincremented and set proper value for next autoindex.
But before changing any primary keys ensure that they are not used in any other tables as foreign keys!
If you perform this command:
show create table Foo
Then you will see what the AUTO_INCREMENT= is set too.
My guess is that it is not set to start with 0.
You should then create your new table to have the AUTO_INCREMENT set to 0 (or 1, I cannot remember from the top of my head). This should do the trick.
Related
What does AUTO_INCREMENT=535 actually mean or do? I have seen this used when creating tables as shown below, but never knew what it does or is used for.
Create Table:
CREATE TABLE `my_table` (
`entry_id` int(11) NOT NULL auto_increment,
`address` varchar(512) NOT NULL,
PRIMARY KEY(entry_id)
) ENGINE=InnoDB AUTO_INCREMENT=535 DEFAULT CHARSET=utf8
Auto increment field allow automatic indexing of the records in a table. Usually serving as a Unique Key
Any table with definition like AUTO_INCREMENT=535 would mean that next auto-generated key will start from the 535.
This usually happen when you take backup from existing database. But also can be used in some special cases to have higher value of starting index.
Tells when to start with auto_increment counting. For example if you want to reserve some number of ID for some dedicated purposes.
The AUTO_INCREMENT attribute can be used to generate a unique identity
You can use a pair of statements: DROP TABLE and CREATE TABLE to reset the auto-increment column. Like the TRUNCATE TABLE statement, those statements removes all the data and reset the auto-increment value to zero.
No value was specified for the AUTO_INCREMENT column, so MySQL assigned sequence numbers automatically.
You can also ** explicitly assign 0 ** to the column to generate sequence numbers. If the column is declared NOT NULL, it is also ** possible to assign NULL ** to the column to generate sequence numbers.
You can retrieve the most recent AUTO_INCREMENT value with the LAST_INSERT_ID()
To start with an AUTO_INCREMENT value other than 1, you can set that value with CREATE TABLE or ALTER TABLE, like this:
mysql> ALTER TABLE tbl AUTO_INCREMENT = 100;
The AUTO INCREMENT interval value is controlled by the MySQL Server variable auto_increment_increment and applies globally. To change this to a number different from the default of 1, use the following command in MySQL:
mysql> SET ##auto_increment_increment = [interval number];
where [interval number] is the interval value you want to use. So, if we want to set the interval to be 5, we would issue the following command:
mysql> SET ##auto_increment_increment = 5;
refrence:-
http://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html
I've got a mysql database with a table (InnoDB) of Games:
gamerooms
id: bigint(20) unsigned not null auto_increment
PRIMARY KEY (`id`)
I'd like to start generating a UUID value for each row which I can share publicly, something like:
gamerooms
id | id_public |
--------------------
1 | abcde
2 | ghijk
3 | lmnop
...
select * from gamerooms where id_public = ...
How do I add this new column, also keeping in mind that there are already records in the table? I'm confused because the column should be marked NOT NULL, but after adding the column, all records that already exist would have empty values.. Do I have to provide a default value?:
ALTER TABLE `gamerooms` ADD COLUMN `id_public` varchar(36) DEFAULT something AFTER `id`
I want to put an index on id_public of course after it's created, so not sure if null values after the column is first created will mess anything up.
Also, I can use varchar(36) with mysqls UUID() output, right?
Thank you
Your ALTER statement is correct:
ALTER TABLE `gamerooms`
ADD COLUMN `id_public` varchar(36) NOT NULL DEFAULT 'something' AFTER `id`
According to my MySQL Pocket Reference, if you don't provide a default value for a column that is defined as NOT NULL:
MySQL picks a value based on the type of the field
In this case, I'm guessing the default would be empty string. Once your column has been added, simply create a new index for the column, and rebuild the index using a null alteration instruction like so:
CREATE INDEX myIndex ON gamerooms(id_public);
ALTER TABLE gamerooms ENGINE = InnoDB;
You may be able to create the index at the same time you do the insert. My MySQL-fu isn't strong enough to know how to do that.
Should the existing records have a value once you create this new column? If yes, you could do this in multiple steps. First, create the new column without constraint or index and then back populate it with the UUID for all existing records. Once everything is populated, add the not null constraint and your indexes.
As a UUID is a 128-bit number, you don't need a varchar column to store it. a char(16) column would just be ok for saving a UUID binary data.
ALTER TABLE `gamerooms` ADD COLUMN `id_public` char(16) NOT NULL DEFAULT '' AFTER `id`
In my MySQL table I've created an ID column which I'm hoping to auto-increment in order for it to be the primary key.
I've created my table:
CREATE TABLE `test` (
`id` INT( 11 ) NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`name` VARCHAR( 50 ) NOT NULL ,
`date_modified` DATETIME NOT NULL ,
UNIQUE (
`name`
)
) TYPE = INNODB;
then Inserted my records:
INSERT INTO `test` ( `id` , `name` , `date_modified` )
VALUES (
NULL , 'TIM', '2011-11-16 12:36:30'
), (
NULL , 'FRED', '2011-11-16 12:36:30'
);
I'm expecting that my ID's for the above are 1 and 2 (respectively). And so far this is true.
However when I do something like this:
insert into test (name) values ('FRED')
on duplicate key update date_modified=now();
then insert a new record, I'm expecting it to be 3, however now I'm shown an ID of 4; skipping the place spot for 3.
Normally this wouldn't be an issue but I'm using millions of records which have thousands of updates every day.. and I don't really want to even have to think about running out of ID's simply because I'm skipping a ton of numbers..
Anyclue to why this is happening?
MySQL version: 5.1.44
Thank you
My guess is that the INSERT itself kicks off the code that generates the next ID number. When the duplicate key is detected, and ON DUPLICATE KEY UPDATE is executed, the ID number is abandoned. (No SQL dbms guarantees that automatic sequences will be without gaps, AFAIK.)
MySQL docs say
In general, you should try to avoid using an ON DUPLICATE KEY UPDATE
clause on tables with multiple unique indexes.
That page also says
If a table contains an AUTO_INCREMENT column and INSERT ... ON
DUPLICATE KEY UPDATE inserts or updates a row, the LAST_INSERT_ID()
function returns the AUTO_INCREMENT value.
which stops far short of describing the internal behavior I guessed at above.
Can't test here; will try later.
Is it possible to change your key to unsigned bigint - 18,446,744,073,709,551,615 is a lot of records - thus delaying the running out of ID's
Found this in mysql manual http://dev.mysql.com/doc/refman/5.1/en/example-auto-increment.html
Use a large enough integer data type for the AUTO_INCREMENT column to hold the
maximum sequence value you will need. When the column reaches the upper limit of
the data type, the next attempt to generate a sequence number fails. For example,
if you use TINYINT, the maximum permissible sequence number is 127.
For TINYINT UNSIGNED, the maximum is 255.
More reading here http://dev.mysql.com/doc/refman/5.6/en/information-functions.html#function_last-insert-id it could be inferred that the insert to a transactional table is a rollback so the manual says "LAST_INSERT_ID() is not restored to that before the transaction"
What about for a possible solution to use a table to generate the ID's and then insert into your main table as the PK using LAST_INSERT_ID();
From the manual:
Create a table to hold the sequence counter and initialize it:
mysql> CREATE TABLE sequence (id INT NOT NULL);
mysql> INSERT INTO sequence VALUES (0);
Use the table to generate sequence numbers like this:
mysql> UPDATE sequence SET id=LAST_INSERT_ID(id+1);
mysql> SELECT LAST_INSERT_ID();
The UPDATE statement increments the sequence counter and causes the next call to
LAST_INSERT_ID() to return the updated value. The SELECT statement retrieves that
value. The mysql_insert_id() C API function can also be used to get the value.
See Section 20.9.3.37, “mysql_insert_id()”.
It's really a bug how you can see here: http://bugs.mysql.com/bug.php?id=26316
But, apparently, they fixed it on 5.1.47 and it was declared as INNODB plugin problem.
A duplicate, but same problem, you can see here too: http://bugs.mysql.com/bug.php?id=53791 referenced to the first page mentioned here in this answer.
I have a table "Bestelling" with 4 columns: "Id" (PK), "KlantId", "Datum", "BestellingsTypeId", now I want to make the column Id auto_increment, however, when I try to do that, I get this error:
ERROR 1062: ALTER TABLE causes auto_increment resequencing, resulting in duplicate entry '1' for key 'PRIMARY'
SQL Statement:
ALTER TABLE `aafest`.`aafest_bestelling` CHANGE COLUMN `Id` `Id` INT(11) NOT NULL AUTO_INCREMENT
ERROR: Error when running failback script. Details follow.
ERROR 1046: No database selected
SQL Statement:
CREATE TABLE `aafest_bestelling` (
`Id` int(11) NOT NULL,
`KlantId` int(11) DEFAULT NULL,
`Datum` date DEFAULT NULL,
`BestellingstypeId` int(11) DEFAULT NULL,
PRIMARY KEY (`Id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Anyone got an idea?
This will happen if the table contains an existing record with an id of 0 (or negative). Updating all existing records to use positive values will allow auto_increment to be set on that column.
Edit: Some people asked how that 0 got in there. For clarification, the MySQL Reference Manual states that "For numeric types, the default is 0, with the exception that for integer or floating-point types declared with the AUTO_INCREMENT attribute, the default is the next value in the sequence." So, if you performed an insert on a table without providing a value for the numeric column before the auto_increment was enabled, then the default 0 would be used during the insert. More details may be found at https://dev.mysql.com/doc/refman/5.0/en/data-type-defaults.html.
I also had this issue when trying to convert a column to auto_increment where one row had a value of 0. An alternative to changing the 0 value temporarily is via setting:
SET SESSION sql_mode='NO_AUTO_VALUE_ON_ZERO';
for the session.
This allowed the column to be altered to auto_increment with the zero id in place.
The zero isn't ideal - and I also wouldn't recommend it being used in an auto_increment column. Unfortunately it's part of an inherited data set so I'm stuck with it for now.
Best to clear the setting (and any others) afterwards with:
SET SESSION sql_mode='';
although it will be cleared when the current client session clsoes.
Full details on the 'NO_AUTO_VALUE_ON_ZERO' setting here.
This happens when MySQL can not determine a proper auto_increment value. In your case, MySQL choose 1 as next auto_increment value, however there is already row with that value in the table.
One way to resolve the issue is to choose a proper auto_increment value yourself:
ALTER TABLE ... CHANGE COLUMN `Id` `Id` INT(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT = 123456;
(Note the AUTO_INCREMENT=123456 at the end.)
The easiest way that I have found to solve this issue is to first set the table's AUTO INCREMENT value before altering the column. Just make sure that you set the auto increment value higher than the largest value currently in that column:
ALTER TABLE `aafest`.`aafest_bestelling`
AUTO_INCREMENT = 100,
CHANGE COLUMN `Id` `Id` INT(11) NOT NULL AUTO_INCREMENT
I tested this on MySQL 5.7 and it worked great for me.
Edit: Don't know exactly how that would be caused, but I do have a workaround.
First, create a new table like the old one:
CREATE TABLE aafest_bestelling_new LIKE aafest_bestelling;
Then change the column
ALTER TABLE `aafest`.`aafest_bestelling_new`
CHANGE COLUMN `Id` `Id` INT(11) NOT NULL AUTO_INCREMENT
Dump in the new data:
INSERT INTO aafest_bestelling_new
(KlantId, Datum, BestellingTypeId)
SELECT
KlantId, Datum, BestellingTypeId
FROM aafest_bestelling;
Move the tables:
RENAME TABLE
aafest_bestelling TO aafest_bestelling_old,
aafest_bestelling_new TO aafest_bestelling;
Maybe there's some corruption going on, and this would fix that as well.
P.S.: As a dutchman, I'd highly recommend coding in english ;)
I had a similar issue. Issue was the table had a record with ID = 0 similar to what SystemParadox pointed out. I handled my issue by the following steps:
Steps:
Update record id 0 to be x where x = MAX(id)+1
Alter table to set primary key and auto increment setting
Set seed value to be x+1
Change record id x back to 0
Code Example:
UPDATE foo SET id = 100 WHERE id = 0;
ALTER TABLE foo MODIFY COLUMN id INT(11) NOT NULL AUTO_INCREMENT;
ALTER TABLE foo AUTO_INCREMENT = 101;
UPDATE foo SET id = 0 WHERE id = 100;
This happens because your primary key column already has values.
As the error says ...
ALTER TABLE causes auto_increment resequencing, resulting in duplicate entry '1' for key 'PRIMARY'
which means that your column already has a primary key value 1 which when you auto_increment that column is reassigned causing duplication and hence this error
the solution to this is to remove the primary constraint and then empty the column. Then alter the table setting the primary key again, this time with auto increment.
This error comes because the any table contains an existing record with an id of 0 (or negative). Update all existing records to use positive values will allow auto_increment to be set on that column.
If this didn't work then export all the data and save it any where in you computer and dont first make foreign key relation then fill data in parent table .
This error will also happen if have a MyISAM table that has a composite AUTO_INCREMENT PRIMARY KEY and are trying to combine the keys
For example
CREATE TABLE test1 (
`id` int(11) NOT NULL,
`ver` int(10) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`,`ver`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
INSERT INTO test1 (`id`, `ver`) VALUES (1,NULL),(1,NULL),(1,NULL), (2,NULL),(2,NULL),(2,NULL);
ALTER TABLE test1 DROP PRIMARY KEY, ADD PRIMARY KEY(`ver`);
Not being able to set an existing column to auto_increment also happens if the column you're trying to modify is included in a foreign key relation in another table (although it won't produce the error message referred to in the question).
(I'm adding this answer even though it doesn't relate to the specific error message in the body of the question because this is the first result that shows up on Google when searching for issues relating to not being able to set an existing MySQL column to auto_increment.)
How to have only 3 rows in the table and only update them?
I have the settings table and at first run there is nothing so I want to insert 3 records like so:
id | label | Value | desc
--------------------------
1 start 10 0
2 middle 24 0
3 end 76 0
After this from PHP script I need to update this settings from one query.
I have researched REPLACE INTO but I end up with duplicate rows in DB.
Here is my current query:
$query_insert=" REPLACE INTO setari (`eticheta`, `valoare`, `disabled`)
VALUES ('mentenanta', '".$mentenanta."', '0'),
('nr_incercari_login', '".$nr_incercari_login."', '0'),
('timp_restrictie_login', '".$timp_restrictie_login."', '0')
";
Any ideas?
Here is the create table statement. Just so you can see in case I'm missing something.
CREATE TABLE `setari` (
`id` int(10) unsigned NOT NULL auto_increment,
`eticheta` varchar(200) NOT NULL,
`valoare` varchar(250) NOT NULL,
`disabled` tinyint(1) unsigned NOT NULL default '0',
`data` datetime default NULL,
`cod` varchar(50) default NULL,
PRIMARY KEY (`eticheta`,`id`,`valoare`),
UNIQUE KEY `id` (`eticheta`,`id`,`valoare`)
) ENGINE=MyISAM
As explained in the manual, need to create a UNIQUE index on (label,value) or (label,value,desc) for REPLACE INTO determine uniqueness.
What you want is to use 'ON DUPLICATE KEY UPDATE' syntax. Read through it for the full details but, essentially you need to have a unique or primary key for one of your fields, then start a normal insert query and add that code (along with what you want to actually update) to the end. The db engine will then try to add the information and when it comes across a duplicate key already inserted, it already knows to just update all the fields you tell it to with the new information.
I simply skip the headache and use a temporary table. Quick and clean.
SQL Server allows you to select into a non-existing temp table by creating it for you. However mysql requires you to first create the temp db and then insert into it.
1.
Create empty temp table.
CREATE TEMPORARY TABLE IF NOT EXISTS insertsetari
SELECT eticheta, valoare, disabled
FROM setari
WHERE 1=0
2.
Insert data into temp table.
INSERT INTO insertsetari
VALUES
('mentenanta', '".$mentenanta."', '0'),
('nr_incercari_login', '".$nr_incercari_login."', '0'),
('timp_restrictie_login', '".$timp_restrictie_login."', '0')
3.
Remove rows in temp table that are already found in target table.
DELETE a FROM insertsetari AS a INNER JOIN setari AS b
WHERE a.eticheta = b.eticheta
AND a.valoare = b.valoare
AND a.disabled = b.disabled
4.
Insert temp table residual rows into target table.
INSERT INTO setari
SELECT * FROM insertsetari
5.
Cleanup temp table.
DELETE insertsetari
Comments:
You should avoid replacing when the
new data and the old data is the
same. Replacing should only be for
situations where there is high
probability for detecting key values
that are the same but the non-key
values are different.
Placing data into a temp table allows
data to be massaged, transformed and modified
easily before inserting into target
table.
Deleting rows from temp table is
faster.
If anything goes wrong, temp table
gives you an additional debugging
stage to find out what went wrong.
Should consider doing it all in a single transaction.