I have seen this issue around (See links at bottom) but I can't seem to figure out an answer. The problem is that I insert data on a table with an auto increment ID that is a primary key, and another field with a UNIQUE index to avoid duplicates. This works, but when that happens the ID is incremented, although no data has been stored.
Would it be better to remove the auto increment, and handle it myself, selecting the max(ID)?
At the moment I have tried several strategies to make it work as is, including INSERT IGNORE and INSERT ... ON DUPLICATE KEY UPDATE
My latest try was using the following query:
INSERT INTO
content(field1, field2)
SELECT(:field1, :field2) FROM DUAL
WHERE NOT EXISTS(
SELECT field1, field2
FROM content
WHERE field1 = :field1
)
Related
In MySQL, when there is a "duplicate entry" error, how do I prevent the primary key from auto incrementing?
SQL Server - How to insert a record and make sure it is unique
Thanks to this question I have been able to fix that error. The problem was that the SELECT(:field1, :field2) shouldn't have the parenthesis. So the query should be:
INSERT INTO
content(field1, field2)
SELECT :field1, :field2 FROM DUAL
WHERE NOT EXISTS(
SELECT field1, field2
FROM content
WHERE field1 = :field1
)
You could just do an ORDER BY :)
UPDATE 'table'
SET column_id = column_id+1
WHERE column_id > [point where you want a un-occupied key]
ORDER BY column_id DESC
mysql executes the where and order by first then your update
Related
I am trying to populate a products table on MySQL with latest products, which are retrieved and stored in products_temp table.
So the method for this is straight forward, simply doing an INSERT to products from products_temp, as such:
INSERT INTO products ( select products_temp.* FROM products_temp )
Problem is, it results in a duplicate primary key error, because of the id from products_temp clashing with the id in products.
Can someone tell me how to fix this please?
I tried declaring the fields in the select statement without the id, but that results in "Column count doesn't match value count at row 1"
Any help would be appreciated.
Thanks!
You'll need to declare the columns except the ID on both the INSERT and the SELECT, since the number of fields need to match, and id (as you noticed) can't be inserted as is into the destination table.
INSERT INTO DestTable (field1, field2, field3)
SELECT field1, field2, field3 FROM SourceTable;
An SQLfiddle to test with.
EDIT: You could do it in a bit more hacky way to simplify the insert. You can create a trigger that simply forces the primary key to NULL on insert.
CREATE TRIGGER t_DT BEFORE INSERT ON DestTable
FOR EACH ROW
SET NEW.id = NULL;
then a copy from table to table can be done as simply;
INSERT INTO DestTable SELECT * FROM SourceTable;
Another SQLfiddle.
How about something like:
INSERT INTO products
(
select products_temp.* FROM products_temp
where key not in (select key from products)
)
I am trying to write a query similar to:
INSERT INTO SomeTable(field1, field2)
SELECT 'blah' AS field1,
MAX(AnotherTable.number) AS field2
FROM AnotherTable
ON DUPLICATE KEY
UPDATE field1= 'blah', field2 = MAX(AnotherTable.number)
I get Error Code: 1111
Invalid use of group function.
Reading through the MySql documentation:
http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
the lines of interest are:
"In the values part of ON DUPLICATE KEY UPDATE, you can refer to columns in other tables, as long as you do not use GROUP BY in the SELECT part. One side effect is that you must qualify nonunique column names in the values part. "
Is this the problem I am seeing? I am not specifically doing a GROUP BY in the Select statement, but by using an aggregate function (Max), then I may be grouping implicitly.
If anyone knows for sure if I am implicitly doing a GROUP BY or if there is any other way I can get the desired result I would be very greatful.
I know I am answering my own question here but...
This eventually got it working (thanks to: a broken link)
INSERT INTO SomeTable(field1, field2)
SELECT 'blah' AS field1,
MAX(AnotherTable.number) AS field2
FROM AnotherTable
ON DUPLICATE KEY
UPDATE field2 = values(field2)
Please have a try if this works:
INSERT INTO SomeTable(field1, field2)
SELECT * FROM (
SELECT * FROM (
SELECT 'blah' AS field1,
MAX(AnotherTable.number) AS field2
FROM AnotherTable
) sq
) sq2
ON DUPLICATE KEY
UPDATE field1= 'blah', field2 = sq2.field2
Not sure if 2 times the subquery is needed. I usually use this to circumvent MySQLs limitation to not be able to update the table with values I read from the same table. Not sure if this works here, too.
Is there a way to use a MySQL INSERT similar to the following:
INSERT INTO doc_details SELECT * FROM doc_details WHERE dd_id = 1
This doesn't work because the primary key is being repeated and it can get very long-winded expanding the columns out.
The purpose of this is to duplicate rows in the same table which will get modified later, retrieving the last_insert_id for the new record. So ideas for other ways to do this would be appreciated too.
Thanks.
Simply name the columns you want to duplicate and omit the primary key:
INSERT INTO doc_details (col1, col2, col3)
SELECT col1, col2, col3
FROM doc_details
WHERE dd_id = 1
I'd suggest you to make ID field with AUTO_INCREMENT option, then use NULL values when inserting -
INSERT INTO doc_details(id, column1, column2)
SELECT NULL, column1, column2 FROM doc_details WHERE dd_id = 1;
In this case old ID will be changed with new ones.
You can depend on temporary table to copy from old record and omitting the key field value.
You have to use at least one named column, i.e. the key field name, to omit its repeating values.
See the following example:
CREATE TEMPORARY TABLE tmp SELECT * from doc_details WHERE dd_id = ?;
ALTER TABLE tmp drop pk_field_name_here; -- drop the key field for not repeating
INSERT INTO doc_details SELECT 0, tmp.* FROM tmp;
DROP TABLE tmp;
You can observe that no other filed names are used but the key field name to omit it's value.
You can also refer to my answer to a similar posting at: Mysql: Copy row but with new id.
Thanks for the answers. Really appreciated. Because most answers specify the column, this led to some extra research that said 'wildcards cannot be used in INSERT statements. Select, Modify and insert into the same table
I managed to solve this in my application with a separate SELECT then the INSERT with the columns expanded with a Perl map function:
SELECT * FROM doc_details WHERE dd_id = 1
Then in Perl, with the row as a hash reference in $data:
$data->{'dd_id'} = 0;$columns = join(',', map {$_ .'='. $dbh->quote( $data->{$_} ) } keys %{$cdh} );
Does the trick nicely - it copies the row regardless of changes to the column structure/order as long as the auto_increment column is maintained.
I know it's not a pure SQL solution - although Ravinder provided one that was.
Thanks to all!
So when trying to add an autoincrement to a field it comes up with #1062 - Duplicate entry '1' for key 1 . I've tried removing the primary key and re-adding it and that works fine (which I assume wouldn't if there was duplicates?)
But when I try to add an autoincrement to the field it throws an error. It gives me a browse option which runs the following SQL:
SELECT *
FROM `myTbl`
WHERE CONCAT_WS( "-", 11 ) = "1"
ORDER BY 11
LIMIT 0 , 30
However this returns a empty result set.. suggesting there are no duplicates. So if there are no duplicates, how come i can't add an autoincrement?
Do you have any rows with value 0 or NULL for this column? ALTER TABLE can cause primary keys to be resequenced. In the case of a key of 0, MySQL will try to give it the value 1, which will fail if the key 1 already exists.
Try changing any 0 or NULL values in the column to something higher (and unused).
Michael Mior's answer works if you can change the data in the table. However, there is also a workaround that lets you keep the data intact (I've tested this on MySQL 5.5). Remember that having a zero value as a primary key in MySQL is not a recommended practice just for this reason. If you can get rid of the zero, then do it.
Disable automatic value generation when a zero is inserted:
SET SQL_MODE=NO_AUTO_VALUE_ON_ZERO;
Add AUTO_INCREMENT to your column:
ALTER TABLE ... AUTO_INCREMENT;
Re-enable automatic value generation:
SET SQL_MODE='';
It should be obvious that inserting data to the table during this whole operation can not be allowed. Otherwise there will be unwanted zero values in the column.
SELECT * <<-- Select * is an anti-pattern
FROM myTbl
WHERE CONCAT_WS( "-", 11 ) = "1" <<-- You are not selecting a column
ORDER BY 11 <<-- This however does refer to a column.
LIMIT 30 OFFSET 0
rewrite the query to
SELECT field1, field2, field3, ...., field11
FROM myTbl
WHERE COALESCE(field1, field2, field3, field11) = '1'
ORDER BY field11
LIMIT 30 OFFSET 0
If you want to insert a row use code like this:
INSERT INTO table1 (/*do not list the PK!*/ field2, field3, field4)
VALUES ('a', 'test' ,'b' ,'example');
If you want to select all duplicate rows use:
SELECT id, count(*) as duplicate_count
FROM table1
GROUP BY id
HAVING duplicate_count > 1
You will need to update those id's that are listed as duplicate.
Another option is to add an extra column and drop the old PK.
ALTER TABLE table1 ADD COLUMN new_id unsigned integer not null auto_increment primary key;
ALTER TABLE table1 DROP COLUMN id;
ALTER TABLE table1 CHANGE COLUMN newid id;
Using MySQL 5.1.49, I'm trying to implement a tagging system
the problem I have is with a table with two columns: id(autoincrement), tag(unique varchar) (InnoDB)
When using query, INSERT IGNORE INTO tablename SET tag="whatever", the auto increment id value increases even if the insert was ignored.
Normally this wouldn't be a problem, but I expect a lot of possible attempts to insert duplicates for this particular table which means that my next value for id field of a new row will be jumping way too much.
For example I'll end up with a table with say 3 rows but bad id's
1 | test
8 | testtext
678 | testtextt
Also, if I don't do INSERT IGNORE and just do regular INSERT INTO and handle the error, the auto increment field still increases so the next true insert is still a wrong auto increment.
Is there a way to stop auto increment if there's an INSERT duplicate row attempt?
As I understand for MySQL 4.1, this value wouldn't increment, but last thing I want to do is end up either doing a lot of SELECT statements in advance to check if the tags exist, or worse yet, downgrade my MySQL version.
You could modify your INSERT to be something like this:
INSERT INTO tablename (tag)
SELECT $tag
FROM tablename
WHERE NOT EXISTS(
SELECT tag
FROM tablename
WHERE tag = $tag
)
LIMIT 1
Where $tag is the tag (properly quoted or as a placeholder of course) that you want to add if it isn't already there. This approach won't even trigger an INSERT (and the subsequent autoincrement wastage) if the tag is already there. You could probably come up with nicer SQL than that but the above should do the trick.
If your table is properly indexed then the extra SELECT for the existence check will be fast and the database is going to have to perform that check anyway.
This approach won't work for the first tag though. You could seed your tag table with a tag that you think will always end up being used or you could do a separate check for an empty table.
I just found this gem...
http://www.timrosenblatt.com/blog/2008/03/21/insert-where-not-exists/
INSERT INTO [table name] SELECT '[value1]', '[value2]' FROM DUAL
WHERE NOT EXISTS(
SELECT [column1] FROM [same table name]
WHERE [column1]='[value1]'
AND [column2]='[value2]' LIMIT 1
)
If affectedRows = 1 then it inserted; otherwise if affectedRows = 0 there was a duplicate.
The MySQL documentation for v 5.5 says:
"If you use INSERT IGNORE and the row is ignored, the AUTO_INCREMENT counter
is **not** incremented and LAST_INSERT_ID() returns 0,
which reflects that no row was inserted."
Ref: http://dev.mysql.com/doc/refman/5.5/en/information-functions.html#function_last-insert-id
Since version 5.1 InnoDB has configurable Auto-Increment Locking. See also http://dev.mysql.com/doc/refman/5.1/en/innodb-auto-increment-handling.html#innodb-auto-inc...
Workaround: use option innodb_autoinc_lock_mode=0 (traditional).
I found mu is too short's answer helpful, but limiting because it doesn't do inserts on an empty table. I found a simple modification did the trick:
INSERT INTO tablename (tag)
SELECT $tag
FROM (select 1) as a #this line is different from the other answer
WHERE NOT EXISTS(
SELECT tag
FROM tablename
WHERE tag = $tag
)
LIMIT 1
Replacing the table in the from clause with a "fake" table (select 1) as a allowed that part to return a record which allowed the insert to take place. I'm running mysql 5.5.37. Thanks mu for getting me most of the way there ....
The accepted answer was useful, however I ran into a problem while using it that basically if your table had no entries it would not work as the select was using the given table, so instead I came up with the following, which will insert even if the table is blank, it also only needs you to insert the table in 2 places and the inserting variables in 1 place, less to get wrong.
INSERT INTO database_name.table_name (a,b,c,d)
SELECT
i.*
FROM
(SELECT
$a AS a,
$b AS b,
$c AS c,
$d AS d
/*variables (properly escaped) to insert*/
) i
LEFT JOIN
database_name.table_name o ON i.a = o.a AND i.b = o.b /*condition to not insert for*/
WHERE
o.a IS NULL
LIMIT 1 /*Not needed as can only ever be one, just being sure*/
Hope you find it useful
You can always add ON DUPLICATE KEY UPDATE Read here (not exactly, but solves your problem it seems).
From the comments, by #ravi
Whether the increment occurs or not depends on the
innodb_autoinc_lock_mode setting. If set to a non-zero value, the
auto-inc counter will increment even if the ON DUPLICATE KEY fires
I had the same problem but didn't want to use innodb_autoinc_lock_mode = 0 since it felt like I was killing a fly with a howitzer.
To resolve this problem I ended up using a temporary table.
create temporary table mytable_temp like mytable;
Then I inserted the values with:
insert into mytable_temp values (null,'valA'),(null,'valB'),(null,'valC');
After that you simply do another insert but use "not in" to ignore duplicates.
insert into mytable (myRow) select mytable_temp.myRow from mytable_temp
where mytable_temp.myRow not in (select myRow from mytable);
I haven't tested this for performance, but it does the job and is easy to read. Granted this was only important because I was working with data that was constantly being updated so I couldn't ignore the gaps.
modified the answer from mu is too short, (simply remove one line)
as i am newbie and i cannot make comment below his answer. Just post it here
the query below works for the first tag
INSERT INTO tablename (tag)
SELECT $tag
WHERE NOT EXISTS(
SELECT tag
FROM tablename
WHERE tag = $tag
)
I just put an extra statement after the insert/update query:
ALTER TABLE table_name AUTO_INCREMENT = 1
And then he automatically picks up the highest prim key id plus 1.