I am little confused with insert on duplicate update query.
I have MySQL table with structure like this:
record_id (PRIMARY, UNIQUE)
person_id (UNIQUE)
some_text
some_other_text
I want to update some_text and some_other_text values for person if it's id exists in my table.person or insert new record in this table otherwise. How it can be done if person_id is not PRIMARY?
You need a query that check if exists any row with you record_id (or person_id). If exists update it, else insert new row
IF EXISTS (SELECT * FROM table.person WHERE record_id='SomeValue')
UPDATE table.person
SET some_text='new_some_text', some_other_text='some_other_text'
WHERE record_id='old_record_id'
ELSE
INSERT INTO table.person (record_id, person_id, some_text, some_other_text)
VALUES ('new_record_id', 'new_person_id', 'new_some_text', 'new_some_other_text')
Another better approach is
UPDATE table.person SET (...) WHERE person_id='SomeValue'
IF ROW_COUNT()=0
INSERT INTO table.person (...) VALUES (...)
Your question is very valid. This is a very common requirement. And most people get it wrong, due to what MySQL offers.
The requirement: Insert unless the PRIMARY key exists, otherwise update.
The common approach: ON DUPLICATE KEY UPDATE
The result of that approach, disturbingly: Insert unless the PRIMARY or any UNIQUE key exists, otherwise update!
What can go horribly wrong with ON DUPLICATE KEY UPDATE? You insert a supposedly new record, with a new PRIMARY key value (say a UUID), but you happen to have a duplicate value for its UNIQUE key.
What you want is a proper exception, indicating that you are trying to insert a duplicate into a UNIQUE column.
But what you get is an unwanted UPDATE! MySQL will take the conflicting record and start overwriting its values. If this happens unintentionally, you have mutilated an old record, and any incoming references to the old record are now referencing the new record. And since you probably won't tell the query to update the PRIMARY column, your new UUID is nowhere to be found. If you ever encounter this data, it will probably make no sense and you will have no idea where it came from.
We need a solution to actually insert unless the PRIMARY key exists, otherwise update.
We will use a query that consists of two statements:
Update where the PRIMARY key value matches (affects 0 or 1 rows).
Insert if the PRIMARY key value does not exist (inserts 1 or 0 rows).
This is the query:
UPDATE my_table SET
unique_name = 'one', update_datetime = NOW()
WHERE id = 1;
INSERT INTO my_table
SELECT 1, 'one', NOW()
FROM my_table
WHERE id = 1
HAVING COUNT(*) = 0;
Only one of these queries will have an effect. The UPDATE is easy. As for the INSERT: WHERE id = 1 results in a row if the id exists, or no row if it does not. HAVING COUNT(*) = 0 inverts that, resulting in a row if the id is new, or no row if it already exists.
I have explored other variants of the same idea, such as with a LEFT JOIN and WHERE, but they all looked more convoluted. Improvements are welcome.
13.2.5.3 INSERT ... ON DUPLICATE KEY UPDATE Syntax
If you specify ON DUPLICATE KEY UPDATE, and a row is inserted that
would cause a duplicate value in a UNIQUE index or PRIMARY KEY, MySQL
performs an UPDATE of the old row.
Example:
DELIMITER //
DROP PROCEDURE IF EXISTS `sp_upsert`//
DROP TABLE IF EXISTS `table_test`//
CREATE TABLE `table_test` (
`record_id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
`person_id` INT UNSIGNED NOT NULL,
`some_text` VARCHAR(50),
`some_other_text` VARCHAR(50),
UNIQUE KEY `record_id_index` (`record_id`),
UNIQUE KEY `person_id_index` (`person_id`)
)//
INSERT INTO `table_test`
(`person_id`, `some_text`, `some_other_text`)
VALUES
(1, 'AAA', 'XXX'),
(2, 'BBB', 'YYY'),
(3, 'CCC', 'ZZZ')//
CREATE PROCEDURE `sp_upsert`(
`p_person_id` INT UNSIGNED,
`p_some_text` VARCHAR(50),
`p_some_other_text` VARCHAR(50)
)
BEGIN
INSERT INTO `table_test`
(`person_id`, `some_text`, `some_other_text`)
VALUES
(`p_person_id`, `p_some_text`, `p_some_other_text`)
ON DUPLICATE KEY UPDATE `some_text` = `p_some_text`,
`some_other_text` = `p_some_other_text`;
END//
DELIMITER ;
mysql> CALL `sp_upsert`(1, 'update_text_0', 'update_text_1');
Query OK, 2 rows affected (0.00 sec)
mysql> SELECT
-> `record_id`,
-> `person_id`,
-> `some_text`,
-> `some_other_text`
-> FROM
-> `table_test`;
+-----------+-----------+---------------+-----------------+
| record_id | person_id | some_text | some_other_text |
+-----------+-----------+---------------+-----------------+
| 1 | 1 | update_text_0 | update_text_1 |
| 2 | 2 | BBB | YYY |
| 3 | 3 | CCC | ZZZ |
+-----------+-----------+---------------+-----------------+
3 rows in set (0.00 sec)
mysql> CALL `sp_upsert`(4, 'new_text_0', 'new_text_1');
Query OK, 1 row affected (0.00 sec)
mysql> SELECT
-> `record_id`,
-> `person_id`,
-> `some_text`,
-> `some_other_text`
-> FROM
-> `table_test`;
+-----------+-----------+---------------+-----------------+
| record_id | person_id | some_text | some_other_text |
+-----------+-----------+---------------+-----------------+
| 1 | 1 | update_text_0 | update_text_1 |
| 2 | 2 | BBB | YYY |
| 3 | 3 | CCC | ZZZ |
| 5 | 4 | new_text_0 | new_text_1 |
+-----------+-----------+---------------+-----------------+
4 rows in set (0.00 sec)
SQL Fiddle demo
How about my approach?
Let's say you have one table with a autoincrement id and three text-columns. You want to insert/update the value of column3 with the values in column1 and column2 being a (non unique) key.
I use this query (without explicitly locking the table):
insert into myTable (id, col1, col2, col3)
select tmp.id, 'col1data', 'col2data', 'col3data' from
(select id from myTable where col1 = 'col1data' and col2 = 'col2data' union select null as id limit 1) tmp
on duplicate key update col3 = values(col3)
Anything wrong with that? For me it works the way I want.
A flexible solution should retain the atomicity offered by INSERT ... ON DUPLICATE KEY UPDATE and work regardless of if it's autocommit=true and not depend on a transaction with an isolation level of REPEATABLE READ or greater.
Any solution performing check-then-act across multiple statements would not satisfy this.
Here are the options:
If there tends to be more inserts than updates:
INSERT INTO table (record_id, ..., some_text, some_other_text) VALUES (...);
IF <duplicate entry for primary key error>
UPDATE table SET some_text = ..., some_other_text = ... WHERE record_id = ...;
IF affected-rows = 0
-- retry from INSERT OR ignore this conflict and defer to the other session
If there tends to be more updates than inserts:
UPDATE table SET some_text = ..., some_other_text = ... WHERE record_id = ...;
IF affected-rows = 0
INSERT INTO table (record_id, ..., some_text, some_other_text) VALUES (...);
IF <duplicate entry for primary key error>
-- retry from UPDATE OR ignore this conflict and defer to the other session
If you don't mind a bit of ugliness, you can actually use INSERT ... ON DUPLICATE KEY UPDATE and do this in a single statement:
INSERT INTO table (record_id, ..., some_text, some_other_text) VALUES (...)
ON DUPLICATE KEY UPDATE
some_text = if(record_id = VALUES(record_id), VALUES(some_text), some_text),
some_other_text = if(record_id = VALUES(record_id), VALUES(some_other_text), some_other_text)
IF affected-rows = 0
-- handle this as a unique check constraint violation
Note: affected-rows in these examples mean affected rows and not found rows. The two can be confused because a single parameter switches which of these values the client is returned.
Also note, if some_text and some_other_text are not actually modified (and the record is not otherwise changed) when you perform the update, those checks on affected-rows = 0 will misfire.
I came across this post because I needed what's written in the title, and I found a pretty handy solution, but no one mentioned it here, so I thought of pasting it here. Note that this solution is very handy if you're initiating your database tables. In this case, when you create your corresponding table, define your primary key etc. as usual, and for the combination of columns you want to be unique, simply add
UNIQUE(column_name1,column_name2,...)
at the end of your CREATE TABLE statement, for any combination of the specified columns you want to be unique. Like this, according to this page here, "MySQL uses the combination of values in both column column_name1 and column_name2 to evaluate the uniqueness", and reports an error if you try to make an insert which already has the combination of values for column_name1 and column_name2 you provide in your insert. Combining this way of creating a database table with the corresponding INSERT ON DUPLICATE KEY syntax appeared to be the most suitable solution for me. Just need to think of it carefully before you actually start using your table; when setting up your database tables.
For anyone else, like me, who is a DB noob....the above things didn't work for me. I have a primary key and a unique key... And I wanted to insert if unique key didn't exist. After a LOT of Stack Overflow and Google searching, I found not many results for this... but I did find a site that gave me a working answer: https://thispointer.com/insert-record-if-not-exists-in-mysql/
And for ease of reading here is my answer from that site:
INSERT INTO table (unique_key_column_name)
SELECT * FROM (SELECT 'unique_value' AS unique_key_column_name) AS temp
WHERE NOT EXISTS (
SELECT unique_key_column_name FROM table
WHERE unique_key_column_name = 'unique_value'
) LIMIT 1;
Please also note the ' marks are wrapped around for me because I use string in this case.
Related
I am using mysql db. I know postgresql and SQL server supports partial Indexing. In my case I want to do something like this:
CREATE UNIQUE INDEX myIndex ON myTable (myColumn) where myColumn <> 'myText'
I want to create a unique constraint but it should allow duplicates if it is a particular text.
I couldn't find a direct way to do this in mysql. But, is there a workaround to achieve it?
Filtered indexes could be emulated with function indexes and CASE expression(MySQL 8.0.13 and newer):
CREATE TABLE t(id INT PRIMARY KEY, myColumn VARCHAR(100));
-- NULL are not taken into account with `UNIQUE` indexes
CREATE UNIQUE INDEX myIndex ON t((CASE WHEN myColumn <> 'myText' THEN myColumn END));
-- inserting excluded value twice
INSERT INTO t(id, myColumn) VALUES(1, 'myText'), (2, 'myText');
-- trying to insert different value than excluded twice
INSERT INTO t(id, myColumn) VALUES(3, 'aaaaa');
INSERT INTO t(id, myColumn) VALUES(4, 'aaaaa');
-- Duplicate entry 'aaaaa' for key 'myIndex'
SELECT * FROM t;
db<>fiddle demo
Output:
+-----+----------+
| id | myColumn |
+-----+----------+
| 1 | myText |
| 2 | myText |
| 3 | aaaaa |
+-----+----------+
I suppose there is only one way to achieve it. You can add another column to your table, create index on it and create trigger or do insert/update inside your stored procedures to fill this column using following condition:
if value = 'myText' then put null
otherwise put value
Hope it helps
I have a table of structure:
+----+-----+-------+
| Id | Key | Value |
+----+-----+-------+
| 1 | A | BCD |
| 2 | B | XYZ |
+----+-----+-------+
Here, id is the primary key. I want to write a query that appends a string to value agains a key if it exists, or else, should insert a new row.
For example, if I need to append PQR to a value where key is A, it should update BCD to BCDPQR. But if key is X, which doesn't exist, it should insert a new row with key as X and Value as PQR.
I am looking to do it in one single query, and not multiple, also not a procedure.
Database: MySQL.
You can't do this in one SQL statement, because commented above, INSERT...ON DUPLICATE KEY UPDATE only works if your KEY column is part of a unique key. So you're going to have to write some code.
First start a transaction, because you'll want to lock the row until you commit:
START TRANSACTION;
Do a locking read for the row:
SELECT * FROM MyTable WHERE `KEY` = 'A' FOR UPDATE;
This will either return a row, or else no rows. You will write code to check the result for this difference.
If the SELECT did return a row, then UPDATE the row with the ID you just fetched:
UPDATE MyTABLE SET VALUE = CONCAT(VALUE, 'PQR') WHERE ID = ?
If no row were fetched by the SELECT, then INSERT a new row:
INSERT INTO MyTABLE SET `KEY` = 'A', VALUE = 'PQR';
Don't forget to COMMIT, which will release the lock on the row so another client can do their work.
COMMIT;
The code and logic to do this task is not difficult, and should be well within the skills of any software developer.
What is the fastest approach to update a row and if the parameters don't exist insert it.
My table has 2 columns for ids quote_id, order_id and those columns combined would make a unique column. I don't want 2 rows containing the same quote_id and order_id but either can have multiple rows.
id | quote_id | order_id
1 | q200 | o100
2 | q200 | o101
3 | q201 | o100
Previously I would have added a third field and combined those 2 fields with a - so I could use the ON DUPLICATE KEY UPDATE. But this is not very efficient as I seem to forget to add those fields sometimes.
My idea is to try to run the update query and if it fails the insert it as I run a lot more update queries then insert. How would I put this into a single query instead of the MySQL server having to return a error and then I rerun the insert query.
if (
UPDATE table_name SET column1=value, column2=value2 WHERE some_column=some_value === ERROR
) THEN
INSERT INTO table_name ....
I looked through some of the MySQL documentation and I couldn't find a example that showed how an error is detected in a IF statement
You should have a PRIMARY or UNIQUE constraint over the column(s) that identify rows uniquely. It's normal to use multiple columns for this, and SQL support syntax for it:
CREATE TABLE MyTable (
quote_id VARCHAR(4) NOT NULL,
order_id VARCHAR(4) NOT NULL,
other_data VARCHAR(4),
...
PRIMARY KEY(quote_id, order_id)
);
Then you can rely on the unique constraint to cause an INSERT to fail and run an UPDATE instead:
INSERT INTO MyTable (quote_id, order_id, other_data) VALUES ('q200', 'o100', 'blah blah')
ON DUPLICATE KEY UPDATE
other_data = VALUES(other_data);
Using the VALUES() clause in the UPDATE part means "use the same value for the respective column that I tried to use in the INSERT part."
Say, I've got a simple table: id as primary key, and field. Then i do:
INSERT INTO table(id,field) VALUES (1,'blah')
and then I do it again. So, how can I make MySQL return following:
id
1
So that it works like SELECT-ing conflicted key?
I don't think that's possible without some code, mysql will return a generic duplicate key error
With-Code solution:
Are you using an AUTO_INCREMENT column ?
If you use INSERT IGNORE and the row is ignored,
the AUTO_INCREMENT counter is not incremented and LAST_INSERT_ID() returns 0,
which reflects that no row was inserted.
so:
INSERT IGNORE
if LAST_INSERT_ID(), then done (new row was inserted)
else your data is duplicate
last_insert_id() documentation >
insert Ignore documentation >
You might consider the following...
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table(my_id INT NOT NULL PRIMARY KEY,my_field VARCHAR(12) NOT NULL,conflict TINYINT NOT NULL DEFAULT 0);
INSERT INTO my_table (my_id,my_field) VALUES (1,'blah') ON DUPLICATE KEY UPDATE conflict = 1;
SELECT * FROM my_table WHERE conflict = 1;
Empty set (0.00 sec)
INSERT INTO my_table (my_id,my_field) VALUES (1,'blah') ON DUPLICATE KEY UPDATE conflict = 1;
Query OK, 2 rows affected (0.07 sec)
SELECT * FROM my_table WHERE conflict = 1;
+-------+----------+----------+
| my_id | my_field | conflict |
+-------+----------+----------+
| 1 | blah | 1 |
+-------+----------+----------+
I'm doing a INSERT ... ON DUPLICATE KEY UPDATE but I need the update part to be conditional, only doing the update if some extra condition has changed.
However, WHERE is not allowed on this UPDATE. Is there any workaround for this?
I can't do combinations of INSERT/UPDATE/SELECT since this needs to work over a replication.
I suggest you to use IF() to do that.
Refer: conditional-duplicate-key-updates-with-mysql
INSERT INTO daily_events (created_on, last_event_id, last_event_created_at)
VALUES ('2010-01-19', 23, '2010-01-19 10:23:11')
ON DUPLICATE KEY UPDATE
last_event_id = IF(last_event_created_at < VALUES(last_event_created_at), VALUES(last_event_id), last_event_id);
This is our final solution, works like a charm!
The insert ignore will make sure that the row exists on both the master and slave, in case they've ever diverted.
The update ... where makes sure that only the most recent update, globally, is the end result after all replication is done.
mysql> desc test;
+-------+--------------+------+-----+-------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+-------------------+-------+
| id | int(11) | NO | PRI | NULL | |
| value | varchar(255) | YES | | NULL | |
| ts | timestamp | NO | | CURRENT_TIMESTAMP | |
+-------+--------------+------+-----+-------------------+-------+
mysql> insert ignore into test values (4, "foo", now());
mysql> update test set value = "foo", ts = now() where id = 4 and ts <= now();
you could use two insert statements .. since you CAN add a where clause to the select part for the source data.
select two sets of data, one that you will insert with 'on duplicate' and the other will be inserted without 'on duplicate'.
Overview
AWUpsertCondy wants to change BEFORE into AFTER
Problem
AWUpsertCondy does not want the insert query to fail if MySQL detects duplicate primary key
MySQL does not support conditional WHERE clause with ON DUPLICATE KEY UPDATE
Solution
MySQL supports conditional clause with the IF() function
Here we have a simple conditional to update only those items with userid less-than 9
INSERT INTO zzdemo_table02
(lname,userid)
SELECT
lname,userid
FROM(
SELECT
lname,userid
FROM
zzdemo_table01
) as tt01
ON DUPLICATE KEY UPDATE
userid=IF(#doupdate:=IF( (tt01.userid < 9) , True, False),
tt01.userid, zzdemo_table02.userid)
,lname=IF(#doupdate, tt01.lname , zzdemo_table02.lname )
;
Pitfalls
We introduce a MySQL variable #doupdate in order to flag whether or not the UPDATE row meets the condition. Then we use that same variable for all the database columns we use in the UPDATE statement
In the first conditional we both declare the variable and determine whether the conditional applies. This approach is arguably more cumbersome than a WHERE clause
See also
MySQL copy column withtout conditional
table php_lock: name:idString, locked:bool,
time:timestamp, locked_by:string values to insert or
update 1, CURRENT_TIMESTAMP, 'script' where name='wwww' AND
locked=2
INSERT INTO `php_lock` (`name`, locked, `time`, `locked_by`)
(SELECT * FROM
(SELECT `name`,1,CURRENT_TIMESTAMP, 'script' FROM `php_lock`
WHERE `name`='wwww' AND `locked`=2
UNION (
SELECT 'wwww',1 ,CURRENT_TIMESTAMP, 'script')
) AS temp LIMIT 1)
ON DUPLICATE KEY UPDATE locked=VALUES(locked), `time`=VALUES(`time`), `locked_by`=VALUES(`locked_by`);
On duplicate key do not allow us to use where clause, so there are two alternative to achieve the same.
If you know most of the time you will get the duplicate key then
a. Update the table first using update query and where clause
b. If update fails then insert the record into table using insert query
If you know most of the time you are going to insert into table then
a. Insert the record into table using insert ignore query - what it does is actually ignore the insert if duplicate key found
b. If insert ignore fails then update the record using update query
For reference of insert ignore click here