I have a situation where I have a table (col1 (pk), col2, col3, col4) and a set of records which I need to insert into a table and on duplicate key update them. I would like to do a batch query to speed things up. However, col4 doesn't have a NOT NULL constraint. The problem arises when I want to update with records (val1, val2, val3, None), (val4, val5, val6, val7). For the first record, I don't want column 4 to be updated (If (val1, val2, val3, val8) existed in DB I wouldn't want to override val8 because None would signify lack of value as opposed to explicit setting to Null). However, for the second record, I would want to update col4 because an explicit value is passed. This would be fine with one record where I would just set the update columns to be col2, col3, and not col4, but I want to batch this query and would need to have col4update when a value is passed for it and not update when I don't have a value. I would logically need something like given below.
INSERT INTO table1
(col1, col2, col3, col4)
VALUES
('val1', 'val2', 'val3'), ON DUP KEY UPDATE col2, col3
('val5', 'val6', 'val7', 'val8'), ON DUP KEY UPDATE col2, col3, col4
('val9', 'val10', 'val11') ON DUP KEY UPDATE col2, col3
Clearly this can be done by just making it a series of separate statements, but I would like to find a way to batch this. Is there any way this, or a different method, can be done in sql?
In the on duplicate key update part of the insert, you can refer to the inserted values with values. You can use coalesce to preserve the pre-update value in case of null:
INSERT INTO YourTable (col1, col2, col3, col4) VALUES
('val1', 'val2', 'val3', null)
, ('val5', 'val6', 'val7', 'val8')
, ('val9', 'val10', 'val11', null)
ON DUPLICATE KEY UPDATE
col1 = values(col1)
, col2 = values(col2)
, col3 = values(col3)
, col4 = coalesce(values(col4), col4)
Example on SQL Fiddle.
In reply to your comment, you can set null explicitly with a case:
, col4 = case values(col4)
when 'None' then null
else coalesce(values(col4), col4)
end
The obvious risk here is that you can no longer update to None :)
The number of fields in VALUES must be the same as in INSERT. But you can just pass NULL for the col4 and use COALESCE in the UPDATE part.
INSERT INTO table1
(col1, col2, col3, col4)
VALUES
('val1', 'val2', 'val3', NULL),
('val5', 'val6', 'val7', 'val8'),
('val9', 'val10', 'val11', NULL)
ON DUPLICATE KEY UPDATE
col2 = VALUES(col2),
col3 = VALUES(col3),
col4 = COALESCE(VALUES(col4), col4)
Is this the thing that you are looking for?
INSERT INTO table1
(col1, col2, col3, col4)
VALUES
('val1', 'val2', 'val3', null)
('val5', 'val6', 'val7', 'val8')
('val9', 'val10', 'val11', null)
ON DUPLICATE KEY UPDATE
col2 = values(col2),
col3 = values(col3),
col4 = coalesce(values(col4), col4)
;
Related
I need to implement concurrent-safe UPSERT using a non-unique key and avoid unnecessary auto-increment of ID.
Traditional INSERT ... ON DUPLICATE KEY doesn't work for me, so I'm performing:
INSERT INTO table (col1, col2, col3, col4, col5)
SELECT 1, 2, 'value3', 'value4', 'value5'
WHERE NOT EXISTS (SELECT 1
FROM table
WHERE col3 = 'value3'
AND col4 = 'value4'
AND col5 = 'value5')
then if it results in no row inserted, I'm performing:
UPDATE table
SET col1 = col1 + 1,
col2 = MAX(col2, 2)
WHERE col3 = 'value3'
AND col4 = 'value4'
AND col5 = 'value5'
There's an index:
CREATE INDEX ON table (col3, col4, col5)
It is non-unique as there are legacy data that does not allow me to declare it unique. Newer records, however, should not have duplicated (col3, col4, col5) rows.
Unsurprisingly, using the given INSERT statement I'm getting mixed results trying to execute it concurrently from two sessions. I can see the second session blocking until the first one commits its transaction, but then the second transaction is also able to insert a new row sometimes (or sometimes it achieves the expected of avoiding to insert a duplicate (col3, col4, col5) row).
I'm currently performing manual unique-check after the insert:
SELECT COUNT(1)
FROM table
WHERE col3 = 'value3'
AND col4 = 'value4'
AND col5 = 'value5'
but I've also tried:
INSERT INTO table (col1, col2, col3, col4, col5)
SELECT 1, 2, 'value3', 'value4', 'value5'
WHERE NOT EXISTS (SELECT 1
FROM table
WHERE col3 = 'value3'
AND col4 = 'value4'
AND col5 = 'value5'
FOR UPDATE)
which appears to work with the examples I'm always getting a duplicate (col3, col4, col5) row, otherwise. Is the given FOR UPDATE usage reliable for the purpose of ensuring no duplicate (col3, col4, col5) row will be inserted?
I'm using READ-COMMITTED transaction isolation.
MySQL 8.0.13 and higher supports functional key parts that index expression values rather than column or column prefix values. (link)
Because you have a unique field, i am assuming this is col, you can add an index like:
CREATE unique INDEX idx2 ON `mytable` ((col1>42),col3, col4, col5);
Where 42 should be the next auto-increment for col1.
Newly create records will be unique on the 3 columns, without affecting your 'old' data.
It is even possible to update the old data (as long as col1<=42).
I am trying to find a solution to update or insert value in my MySQL table (like in firebird you have UPDATE OR INSERT INTO TABLE (COL1, COL2, COL3) VALUES (VAL1, VAL2, VAL3) MATCHING (COL1, COL2).
I know MySQL have on duplicate key but I do not want that! I do not want to set a key on some of my columns because that one table is used depending on the situation.
So is there any other way of performing update or insert action?
if I remember correctly COL1 should be unique
INSERT INTO table
(COL1, COL2, COL3)
VALUES
(?, ?, ?)
ON DUPLICATE KEY UPDATE
COL2 = VALUES(COL2)
COL2 = VALUES(COL2 )
I have a table named Table1 with three columns col1, col2, col3.
col1 can have one of the three values(1 or 2 or 3).
I need to add a check constraint such that it checks, if col1 has value 1 then the values of col2 and col3 should be same else if col1 has values 2 or 3, then col2 and col3 values may or may not be same.
Can anyone tell me how to add the constraint for this ?
Thanks in advance.
You can add a check constraint like this:
ALTER TABLE Table1 ADD CONSTRAINT chk_table1_cols
CHECK ( (col1 = 1 AND col2 = col3) OR (col1 IN (2, 3)) );
Note that "is the same" presumes that the values are not NULL. That logic can be added, if you want to consider NULL values as equal.
You can express these condition with a series of logical operators:
(col IN (1, 2, 3)) AND (col2 = col3 OR col1 IN (2, 3)
I didn't verify (I don't have MySql) but in Oracle the following works:
ALTER TABLE Table1 ADD
(
CONSTRAINT CHK_1 CHECK ((col1 BETWEEN 1 AND 3) AND (col1 <> 1 OR col2=col3))
)
I have a table with lets say 4 fields.
table:
id(autoincremental), col1, col2, col3
There are several rows with data for id, col1 and col2.
Col3 is empty.
I would like to fill col3 with values, in the existing rows with one query like this:
INSERT INTO table(id, col3)
VALUES
(1, 'value1'),
(2, 'value2'),
(3, 'value3'),
...
ON DUPLICATE KEY UPDATE
id = VALUES(id),
col3 = VALUES(col3);
But I get an error because col1 and col2 don't have a default value.
I just want to update col3, and preserve other column values.
How can I do?
You can use a simple update statement :
Update yourtable t
Set t.col3 = case when t.id = 1 then ‘value1’
case when t.id = 2 then ‘value2’
.....
else t.col3 end;
And you can also filter the desired IDs for better performance
I believe that #Lukasz Szozda comment is the best approach for this issue. So I'll choose it as a solution. Thanks.
Mark col1, col2 as nullable.
You can update like this :
ON DUPLICATE KEY UPDATE
col3 = VALUES(col3);
This will keep the current values for the fields that are not in the update statement.
I read the question again and it seems like your insert would need col1 and col2 as well. You said that they don't have a default value, so I would either
add a default value for each
mark them as nullable
make these parameters mandatory in your application.
So you can't change your table structure my final query would be the following :
INSERT INTO table(id, col1, col2, col3) VALUES
(1, '', '', 'value1'),
(2, '', '', 'value2'),
(3, '', '', 'value3')
-- ...
ON DUPLICATE KEY UPDATE
col1 = col1,
col2 = col2,
col3 = VALUES(col3);
When a field does not have a default value and is not nullable, the insert query must include a value for that field. So your problem is not in the update part of your query, it is in the insert part.
How can I at an insert query, check if a specific column has a specific value and then update the row. Otherwise it should insert a new row.
Somethings like this is what I want:
INSERT INTO table (col1, col2, col3) VALUES (val1, val2, val3) UPDATE IF col3 = $number
Create a UNIQUE key over your col3:
ALTER TABLE `table` ADD UNIQUE KEY (col3)
Then use INSERT ... ON DUPLICATE KEY UPDATE:
INSERT INTO `table` (col1, col2, col3) VALUES (val1, val2, val3)
ON DUPLICATE KEY UPDATE col1 = VALUES(col1), col2 = VALUES(col2)
Have a look at the REPLACE INTO command (MySQL docu).
REPLACE INTO table (col1, col2, col3) VALUES (val1, val2, val3);
It checks primary keys and unique indexes and when there is a matching entry already present, that row is replaced by the new one, else a new row is inserted.
EDIT
As #eggyal mentioned this the behaviour in the case, when the row is replaced, is actually a deletion of the old row and insertion of the new row afterwards. This may lead to some problems, when you're using triggers, foreign keys or alike.
Actually you might loose the content of some columns as well. Suppose you have a table with 3 columns (col1 to col3), but the REPLACE just sets two of them (col1, col2), the third one col3 will receive the default value specified and not retain the old value.
if you have unique index on that column then use this
INSERT INTO table (col1, col2, col3) VALUES (val1, val2, val3)
ON DUPLICATE KEY UPDATE col3 = $number