SQL Syntax for checking duplicates prior to update (not DUPLICATE KEY UPDATE) - mysql

I have a syntactical question with attempting an UPDATE if 2 non key fields are matched. -- INSERT if not matched.
Let me start by saying I have a working query that involves a SELECT with an ON DUPLICATE KEY UPDATE. Now I am just curious, can it be done differently?
Out of sheer curiosity, I am trying this in a manner that will not require the primary key. This really is just an experiment to see if it can be done.
What I want is like ON DUPLICATE KEY UPDATE -- However:
let's pretend I don't know the key , and
let's pretend I can't get the key with a SELECT
Here is the data structure I have:
+--------------------------------------------------------------------------+
| id | contractor_id | email_type_id | email_address |
+--------------------------------------------------------------------------+
Table creation:
CREATE TABLE `email_list` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`contractor_id` int(11) NOT NULL,
`email_type_id` int(11) NOT NULL,
`email_address` varchar(45) COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id_UNIQUE` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=13 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Now what I am trying to do without a select and without ON DUPLICATE KEY UPDATE is -- If contractor_id and email_type_id are matched -- UPDATE the email_address -- else INSERT.
If have tried this -- (I know I am breaking my own rule of no SELECT):
IF EXISTS (SELECT * FROM email_list WHERE contractor_id = 1166)
UPDATE email_list SET (email_address='herky_jerky#snailmail.com')
WHERE contractor_id = 1166 AND email_type_id = 4
ELSE
INSERT INTO email_list VALUES (
contractor_id = 1166,
email_type_id = 4,
email_address = 'herky_jerky#snailmail.com');
I understand why this doesn't work .. I just don't know what the fix for it is -- It feels a little clunky too using an IF - ELSE statement. Also I don't want to use a SELECT -- So then I thought about using just an IF like:
UPDATE email_list SET email_address = 'herky#jerky.com'
WHERE contractor_id = 1166 AND email_type_id = 4
IF ##ROWCOUNT=0
INSERT INTO email_list VALUES (
contractor_id = 1166,
email_type_id = 4,
email_address = 'herky#jerky.com');
But I don't understand why that one doesn't work. This is just an exercise to see how creative one can be with this type of query. I think both of my ideas are doable -- Can anyone find a fix for either query to make it work?
I'd also love to see other, more creative, ways of attempting what I am asking as well!

I'd implement this using an UPDATE followed by a test of ROW_COUNT() and if no rows were updated, then INSERT.
drop table if exists t;
create table t (id int, x int, y int, str varchar(255));
insert into t (id, x, y, str) values (1, 2, 3, 'foo');
select * from t;
update t set str = 'bar'
where x = 2 and y = 3;
insert into t (id, x, y, str)
select 1, 2, 3, 'inserted'
from dual
where row_count() = 0;
select * from t;
update t set str = 'baz'
where x = 20 and y = 30;
insert into t (id, x, y, str)
select 10, 20, 30, 'baz'
from dual
where row_count() = 0;
select * from t;
drop table t;
You can see it in action here: https://rextester.com/FRFTE79537
The idea here is you do the UPDATE first, followed by an INSERT ... SELECT where the SELECT only returns a row if ROW_COUNT() = 0 is true, and that's only true if the UPDATE didn't match any rows.

Related

MySQL Upsert with Insert Only Column Value

In MySQL, is it possible to do an upsert but only set a column value on insert (and not set the column value on update).
For example, for a createdBy column, we only want to set the value on insert, we don't want to override that value on update (because we lose who originally inserted the column).
Note that we only know the currently logged in user. So updatedBy is simple -- always use the value of the logged in user. But createdBy is hard. Use the value of the logged in user but only for an insert -- don't override this on update.
Example schema:
CREATE TABLE `movie` (
`id` INT AUTO_INCREMENT PRIMARY KEY,
`name` NVARCHAR(100) NOT NULL,
`createdBy` NVARCHAR(100) NOT NULL,
`updatedBy` NVARCHAR(100) NOT NULL,
UNIQUE INDEX (`name`)
);
Example of a standard upsert:
INSERT INTO `movie` (`name`, `createdBy`, `updatedBy`)
VALUES ('The Matrix', 'Jill', 'Jill')
ON DUPLICATE KEY UPDATE
`id` = LAST_INSERT_ID(`id`),
`name` = VALUES(`name`),
`createdBy` = VALUES(`createdBy`),
`updatedBy` = VALUES(`updatedBy`)
;
Here's my attempt to only set the createdBy column on insert using IFNULL. But this doesn't work and results in createdBy always being null.
INSERT INTO `movie` (`name`, `createdBy`, `updatedBy`)
VALUES ('The Matrix', IFNULL(`createdBy`, 'Jill'), 'Jill')
ON DUPLICATE KEY UPDATE
`id` = LAST_INSERT_ID(`id`),
`name` = VALUES(`name`),
`createdBy` = VALUES(`createdBy`),
`updatedBy` = VALUES(`updatedBy`)
;
Results wanted:
Case 1: Jill runs an upsert that inserts a row.
id = 1
name = 'The Matrix'
createdBy = 'Jill' // Created by Jill
updatedBy = 'Jill' // Last updated by Jill
Case 2: Bob runs an upsert that updates the same row.
id = 1
name = 'The Matrix Reloaded'
createdBy = 'Jill' // Created by Jill (do not change value on update)
updatedBy = 'Bob' // Last updated by Bob
I created a fiddle guessing that Name is the Key, feel free to give it a try here.
This is the basic syntax:
INSERT INTO `movie` (`name`, `UpdatedBy`,`CreatedBy`)
VALUES ('Star wars', 'NameA','NameB')
ON DUPLICATE KEY UPDATE `UpdatedBy` = VALUES(`UpdatedBy`)
;
Notice: NameA and NameB can be the same so you dont get nulls on inserts
Hope it helps :)
Try this:
INSERT INTO `movie` (`name`, `createdBy`)
VALUES ('The Matrix', 'Jill')
ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)
;

inserting bulk values and updating existing entries on the database [duplicate]

I'm pushing data from a data-frame into MySQL, right now it is only adding new data to the table if the data does not exists(appending). This works perfect, however I also want my code to check if the record already exists then it needs to update. So I need it to append + update. I really don't know how to start fixing this as I got stuck....someone tried this before?
This is my code:
engine = create_engine("mysql+pymysql://{user}:{pw}#localhost/{db}"
.format(user="root",
pw="*****",
db="my_db"))
my_df.to_sql('my_table', con = engine, if_exists = 'append')
You can use next solution on DB side:
First: create table for insert data from Pandas (let call it test):
CREATE TABLE `test` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`name` VARCHAR(100) NOT NULL,
`capacity` INT(11) NOT NULL,
PRIMARY KEY (`id`)
);
Second: Create table for resulting data (let call it cumulative_test) exactly same structure as test:
CREATE TABLE `cumulative_test` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`name` VARCHAR(100) NOT NULL,
`capacity` INT(11) NOT NULL,
PRIMARY KEY (`id`)
);
Third: set trigger on each insert into the test table will insert ore update record in the second table like:
DELIMITER $$
CREATE
/*!50017 DEFINER = 'root'#'localhost' */
TRIGGER `before_test_insert` BEFORE INSERT ON `test`
FOR EACH ROW BEGIN
DECLARE _id INT;
SELECT id INTO _id
FROM `cumulative_test` WHERE `cumulative_test`.`name` = new.name;
IF _id IS NOT NULL THEN
UPDATE cumulative_test
SET `cumulative_test`.`capacity` = `cumulative_test`.`capacity` + new.capacity;
ELSE
INSERT INTO `cumulative_test` (`name`, `capacity`)
VALUES (NEW.name, NEW.capacity);
END IF;
END;
$$
DELIMITER ;
So you will already insert values into the test table and get calculated results in the second table. The logic inside the trigger can be matched for your needs.
Similar to the approach used for PostgreSQL here, you can use INSERT … ON DUPLICATE KEY in MySQL:
with engine.begin() as conn:
# step 0.0 - create test environment
conn.execute(sa.text("DROP TABLE IF EXISTS main_table"))
conn.execute(
sa.text(
"CREATE TABLE main_table (id int primary key, txt varchar(50))"
)
)
conn.execute(
sa.text(
"INSERT INTO main_table (id, txt) VALUES (1, 'row 1 old text')"
)
)
# step 0.1 - create DataFrame to UPSERT
df = pd.DataFrame(
[(2, "new row 2 text"), (1, "row 1 new text")], columns=["id", "txt"]
)
# step 1 - create temporary table and upload DataFrame
conn.execute(
sa.text(
"CREATE TEMPORARY TABLE temp_table (id int primary key, txt varchar(50))"
)
)
df.to_sql("temp_table", conn, index=False, if_exists="append")
# step 2 - merge temp_table into main_table
conn.execute(
sa.text(
"""\
INSERT INTO main_table (id, txt)
SELECT id, txt FROM temp_table
ON DUPLICATE KEY UPDATE txt = VALUES(txt)
"""
)
)
# step 3 - confirm results
result = conn.execute(
sa.text("SELECT * FROM main_table ORDER BY id")
).fetchall()
print(result) # [(1, 'row 1 new text'), (2, 'new row 2 text')]

Allow only unique combination of values for a composite primary key

I created the following table called books:
CREATE TABLE IF NOT EXISTS `books` (
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
`name` TEXT NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
I created another tablet called compare to compare any 2 books from the books table:
CREATE TABLE IF NOT EXISTS `compare` (
`id_1` BIGINT UNSIGNED NOT NULL,
`id_2` BIGINT UNSIGNED NOT NULL,
PRIMARY KEY (`id_1`,`id_2`),
FOREIGN KEY (`id_1`) REFERENCES books(`id`),
FOREIGN KEY (`id_2`) REFERENCES books(`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The previous works as expected but I need to force MYSQL to allow ONLY unique combination of values in the compare table.
For example if I have the following row in compare table:
id_1 | id_2
------------
1 | 2
I want to force MYSQL to NOT allow me to insert the following row:
id_1 | id_2
------------
2 | 1
So I want MYSQL to allow only either 1,2 or 2,1 NOT both.
Another way would be to swap the values in a before-insert trigger to make sure id_1 < id_2:
CREATE TRIGGER `compare_before_insert` BEFORE INSERT ON `compare` FOR EACH ROW
BEGIN
DECLARE t BIGINT;
IF(NEW.id_1 > NEW.id_2) THEN
SET t = NEW.id_1;
SET NEW.id_1 = NEW.id_2;
SET NEW.id_2 = t;
END IF;
END
This will allow you to use the key to make the pairs unique.
In MySQL (at least prior to version 8), you need to do this using a trigger if you want the database to enforce the constraint.
delimiter $$
create trigger compare_insert_trigger before insert on compare
for each row
begin
if (exists (select 1
from compare c
where c.id2 = new.id1 and c.id1 = new.id2
)
) then
signal sqlsate '45000'
set message_text = 'Book pair already exists in compare';
end if;
end;$$
delimiter ;
Of course, you still need the unique constraint.

inner join on null condition : bug or feature?

Here is the test setup:
CREATE TABLE A (
id bigint NOT NULL AUTO_INCREMENT,
value bigint,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO A (id, value) VALUES (1, 22);
INSERT INTO A (id, value) VALUES (2, 25);
INSERT INTO A (id, value) VALUES (3, 25);
CREATE TABLE B (
id bigint NOT NULL AUTO_INCREMENT,
value bigint,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Important note: table B does not contain any row!!!
Test query:
select * from A inner join B on (A.value=25 OR B.value=null);
Surprise: Empty result set fetched
If table B contain anything like:
INSERT INTO B (id, value) VALUES (3, 66);
Then the same query will return 2 rows:
id value id value
-- ----- -- -----
2 25 3 66
3 25 3 66
This is a bug or a feature of MySQL?
INNER JOIN by definition returns matching records only. If a table does not have any records, then there cannot be any metching record. This is a standard behaviour across all RDBMs. Use left or right join instead of inner if you want to return rows from a table regardless of matching rows from another.
select * from A left join B on ... where A.value=25 ;
Moreover, anything=NULL comparison will always retur false, because NULL does not equal to anything, not even to another NULL value. If you want to test if a field has a value of NULL, then use fieldname IS NULL expression.

How to use INSERT... ON DUPLICATE?

Structure table:
CREATE TABLE IF NOT EXISTS `table` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`rows_id` int(11) NOT NULL,
`url_id` int(11) NOT NULL,
`keyword_id` int(11) NOT NULL,
`date` datetime NOT NULL,
`seet` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=7 ;
STRUCTURE TABLE WITH TEST ROWS ON SQLFIDDLE
query insert:
INSERT INTO `table` (`id`, `rows_id`, `url_id`, `keyword_id`, `date`, `seet`)
VALUES
(1, 1, 2, 1, '2014-05-01 00:00:00', 1);
I would like insert this row or update row if table already have row with date = '2014-05-01 00:00:00' and rows_id = '1' and keyword_id = '1'.
In dev.mysql.com i see query INSERT ... ON DUPLICATE KEY UPDATE but how make insert or update in my case ?
INSERT...ON DUPLICATE KEY UPDATE (IODKU) does an update only if the values you insert conflict with an existing row based on a PRIMARY KEY or UNIQUE KEY.
So you need to define a UNIQUE KEY over the three columns or else IODKU can't tell if the row conflicts with an existing one.
ALTER TABLE `table` ADD UNIQUE KEY (date, rows_id, keyword_id);
You said in a comment above that you can have more than one row with the same rows_id. That's fine -- the unique constraint says that you can't have two rows that duplicate the same combination of three values in date, rows_id, and keyword_id. But you can have multiple rows that have the same value in any one of those columns, as long as the combination of the three is unique.
Here's a quick demo:
mysql> create table t (
id int auto_increment primary key,
d int,
r int,
k int,
unique key(d,r,k)
);
mysql> insert into t values (1,1,1,1);
Query OK, 1 row affected (0.01 sec)
mysql> insert into t values (1,1,1,1) on duplicate key update d = values(d)+1;
Query OK, 2 rows affected (0.00 sec)
mysql> select * from t;
+----+------+------+------+
| id | d | r | k |
+----+------+------+------+
| 1 | 2 | 1 | 1 |
+----+------+------+------+
From what I'm understanding from your question, this might help. This SQL Query will test if a row exists and update the row or create a new one based on whether or not that row already exists.
IF EXISTS (SELECT * FROM table WHERE date = 2014-05-01 00:00:00 AND rows_id = 1 AND keyword_id = 1)
BEGIN
UPDATE table SET Params=values
END
ELSE
BEGIN
INSERT INTO `table` (`id`, `rows_id`, `url_id`, `keyword_id`, `date`, `seet`) VALUES (1, 1, 2, 1, '2014-05-01 00:00:00', 1)
END
If the row exists (if the SELECT statement returns data) the UPDATE query will run. If the row does not exist (SELECT statement returns no data) then the INSERT INTO statement will run.