I am trying to fetch customer id from ":ao.mos:pro:%" and update corresponding doc ":pd:pro:%", we have thousands of ":ao.mos:pro:%" docs and corresponding ":pd:pro:%" for each mo:pro doc.
I am using below query and it works but it updates only one doc it can find indexes.
MERGE INTO `bucket1` AS d
USING `bucket1` AS p
ON d.icc = p.icc AND META(d).id LIKE ':pd:pro:%' AND META(p).id LIKE ':ao.mos:pro:%'
WHEN MATCHED THEN
UPDATE SET d.customerId = p.customerId;
Any suggestions on how to get this working for all matching docs in the bucket.
The following works.
CREATE INDEX ix11 ON `default` (icc, customerId) WHERE META().id LIKE ":ao.mos:pro:%";
CREATE INDEX ix12 ON `default` (icc, customerId) WHERE META().id LIKE ":pd:pro:%";
INSERT INTO default VALUES(":ao.mos:pro:1", {"icc":1, "customerId":100});
INSERT INTO default VALUES(":ao.mos:pro:2", {"icc":2, "customerId":101});
INSERT INTO default VALUES(":pd:pro:1", {"icc":1});
INSERT INTO default VALUES(":pd:pro:2", {"icc":1});
INSERT INTO default VALUES(":pd:pro:3", {"icc":2});
MERGE INTO `default` AS m
USING (SELECT p1.icc, p1.customerId
FROM `default` AS p1
WHERE META(p1).id LIKE ':ao.mos:pro:%' AND p1.icc IS NOT NULL) AS p
ON m.icc = p.icc AND META(m).id LIKE ':pd:pro:%'
WHEN MATCHED THEN
UPDATE SET m.customerId = p.customerId;
MERGE INTO `default` AS m
USING default AS p
ON m.icc = p.icc AND META(m).id LIKE ':pd:pro:%' AND META(p).id LIKE ':ao.mos:pro:%'
WHEN MATCHED THEN
UPDATE SET m.customerId = p.customerId;
Are you having following issue:
First source row matches 2 target rows. Second source matches same target row (by changing icc value to 1 of document key ":ao.mos:pro:2") then returns error (5320 Multiple UPDATE/DELETE of the same document (document key 'xxx') in a MERGE statement) (as with partial updated rows, You need transactions don't want update anything) as you can't update same row again. To resolve that you need to change your ON clause.
If it is a one off operation you can use Eventing Service and create a function with a bucket alias of "src_bkt" to map to bucket1 in r+w mode and the deployment feed boundary set to "From Everything" can easily do 100's of millions of items quickly without indexes.
function OnUpdate(doc,meta) {
// Filter out all non-interesting items
if (!meta.id.startsWith(":pd:pro:")) return;
// No need to do anything already updated.
if (doc.customerId)return;
var myint = meta.id.substring(8);
var otherkey = ":ao.mos:pro:" + myint;
var otherdoc = src_bkt[otherkey];
if (otherdoc) {
if (doc.icc === otherdoc.icc ) {
// update the field as we have a match and it is missing.
doc.customerId = otherdoc.customerId;
src_bkt[meta.id] = doc;
}
}
}
So now you just insert some test documents using the QWB
INSERT INTO bucket1 VALUES(":ao.mos:pro:1", {"icc":1, "customerId":100});
INSERT INTO bucket1 VALUES(":ao.mos:pro:2", {"icc":2, "customerId":101});
INSERT INTO bucket1 VALUES(":pd:pro:1", {"icc":1});
INSERT INTO bucket1 VALUES(":pd:pro:2", {"icc":1});
INSERT INTO bucket1 VALUES(":pd:pro:3", {"icc":2});
Then inspect the documents you should have
:ao.mos:pro:1{"customerId":100,"icc":1}
:ao.mos:pro:2{"customerId":101,"icc":2}
:pd:pro:1{"icc":1}
:pd:pro:2{"icc":1}
:pd:pro:3{"icc":2}
Deploy the Eventing function and look at your documents again.
:ao.mos:pro:1{"customerId":100,"icc":1}
:ao.mos:pro:2{"customerId":101,"icc":2}
:pd:pro:1{"icc":1,"customerId":100}
:pd:pro:2{"icc":1}
:pd:pro:3{"icc":2}
As expected only one doc with matching keys and matching icc properties updated.
For more performance (doing millions) you can up the workers in the functions settings to 2x the physical cores.
I am trying to use 'ON DUPLICATE KEY UPDATE' statement to update column when duplicate primary key are already in table.
but even if table has duplicated primary key, it does not update column.
below is 'ON DUPLICATE KEY UPDATE' statement.
is there somthing wrong?
ON DUPLICATE KEY UPDATE authenticated = authenticated
and notAuthenticated = notAuthenticated
and stoped = stoped
and deleted = deleted
and updatedDate = now()
;
use VALUES(Column) and replace all this AND with comma ,:
ON DUPLICATE KEY UPDATE authenticated = VALUES(authenticated),
notAuthenticated = VALUES(notAuthenticated),
stoped = VALUES(stoped),
deleted = VALUES(deleted),
updatedDate = now()
Don't use AND, but use commas:
ON DUPLICATE KEY UPDATE
authenticated = VALUES(authenticated),
notAuthenticated = VALUES(notAuthenticated),
stoped = VALUES(stoped),
deleted = VALUES(deleted),
updatedDate = now()
How do i update only if a second condition is true, it seems my version of mysql doesnt allow a WHERE at the end of ON DUPLICATE syntax.
INSERT INTO `proxies` (`proxy`,`response`,`PAYMENT`,`type`,`country`,`status`,`tier`,`last_checked`,`last_active`,`response_time`)
VALUES ('111.9.204.96:8123','200','coolproxies','anon','China','active','3','1400624136','1400624136','1.577639')
ON DUPLICATE KEY UPDATE `response`='200',`response_time`='1.577639',`type`='anon',`country`='China',`status`='active',`tier`='2',`last_checked`='1400624137'
this works ok, but I need to only update when WHERE last_checked < '1400624137' is true.
This is what the query that does this looks like.
INSERT INTO `proxies` (`proxy`,`response`,`PAYMENT`,`type`,`country`,`status`,`tier`,`last_checked`,`last_active`,`response_time`)
VALUES ('207.204.249.193:21320','200','scanner','anon','United States','active','1','1400633866','1400633866','1.59696')
ON DUPLICATE KEY UPDATE
`response_time` = IF(`last_checked` < '1400633866', '1.59696', `response_time`),
`status` = IF(`last_checked` < '1400633866', 'active', `status`),
`last_checked` = IF(`last_checked` < '1400633866', '1400633866', `last_checked`),
`last_active` = IF(`last_checked` < '1400633866', '1400633866', `last_active`);
An alternative to doing it in 2 queries as #GordonLinoff mentioned, is to use the method described in this related question you can use an IF to update based on a condition.
INSERT INTO `proxies` (...)
VALUES (...)
ON DUPLICATE KEY UPDATE
my_column = IF(last_checked < '1400624137', VALUES(my_column), my_column)
Basically what the IF does is when TRUE then update the field to the new value, otherwise if FALSE set it to the current value, which means no change to your existing data.
I think you need to do this as two statements:
update . . .
where last_checked < '1400624137';
insert ignore into proxies(. . .);
(or use on duplicate key update to do nothing).
I have a complex query with a ON DUPLICATE KEY Update inside. I only want to insert the last value if there is no row with “timestamp_dag” = 1420070400, if there is a row with that condition I want to do nothing.
INSERT INTO data_prijzen_advertentie (
`ID_advertentie`,`jaar`,`rijnr`,`status_prijs`,`datum_dag`,`timestamp_dag`,
`prijs_maand`,`prijs_week`,`prijs_midweek`,`prijs_langweekend`,`prijs_weekend`,
`prijs_dag`,`prijs_ochtend`,`prijs_middag`
)
VALUES
(100,2014,1,1,'12-05-2014',1399852800,0,100,0,75,0,0,0,0),
(100,2014,2,1,'23-05-2014',1400803200,0,75,0,101,0,0,0,0),
(100,2014,3,1,'30-05-2014',1401408000,0,100,0,75,0,0,0,0),
(100,2014,4,1,'01-01-2015',1420070400,0,0,0,0,0,0,0,0)
ON DUPLICATE KEY UPDATE
status_prijs = VALUES(status_prijs), datum_dag = VALUES(datum_dag),
timestamp_dag = VALUES(timestamp_dag), prijs_maand = VALUES(prijs_maand),
prijs_week = VALUES(prijs_week), prijs_midweek = VALUES(prijs_midweek),
prijs_langweekend = VALUES(prijs_langweekend), prijs_weekend = VALUES(prijs_weekend),
prijs_dag = VALUES(prijs_dag), prijs_ochtend = VALUES(prijs_ochtend),
prijs_middag = VALUES(prijs_middag);
it's the best if you can handle this before the query string creation..
but if you can't, and your exceptional value is static, you can try something like this:
(I haven't tried to run it yet though)
INSERT INTO data_prijzen_advertentie (
`ID_advertentie`,`jaar`,`rijnr`,`status_prijs`,`datum_dag`,`timestamp_dag`,
`prijs_maand`,`prijs_week`,`prijs_midweek`,`prijs_langweekend`,`prijs_weekend`,
`prijs_dag`,`prijs_ochtend`,`prijs_middag`
)
VALUES
(100,2014,1,1,'12-05-2014',1399852800,0,100,0,75,0,0,0,0),
(100,2014,2,1,'23-05-2014',1400803200,0,75,0,101,0,0,0,0),
(100,2014,3,1,'30-05-2014',1401408000,0,100,0,75,0,0,0,0),
(100,2014,4,1,'01-01-2015',1420070400,0,0,0,0,0,0,0,0)
ON DUPLICATE KEY UPDATE
status_prijs = IF(timestamp_dag<>1420070400, VALUES(status_prijs), status_prijs),
datum_dag = IF(timestamp_dag<>1420070400, VALUES(datum_dag), datum_dag),
timestamp_dag = IF(timestamp_dag<>1420070400, VALUES(timestamp_dag), timestamp_dag),
prijs_maand = IF(timestamp_dag<>1420070400, VALUES(prijs_maand), prijs_maand),
prijs_week = IF(timestamp_dag<>1420070400, VALUES(prijs_week), prijs_week),
prijs_midweek = IF(timestamp_dag<>1420070400, VALUES(prijs_midweek), prijs_midweek),
prijs_langweekend = IF(timestamp_dag<>1420070400, VALUES(prijs_langweekend), prijs_langweekend),
prijs_weekend = IF(timestamp_dag<>1420070400, VALUES(prijs_weekend), prijs_weekend),
prijs_dag = IF(timestamp_dag<>1420070400, VALUES(prijs_dag), prijs_dag),
prijs_ochtend = IF(timestamp_dag<>1420070400, VALUES(prijs_ochtend), prijs_ochtend),
prijs_middag = IF(timestamp_dag<>1420070400, VALUES(prijs_middag), prijs_middag);
You will only have the ON DUPLICATE KEY UPDATE ... portion run if the timestamp_dag matches the criteria for triggering it. From the MySQL docs:
If you specify ON DUPLICATE KEY UPDATE, and a row is inserted that
would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an
UPDATE of the old row is performed.
So assuming your table is built this way, then the update will trigger.
You can add a UNIQUE index using the syntax as described here.
CREATE UNIQUE INDEX ind_unique_timestamp_dag ON data_prijzen_advertentie (timestamp_dag)