I have a SSIS package that copies data from table A to table B and sets a flag in table A so that the same data is not copied subsequently. This works great by using the following as the SQL command text on the ADO Net Source object:
update transfer
set ProcessDateTimeStamp = GetDate(), LastUpdatedBy = 'legacy processed'
output inserted.*
where LastUpdatedBy = 'legacy'
and ProcessDateTimeStamp is not null
The problem I have is that I need to run a similar data copy but from two sources table, joined on a primary / foreign key - select from table A join table B update flag in table A.
I don't think I can use the technique above because I don't know where I'd put the join!
Is there another way around this problem?
Thanks
Rob.
You can use a join in an update statement.
update m
set ProcessDateTimeStamp = GetDate(),
LastUpdatedBy = 'legacy processed',
somefield = t.someotherfield
output inserted.*
from transfer t
join mytable m
on t.id = m.id
where m.LastUpdatedBy = 'legacy'
and m.ProcessDateTimeStamp is null
and t.ProcessDateTimeStamp is not null
The key is to not alias the fields on the left side of the set but to alias everything else. And use the table alias for the table you are updating after the update key word so it knows which table of the join to update.
Related
I trying to update a table with another table of other database, I the last updated I created a function to save more than one category in one product, so, I when a run the old script to update my table, a constraint error appears. I understand the situation and why that is happenning, but how I allow the update with the table with duplicates data? Is there a way to disable the constraint?
My query
UPDATE novourbano.oc_product_to_category oc
INNER JOIN erp_product_category erp ON oc.product_id = erp.erp_productid
SET oc.product_id = erp.erp_productid,
oc.category_id = erp.erp_categoryid
WHERE oc.product_id <> 0
I try to use that:
SET GLOBAL FOREIGN_KEY_CHECKS=0;
But still not working. Any suggestion? Thank's in advance!
If the table name reflects the table purpose the primary key should be
PRIMARY KEY(product_id, category_id)
in order to avoid duplicates like several rows with the same product_id and category_id.
You can use IGNORE for this update:
UPDATE IGNORE novourbano.oc_product_to_category oc
INNER JOIN erp_product_category erp ON oc.product_id = erp.erp_productid
SET oc.product_id = erp.erp_productid,
oc.category_id = erp.erp_categoryid
Any SQL gurus out there who can rewrite the following query:
UPDATE cmsTemplate
SET master = NULL
WHERE master IS NOT NULL
AND master NOT IN (SELECT nodeId
FROM
( SELECT * FROM cmsTemplate a) b
)
So that it does not produce the following error:
You can't specify target table 'cmsTemplate' for update in FROM clause
Issue documented here:
http://dev.mysql.com/doc/refman/5.6/en/update.html
Thanks,
Steve
UPDATE: Description of query
The idea of the query is to do as follows:
SET the master field TO NULL
WHERE the master field IS NOT NULL
AND WHERE the master field IS NOT EQUAL TO ANY of the values for the nodeId field records (within the same table)
You can use update with join:
update cmsTemplate c1 left join cmsTemplate c2
on c1.`master` = c2.nodeId
set c1.`master` = null
where c2.nodeId is null;
I have a table Provision with this structure:
ONT_ID varchar(12) PK
neID set('7360-1','7360-2','7360-3','5000-1','5000-2') not null
and some other crap
I have loaded a temporary table called tempTable that has the same structure and the same data. Prior to trying what I'm trying the neID in the Provision table was a varchar field. The values that were not the same as the set were deleted. (I've done this before without a problem.)
Neither this:
UPDATE Provision P
INNER JOIN tempTable TT ON TT.ONT_ID = P.ONT_ID
SET P.neID = TT.NE_ID
Nor this (broken up for readability):
update Provision P
set P.neID = (
select TT.NE_ID from tempTable TT where TT.ONT_ID = P.ONT_ID
)
...accomplishes what they are supposed to. What is going on?
The Provision table has a record of the ONT_ID and the neID is an empty string. The temp table has the same ONT_ID and a pertinent NE_ID. I'm trying to update the neID in the Provision table with the value that is in the temporary table.
Data in tempTable was wrong. Changed data and updated Provision table.
I'm using MySQL Stored Procedures and I want to insert some rows from a table's database to another table's database through a stored procedure. More specifically from database "new_schema", table "Routers" and field "mac_address" to database "data_warehouse2", table "dim_cpe" and field "mac_address".
This is the code I used in the first insertion, that worked perfectly.
insert into data_warehouse2.dim_cpe (data_warehouse2.dim_cpe.mac_address, data_warehouse2.dim_cpe.ssid)
(select new_schema.Routers.mac_address, new_schema.Routers.ssid from new_schema.Routers, data_warehouse2.dim_cpe);
Now I have more rows in the table "Routers" to be inserted into "dim_cpe" but, since there are rows already there, I want just to insert the new ones.
As seen in other posts, I tried a where clause:
where new_schema.device_info.mac_address != data_warehouse2.dim_cpe.mac_address
and a:
on duplicate key update new_schema.Routers.mac_address = data_warehouse2.dim_cpe.mac_address"
Both didn't work. What's the best way to do this?
Thanks in advance.
You could leave the source table out of the from clause, and use a not exists clause instead:
where not exists
(select mac_address from dim_cpe mac_address = new_schema.Routers.mac_address
and ssid = new_schema.Routers.ssid)
Or you could left join and check whether the fields from dim_cpe are null:
insert into data_warehouse2.dim_cpe
(data_warehouse2.dim_cpe.mac_address, data_warehouse2.dim_cpe.ssid)
(select new_schema.Routers.mac_address, new_schema.Routers.ssid
from new_schema.Routers
left join data_warehouse2.dim_cpe on
new_schema.Routers.mac_address = data_warehouse2.dim_cpe.mac_address
and new_schema.Routers.ssid = data_warehouse2.dim_cpe.ssid
where dim_cpe.mac_address is null and dim_cpe.ssid is null);
Edit to say this is a general SQL solution. I'm not sure if there's a better MySql-specific approach to this.
Edit to show your query:
insert into data_warehouse2.dim_cpe (mac_address, ssid)
select new_schema.Routers.mac_address, new_schema.Routers.ssid
from new_schema.Routers where not exists
(select data_warehouse2.dim_cpe.mac_address from data_warehouse2.dim_cpe
where data_warehouse2.dim_cpe.mac_address = new_schema.Routers.mac_address
and data_warehouse2.dim_cpe.ssid = new_schema.Routers.ssid);
We are currently importing very large CSV files into a mySQL data warehouse. A key part of the processing is to flag whether a record in the CSV file match an existing record in the warehouse. The "match" is done by comparing specific fields in the new data against the previous version of the table. If the record is "new" or if there have been updates, we want to add it to the warehouse.
At the moment the processing plan is as follows :
~ read CSV file into mySQL table A
~ is primary key on A on old-A? If it isnt set record status to "NEW"
~ if key is on old-A, issue update statement , JOINING old-A to A
~ if A.field1 = old-A.field1 OR A.field2 = A.old-A.field2 OR A.field3 = old-A.field3 THEN flag record status as "UPDATE"
~ process NEW or UPDATEd records according to record status
File-size on A and old-A is currently in the order of 50M records. We would expect new records to be 1M, updates to be 5-10M.
Although we are currently using MYSQL for this processing, I am wondering whether it would simply be better to do this using a scripting language? We are finding in particular that the step to flag the updates is very time consuming. Essentially we have an UPDATE statement that is unable to use any indexation.
so
CREATE TABLE A (key1 bigint,
field1 varchar(50),
field2 varchar(50),
field 3 varchar(50) );
LOAD DATA ...
... add field rec_status to table A
... then
UPDATE A
LEFT JOIN old-A ON A.key1 = old-A.key1
SET rec_status = 'NEW'
WHERE old-A.key1 = NULL;
UPDATE A
JOIN old-A ON A.key1 = old-A.key1
SET rec_status = 'UPDATED'
WHERE A.field1 <> old-A.field1
OR A.field2 <> old-A.field2
OR A.field3 <> old-A.field3;
...
I will consider skipping the "flag" step. Process the CSV file using script or MySql table A using MySQL statement, select a record from old-A table base on whatever criteria, such as field1, or/and field2... of table A, if found, lock and update old-A record, delete processed record from CSV or table A. If not found, create record in old-A with data.