I'm synchronizing two applications and ERP and CRM. We have one process that runs in real time with triggers and second method that runs as a batch. When I run the batch I need to check for any missed records in the event that a batch didn't run. I've used a process yes/no column at times. But need a way to determine how to check that the insert completed and then set the processed column to yes. It must be something like if insert completes then set processed column to Y. And is it better to use a cursor and go one insert at a time or run the insert statement for the batch table?
SET IDENTITY_INSERT CONTACTS ON
INSERT INTO CONTACTS
(CODE, ID, CODE_ID, FIRSTNAME, MIDDLENAME, LASTNAME, ADDR, DECEASED_FLAG, CREATEDATE, CREATETIME, IDSeq)
SELECT CC.CODE, CC.ID, CC.CODE_ID, CC.FIRSTNAME, CC.MIDDLENAME, CC.LASTNAME, CC.CREATE_DATE, CC.CREATE_TIME, CC.IDSeq
FROM CONTACTSCHANGES AS CC INNER JOIN
CONTACTS AS C ON CC.ID = C.ID
SET IDENTITY_INSERT CONTACTS OFF
UPDATE CONTACTSCHANGES
SET Processed = N'Y'
Related
i had query like this
CREATE TRIGGER `tambah_riwayatobat` AFTER INSERT ON `obat`
FOR EACH ROW insert into riwayat_obat(nama, keterangan, distributor,tanggal)
(select new.nama, 'Masuk', d.nama ,now()
From distributor d
join obat ON new.id_distributor = d.id_distributor)
i try to insert data with trigger and one of part data i fetch with constraint, but why the data be duplicate entry ?
Output :
example, if i try to insert data obat 1st time, data on tambah_riwayatobat insert 1 too
if i try to insert data obat 2nd time, data on tambah_riwayatobat insert 2 times with same data
if i try to insert data obat 3rd time, data on tambah_riwayatobat insert 3 times with same data
I'm not certain exactly what's happening, but it's a result of the join in your trigger code. You’re joining obat to distributor, but your join condition makes no mention of obat so you're getting some sort of cross-product where on the second and subsequent INSERT your SELECT subquery is selecting more than one row.
You shouldn't (and don't need to) use the join, since all the data you need from obat is already in the pseudorecord NEW. The following code should work much better:
CREATE TRIGGER `tambah_riwayatobat`
AFTER INSERT ON `obat`
FOR EACH ROW
INSERT INTO riwayat_obat
(nama, keterangan, distributor, tanggal)
(SELECT NEW.nama, 'Masuk', d.nama, now()
FROM distributor d
WHERE new.id_distributor = d.id_distributor
LIMIT 1);
The LIMIT clause will ensure that the SELECT selects only one row, so the INSERT inserts only one row; if distributor.id_distributor is a primary key the LIMIT clause is unnecessary.
I'm migrating data from an old database to a new one in SSIS (2008 R2 Enterprise Edition). In the old database, I have a table called [Financial] and a column named: [Installments]. This column has a numeric value in it: 1, 2, 3 or 4. These are payments in installments. The old database only stores this numeric value and do not provide any more information about individual installments. The new database, however, provide more information of each installment, with columns like: [InstallmentPaid] (if the customer paid the installment), [DateInstallmentPaid] (when the customer paid the installment), [InstallmentNumber] (this is important to specify which installmentnumber it is. If the customer wants to pay in 4 installments, then 4 records will be created. 1 with InstallmentNr1, second with InstallmentNr2, third with InstallmentNr3 and fourth with InstallmentNr4.) and of course the [InstallmentPrice].
So the old database has the table [Financial] with the column [Installments]. The new database has the same [Financial] table, but instead of having a column called [Installments], it has a new relationship called [CustInstallments] ([CustInstallments] has FK FinancialID (1-to-many relationship)
So now that I'm migrating the data from the old database to the new one, I don't want to lose the information about the amount of installments. The following logic should be executed in SSIS in order to prevent information loss:
Foreach [Installments] in [Financial] from the old database, insert a
new [CustInstallment] referencing to the corresponding [FinancialID]
within the new database
So if in the old database the numeric value within [Installments] is 3, then I need to INSERT INTO CustInstallments (FinancialID, InstallmentNumber) VALUES (?, ?) This ? should be 1 at the first insert, 2 and the 2nd and 3 at the 3rd. So I need some kind of a loop here? Is that even possible within the data flow of SSIS?
Below the visualization (figure) and description of my flow so far.
I select the old database source [Financial]
I convert the data so it matches the current database data types
Since I already migrated the old [Financial] database data to the new one, I can use the lookup on the FinancialID's in the new database, so the first variable ? of the INSERT query can be linked to the lookup output.
I split all the possibilities, like when the Installment contains NULL, 1, 2, 3 or 4.
The 5th step is what I'm looking for. Some clue, some direction towards something useful. When NumberOfInstallments is 1, I need to INSERT INTO CustInstallments (FinancialID, InstallmentNumber) VALUES (?, ?) with the second ? variable as 1. When the NumberOfInstallments are 2, then I need to do two inserts, one with InstallmentNumber 1, and one with InstallmentNumber 2. When NumberOfInstallmentNumber is 3, then 3 inserts with a counting NumberOfInstallmentNumber. When 4, then four.
Is there any smart way to achieve this? Is there any built-in function available of SSIS that I am not aware of, and could be used here?
I appreciate any input here!
Thank you.
EDIT 10/02/2014
I have tried the following code:
INSERT INTO CustInstallments (FinancialID, InstallmentNumber) values (?, X);
WITH nums AS(select top 4 row_number() over (order by (select 1)) AS id
from sys.columns
) SELECT f.* FROM CustInstallments f
JOIN nums n on f.InstallmentNumber>= n.id
But this query doesn't create X-amount of records, instead, the JOIN nums just replicates it X-amount of times, so I still can't track every installment individually.
I have written my own code - toke me a while since I never worked with TSQL before - and this works like a charm in SQL Server:
DECLARE #MyCounter tinyint;
SET #MyCounter = 1;
WHILE (SELECT COUNT(*) FROM CustInstallments WHERE FinancialID = #ID) < 4
BEGIN
INSERT INTO CustInstallments (FinancialID, InstallmentNumber) VALUES (#ID, #MyCounter)
IF (SELECT COUNT(*) FROM CustInstallments) > 4
BREAK
ELSE
SET #MyCounter = #MyCounter +1;
CONTINUE
END
Now in SSIS, I cannot change the #ID to a ?-variable, and use the lookup FinancialID, because as soon as I do, I get the following error:
Could anyone explain me why SSIS doesn't like this?
EDIT 10/02/2014
My last and least preferable option would be to use multicast to cast an insert query X amount of times, where each X is an OLE DB Command. For example, when there are 3 [Installments] in the old column, I would create a multicast with 3 OLE DB commands, with their SqlCommand:
OLE DB Command 1: INSERT INTO CustInstallments (FinancialID, InstallmentNumber) values (?, 1);
OLE DB Command 2: INSERT INTO CustInstallments (FinancialID, InstallmentNumber) values (?, 2);
OLE DB Command 3: INSERT INTO CustInstallments (FinancialID, InstallmentNumber) values (?, 3);
This is an ugly approach, but with the small amount of data I am using, perhaps it's not a big deal.
I would try to resolve this with TSQL in your source query. Do a join to some kind of numbers table like this:
create table #fininancial (id int not null identity(1,1), investments int);
go
insert into #fininancial (investments) values (1),(2);
GO
with nums as (select top 5 row_number() over (order by (select 1)) as id
from sys.columns
)
select f.* from #fininancial f
JOIN nums n on f.investments >= n.id
EDIT:
The above example is unclear - sorry about that. I was only presenting the concept of replicating the rows, but not completing the thought of how you will apply it. Try this out:
create table #fininancial (financialid int not null, investments int);
go
insert into #fininancial (financialid, investments) values (123, 1),(456, 2);
GO
with nums as (select top 5 row_number() over (order by (select 1)) as id
from sys.columns
)
select f.financialid, n.id as investments from #fininancial f
JOIN nums n on n.id <= f.investments
So for each financialid you will get multiple investments with different investment ids. This is a set-based way to handle the operation, which will perform better than a procedural method and will require less effort in ssis. Does that make more sense?
I have one table which I have imported into mysql.
Now I need to create multiple related tables.
So basically I have currently got the following
start transaction;
Insert into Address (AddressLine1, AddressLine2, Town, County, Postcode)
Select Distinct Address_line_1, Address_Line_2, Town, County, Postcode from Import;
set addressLastId = last_insert_id();
INSERT INTO Organisation (Name, AddressID)
SELECT DISTINCT Supplier_Name, addressLastId FROM Import;
commit;
The second part where I use the last_insert_id never increments probably because it gets the last insert.
So I need to workout how i can get the previous id for each row inserted into the address table?
Would i need to have an inner loop within the address insert ?
Cheers
I agree with Tim.
After you've populated Address, then you could just run
INSERT INTO Organisation (Name, AddressID)
SELECT DISTINCT Import.Supplier_Name, Address.id
FROM Import INNER JOIN Address ON (set all the address lines and city etc =, since Im guessing there wasnt an address ID in the original import)
You could use a join for the second insert - join Address and Import, and insert the required fields from each into Organisation.
Getting the last insert ID will only work if you process each record sequentially.
There is a lot of information that I could find on SQL Merge, but I can't seem to get this working for me. Here's what's happening.
Each day I'll be getting an Excel file uploaded to a web server with a few thousand records, each record containing 180 columns. These records contain both new information which would have to use INSERT, and updated information which will have to use UPDATE. To get the information to the database, I'm using C# to do a Bulk Copy to a temp SQL 2008 table. My plan was to then perform a Merge to get the information into the live table. The temp table doesn't have a Primary Key set, but the live table does. In the end, this is how my Merge statement would look:
MERGE Table1 WITH (HOLDLOCK) AS t1
USING (SELECT * FROM Table2) AS t2
ON t1.id = t2.id
WHEN MATCHED THEN
UPDATE SET (t1.col1=t2.col1,t1.col2=t2.col2,...t1.colx=t2.colx)
WHEN NOT MATCHED BY TARGET THEN
INSERT (col1,col2,...colx)
VALUES(t2.col1,t2.col2,...t2.colx);
Even when including the HOLDLOCK, I still get the error Cannot insert duplicate key in object. From what I've read online, HOLDLOCK should allow SQL to read primary keys, but not perform any insert or update until after the task has been executed. I'm basically learning how to use MERGE on the fly, but is there something I have to enable for SQL 2008 to pick up on MERGE Locks?
I found a way around the problem and wanted to post the answer here, in case it helps anyone else. It looks like MERGE wouldn't work for what I needed since the temporary table being used had duplicate records that would be used as a Primary Key in the live table. The solution I came up with was to create the below stored procedure.
-- Start with insert
INSERT INTO LiveTable(A, B, C, D, id)
(
-- Filter rows to get unique id
SELECT A, B, C, D, id FROM(
SELECT A, B, C, D, id,
ROW_NUMBER() OVER (PARTITION BY id ORDER BY id) AS row_number
FROM TempTable
WHERE NOT EXISTS(
SELECT id FROM LiveTable WHERE LiveTable.id = TempTable.id)
) AS ROWS
WHERE row_number = 1
)
-- Continue with Update
-- Covers skipped id's during insert
UPDATE tb_TestMLS
SET
LiveTable.A = T.A,
LiveTable.B = T.B,
LiveTable.C = T.C,
LiveTable.D = T.D
FROM LiveTable L
INNER JOIN
TempTable T
ON
L.id= T.id
I have seen similar questions to mine, however nothing seems to fit the scenario exactly.
I have 2 tables, bills and accounts.
If a bill is pushed into the bills table and there is not already an account, I want to create a record in the accounts table.
I have:
...
BEFORE INSERT ON bills
FOR EACH ROW
BEGIN
INSERT INTO accounts (id, field1, field2, field3, field4)
SELECT accountID, 'value1', value2, value3, value4
FROM bills
WHERE ((SELECT LEFT(accountID, 5) = 'XXXXX' |
(SELECT LEFT(accountID, 5) = 'XXXXX' |
(SELECT LEFT(accountID, 5) = 'XXXXX' |
(SELECT LEFT(accountID, 5) = 'XXXXX' |
(SELECT LEFT(accountID, 5) = 'XXXXX')) &&
accountID NOT IN (SELECT id from accounts); ## I know this line is not right
END$$
From What I have read elsewhere on SO my options seem to be ON DUPLICATE KEY UPDATE or INSERT IGNORE, neither of which I can seem to make work, nor can I find an example with INSERT, SELECT, FROM and WHERE all involved.
What am I missing?
EDIT: I can make it insert the record, but depending on how I have tried it I always get either "duplicate id" error, or it doesnt insert the record.
Havent coded in MySQL for quite a while hence please excuse if there is some small syntax error. However, you will get the idea what needs to be done in the following code:
before insert on bills
for each row
begin
declare #li_cnt int;
select count('') into #li_cnt from accounts where accounts.id_account = inserted.id_account;
if #li_cnt = 0 then
insert into accounts (id_account) values (inserted.id_account)
end
end
However, this type of issue should be handled at the middle tier application server level and not at the database trigger level where one should be actually implementing strict policies and not complying to business requirements. So in that respect, assuming your app to be a PHP web app, you should have the bill's class method save() which should be checking the same and inserting the value in accounts table using the accounts table's class and then inserting the row in the bills table. Using this method one can really make modern sophisticated logic e.g. you can restrict insertion of bills for a specific account if the accounts class method that you call from the bill insertion sends back a decline response. Although it seems you require a simple insertion of data into accounts table when a bill is inserted with an accounts code / ID that doesnt exist in the master accounts table, I still feel coding the right way even in simple requirements makes for a good platform for an app to later scale and become pretty complicated with minimum issues maintenance cost.