I have 2 different sources to be loaded in Datawarehouse with a condition.
1st source is flat file and the second source is SQL database. Condition is the SSIS package should check the flag value in a table, If the value is 0, it should process the flat file and if the value is 1, then it should process the data from SQL source. I know this can be done in built in task but it must be done in Script task by dynamically changing the connection type (data provider) based on the flag value. Anybody who is well versed in .NET, Please help on this. Kindly let me know if something is not clear
Thanks.
Related
I am working on a Microsoft Integration Service project, and I have a flat source file (Product.txt) that contains some data that I am saving in a SQL Server DB when I run the project.
The data is saved successfully, but when I change some values in my source Product.txt and re-run the project, the data in the SQL server are not updated.
Is there any thing that must be done to enable the update? Thank you.
There are several things you can do but you haven't provided enough info. I am guessing here based on the word "changed file", to me that means update.
That generally means in your data flow you should start with source, then use a lookup based on your destination to see if your "key" exists. Change the test to redirect no match.
Map no match to your inserts. And map matches to an update SQL statement.
I'm writing a (what I thought to be a) simple SSIS package to import data from a CSV file into a SQL table.
On the Control Flow task I have a Data Flow Task. In that Data Flow Task I have
a Flat File Source "step",
followed by a Data Conversion "step",
followed by a OLE DB destination "step".
What I want to do is to move the source CSV file to a "Completed" folder or to a "Failed" folder based on the results of the Data Flow Task.
I see that I can't add a File System step inside the Data Flow Task, but I have to do it in the Control Flow tab.
My question is how do I do a simple thing like assign a value to a variable (I saw how to create variable and assign them a value at the bottom pane of Data Tools (2012)) depending of if the "step" succeeds or fails?
Thanks!
(You can tell by my question that I'm an SSIS rookie - and don't assume I can write a C# script, please)
I have used VB or C# scripts to accomplish this myself. Since you do not want to use scripts I would recommend using a different path for the project to flow. Have your success path lead to moving the file to completed and failure path lead to moving the file to failed. This keeps it simple and accomplishes what you are looking for.
I'm working with legacy tsql code that outputs to txt files.
For security reasons, I'm replacing these outputs with SSIS packages.
I've gotten most of them to work, but one particular one gives me the following error:
TITLE: Microsoft Visual Studio
Cannot create connector.
The destination component does not have any available inputs for use in creating a path.
The data flow itself is very simple. OLE DB Source runs an SQL command, then outputs to a flatfile source that points to an existing txtfile that was created by the TSQL.
Anyone know what the error means in regards to the available inputs?
Your SSIS toolbox is divided into 3 general groupings (pre 2012/2014)
Sources
Transformations
Destinations
A source has 1 to N output paths. Nothing can feed into a source. Things can only consume what a Source emits.
A Transformation does not generate* rows, it accepts rows from an upstream provider (either a Source or another Transformation). A Transformation has 1 to N output paths.
A Destination is the terminus for data. I'm not aware of any destinations that accept more than one input. It has one optional output path, Error.
Your problem, therefore, is that you are trying to route data into a Source. Change that to a Flat File Destination.
Replace the Flat File Source with a Flat File Destination task. Click on the RowCount task and drag the green arrow to the new destination.
Observe the following snippet of an SSIS package:
Both of the Flat File Destination tasks on the right are configured to write to the same Flat File Connection Manager, because I want all the failing records to be redirected to the same file regardless of which task caused them to fail.
SSIS is complaining on the second Flat File Destination, saying that the output file is in use by another process.
Is my stated goal simply impossible, or is there a way to redirect all failing records to the same output file?
No, you cannot do it this way. You need to use an Union All to combine the output before writing the output from different sources/transformations to the same destination.
If you have a process flow as described in the question, the package will fail on the second destination component.
The error message would be `The process cannot access the file because it is being used by another process.
To fix the issue, add a Union All transformation that will take the error output from Derived Column transformation and combines it with the error output from OLE DB Destination and the output of Union All transformation is then passed on to the Flat File destination.
Hope that helps.
I need to pull data from csv file to SQL Server table. Which Control task should I use ? Is it Flat File ? What is the correct method to pull data ?
The problem is I have used Flat File Task for pulling csv file. But the csv file whihc I am having, contains headings as first row, then on the third row, I have the columns, and data starting from fifth row.
Another problem is, in this file column details comes again after 1000 data ie columns appears in two rows. Is it possible to pull data ? If so, HOW ?
While Valentino's suggestion should work, I suggest that first you work with the provider of the file to get them to provide the data in a better format. When we get stuff like this we almost always push it back and ask for properly formatted data. We get it too about 90% of the time. It will save you work if they will fix their own drek. In our case, the customers providing the data are paying for our programming services and when they understand how substantial an increase in the cost to them, they are usually nmore than willing to accomodate our needs.
I believe you'll first have to transform your file into a proper CSV file so that the SSIS Flat File Source component (Data Flow) can read it. If the source system cannot produce a real CSV file, we usually create custom .NET applications for the cleanup/conversion task.
An Execute Process task (Control Flow) that executes the custom app can then be called prior to the Data Flow.