I am trying to implement logging in my SSIS package. I would like to start logging by writing a message- Batch started -- to my custom log table. I am currently having the following tasks created which run in parallel. I am thinking of creating a execute sql task which will log that. How do I tell SSIS to execute sql task before executing the other tasks in the package. Do I need to move all these tasks in a sequence container.
Please find the screenshot of my package.
The best way I feel to implement logging is to have it in a non-intrusive manner i.e. to have them run in the background. SSIS provides this facility via 'Event Handlers'.
Open your package and go to 'Event Handlers'. With 'Executable' set as "Your Package Name", select 'OnPreExecute' as 'Event handler'. You then click on the hyperlink for you to proceed with adding the 'Execute SQL Task' for auditing.
Using the same above, you can then go on to add 'OnPostExecute'. Additionally this technique can be implemented across individual task level as well.
For more information you can refer to - https://learn.microsoft.com/en-us/sql/integration-services/integration-services-ssis-event-handlers
Related
I have developed an SSIS package to run 3 reports from Reporting Services that are data driven subscriptions.
When I run the SSIS job it executes all the 3 reports at once, what I need is to run the reports sequentially, in other words, one by one. How can I do this?
This is an expected behavior. When you trigger a data driven subscription job, the SQL Server Agent starts the job and that completes the whole transaction. The SSIS package would then go on to trigger the next data driven subscription job and the next ( assuming you have put the job-triggering in sequence).
Now if you want to create a dependency in the way the jobs should run i.e. Job1 followed by Job2 followed by Job3, you need to manually write additional piece of code. The way to go about it would be to monitor the status code of the subscription.
In the ReportServer database there is a table called dbo.Subscriptions containing a column 'LastStatus'. Currently in my local db, I don't have any subscriptions and also am not able to find any documentation for the table. But I am pretty sure this would be either a boolean or a status flag such as 'Sucess' or 'Failure. Upon triggering the first job, you would need to write a .net Code to monitor this status with a polling interval. Once you get the desired outcome, move on to triggering the next job.
Hope this is clear. I would edit this answer with an working example.
It is possible to run only a specific Data Flow task via Visual Studio (right click on the Data Flow and execute). E.g. you have multiple data flows- but only execute one of them.
I am now trying to implement this ability via a job
- e.g. I have 1 Control Flow with 2 Data Flows (DataFlow1, DataFlow2) -
and in
SQLJob1 - it will fire DataFlow1,
SQLJob2 - will fire DataFlow2 of the same SSISPackage
The aforementioned link states "You can build a special control flow logic using expressions on precendance constraints to define optional execution paths."
I dont want to create special control flow logic - or have 2 separate SSIS packages installed - what would the SQL command be in the Job to fire
only DataFlow1 please?
I see I can in VS right click and disable a specific data flow - and then run the package. I tried to see the command for disabling/enabling a specific data flow, but there is no SQL Query for it - or is it possible to run SQL query to disable/enable specific data flow?
This is the SSIS-way to do this, and I'm sorry, but it involves "special control flow logic" because there is no other way:
Add a package-level variable in your SSIS package.
Add a script task to the control flow of your SSIS package. The script doesn't have to do anything. Think of it as a "starting anchor". It will be the first thing that executes in your package.
Add separate precedence constraints (the little arrows that link tasks) from the script task to your two dataflows.
Double-click on each precedence constraint and set them to use an expression for validation. Use the variable you created, and set one of the paths to be true if the variable is set to "DataFlowA" and the other if it is set to "DataFlowB".
In the jobs, set the value of the variable to the appropriate value for the dataflow you want to have execute.
This is the answer. I'm sorry it's not what you were hoping for but the answer to these questions:
what would the SQL command be in the Job to fire only DataFlow1
please?
is it possible to run SQL query to disable/enable specific
data flow?
Are, respectively,
There is no such command.
No, it is not possible to programmatically enable and disable tasks.
I have created an SSIS solution (using SQL SERVER 2012) to extract data from ServiceNow. All the tasks are wrapped in a Sequence Container. When I right click on the Sequence Container and click Execute Task the package behaves as expected. It extracts the data from ServiceNow and performs the rest of the etl. However when I click Execute Package from the Solution Explorer the package successfully completes but it does not extract any data from Service Now.
I have played around with settings and different package designs but with no change in behavior. I can execute with true success at the task level but not the package level. This behavior is even apparent after deployment. I can execute with success from the SSISDB but with no data extraction. When I hook the deployed package to a job I still get no data extraction.
My thinking it has to be some kind of bug or hidden proxy setting because I only get true success (with data extraction) when I manually execute at the task level - i.e. the Sequence Container.
I have been using SSIS for a couple years now and have not come across this issue. Hopefully someone can help or have some ideas.
Got it. I needed a break and to think about this clearly. I needed to update the Package Property Delay Validation from False to True. That solved the issue. I guess the SSIS engine needed the extra time to validate the variable and expressions set up within the package.
I am new to SSIS. My problem is , I have to do a databse query. Then, I have to do a web service call passing in the result that I got from the db query, to the web service .
Either I can use sql task first for db query and web service task next for web service call. Another way is to do do entire thing in the c# code and put that in script task.
I can use enterprise library for doing db query through the C# code and then call the web service.
What is the better approach to do it ?Which one gives better performance?
Both approaches can work. Both will likely result in approximately the same performance - usually the query execution on SQL Server is the slowest part, and it does not matter how you invoke it.
But if you do everything in C# code and put it in script task - what is the benefit of SSIS? You can as well do everything in C# and put it in standalone console app.
Usually one uses SSIS to avoid writing code, and use declarative program definition - which is more maintainable, easier for others to understand and support, etc. Someone who opens a package and sees SQL task and Web Service task will be able to understand and tweak what's going on without opening Script Task and examining the code. It also requires less developer expertise to maintain such package.
Script Task is then only used when something can't be done directly by existing SSIS task or transform. If I saw a package with all the code in a Script Task - I would ask why the person used SSIS at all? Doing it with standalone console app would be simpler, more reliable (than the same code inside Script Task), avoid unnecessary dependency, etc.
Short: if you DO use SSIS - avoid Script Task when possible.
In VB pseudocode, I'm trying to accomplish the following using SSIS (either 2008 or 2005)
If FileHasAlreadyBeenDownloaded = False Then
DownloadTheFileFromFTP
End If
ImportTheDownloadedFile
To do this in SSIS I have a script task to check for the file, and if it exists it transfers directly to the DataFlow Task using conditional expressions. If the file doesn't exist, it transfers to the FTP Task, and the FTP Task transfers to the DataFlow Task.
It seems, though, that I can't have two tasks lead into one common shared task, because no matter which path the code takes it won't execute the DataFlow Task. If I make a copy of the DataFlow task and have each path go to its own Task, then all works perfectly.
Is this a documented thing with SSMS that I just haven't found? I looked through 31 pages of questions on SSMS before posting, so hopefully this isn't a stupid question.
I also tried using Expressions on the FTP task to set "Disabled=#FileAlreadyDownloaded=True" but that works only in SSMS 2008 and didn't seem to work in SSMS 2005.
Thanks so much for any pointers on this!
It might be worth trying putting the script task and the FTP task inside a container task, and link the the container to the data flow task on success.