I got question from interview!!!
How to achieve Check point functionality in SSIS with out using check point.
If package fails it has to be re-run from the point that it failed instead of rerunning the entire package.without using check point.
one way is create a task dispatcher as entry point in the control flow. This will only keep track of the last task successfully executed and call the next task depending on that persisted variable. The first time it runs the variable would be 0 and therefore it will start in the first task, if the first task completes successfully the variable will be set to 1. If the task 2 fails, then the next time the control flows run it will restart on task1 and so on... the control flow will look like this:
Related
I have SSIS package which has few task(Execute SQL Tasks, Data flow task).I have two event handlers(OnPostExecute,OnError) for custom logging on package level.
I am using system::containerStartTime for the TaskStartTime(dateTime) on OnPostExecute event handler. But it's not inserting the correct start time. always all the tasks has the same time with milliseconds different. but there is minute of difference in the different task to start.
it's look like to me it's static variable which sets at the start of the package if this is the case than what is the difference between system:starttime and system:containerStartTime.
This variable should show the same time which appear against each task as start time in execution result tab in Visual studio.
please let me know how i get the correct start time of each tasks in the package.
Thanks,
Zaim Raza.
containerStartTime is the time of the container of the task not the tasks themselves.
I'm afraid there is no system variable for time at task level as per Microsoft documentation here:
https://learn.microsoft.com/en-us/sql/integration-services/system-variables
One thing you can do is save the time to a user variable at the beginning or immediately before the tasks being monitored and use this on your event handler.
I have an SSIS package with about 50 different steps. Many of the steps will run in parallel.
I would like to put a ScriptTask at the end of the package such that, regardless of success or failure, I send out an email which reports on the status of the package run (the package logs things to a database as it runs, so the ScriptTask involves querying the status tables and formatting them into HTML).
Is there a way to have a task that executes when all other tasks have either completed running, or been skipped due to dependencies?
I have tried to add my task to the OnError event handler, but this has it raise up to n times when tasks fail in parallel. I don't want to have every single step flow to the ScriptTask with the OnFailure condition, because it will make the package functionally impossible to use due to the volume of connections. I also believe that connecting it as such would not ensure it ran, as some steps would be skipped and thus not have a success or failure status.
I can do this in two different ways
Add a Sequence Container to your Control Flow.
Move all existing Tasks inside of the the Sequence Container
Add your Script Task outside the Sequence Container. Change the Precedence constraint from Success to Completion by double clicking on the line
Option b
Create an package that has your script task in it.
Create a master package that runs the original package and then calls the package with the script task, passing any variables as needed for it to generate the email.
It is possible to run only a specific Data Flow task via Visual Studio (right click on the Data Flow and execute). E.g. you have multiple data flows- but only execute one of them.
I am now trying to implement this ability via a job
- e.g. I have 1 Control Flow with 2 Data Flows (DataFlow1, DataFlow2) -
and in
SQLJob1 - it will fire DataFlow1,
SQLJob2 - will fire DataFlow2 of the same SSISPackage
The aforementioned link states "You can build a special control flow logic using expressions on precendance constraints to define optional execution paths."
I dont want to create special control flow logic - or have 2 separate SSIS packages installed - what would the SQL command be in the Job to fire
only DataFlow1 please?
I see I can in VS right click and disable a specific data flow - and then run the package. I tried to see the command for disabling/enabling a specific data flow, but there is no SQL Query for it - or is it possible to run SQL query to disable/enable specific data flow?
This is the SSIS-way to do this, and I'm sorry, but it involves "special control flow logic" because there is no other way:
Add a package-level variable in your SSIS package.
Add a script task to the control flow of your SSIS package. The script doesn't have to do anything. Think of it as a "starting anchor". It will be the first thing that executes in your package.
Add separate precedence constraints (the little arrows that link tasks) from the script task to your two dataflows.
Double-click on each precedence constraint and set them to use an expression for validation. Use the variable you created, and set one of the paths to be true if the variable is set to "DataFlowA" and the other if it is set to "DataFlowB".
In the jobs, set the value of the variable to the appropriate value for the dataflow you want to have execute.
This is the answer. I'm sorry it's not what you were hoping for but the answer to these questions:
what would the SQL command be in the Job to fire only DataFlow1
please?
is it possible to run SQL query to disable/enable specific
data flow?
Are, respectively,
There is no such command.
No, it is not possible to programmatically enable and disable tasks.
I have been working with a co-worker in an attempt to setup a master and sub SSIS package templates. These templates look into an audit table in order to identify the correct batch or create a new one. The start and end times of the batch are passed into the CDC (change data capture) table valued functions which returns the Start and END LSN's of the table.
This works by identifying the start and end dates of the batch in the master package. These dates then get set to user variables in the master package. The master package then calls a sub package and passes the user variable start time as well as user variable end time to the sub packages package level parameters. At this point I can print or write the dates to a random table from the sub package and they are showing the correct values.
Next I want to add a data flow task that will use the sub packages package level parameters (Passed from master package) to run the table valued function for the correct time period. This should return the Start & End LSN's that can be used in the source query. Unfortunately what I am seeing is the Data Flow Task never executes any of the task inside the Data Flow. The Data Flow Task gets a nice green checkmark but if you open the dataflow none of the tasks have been executed.
We are scratching our heads on this one and have even created a simple proof of concept that passes a value from a master package to the sub and then attempts to print it from the sub packages DFT but it never executes.
Any ides what could be causing the data flow to be ignored but yet show a success status?
After a lot of headache and frustration I finally found the answer, or rather my co-worker. In the past I have always created stand alone packages that get called from a job in SSMS. When developing individual packages I typically add either a Data Conversion or Derived Column component at the end of the last step completed in the Data Flow. I then setup a data viewer on the path to the Data Conversion/Derived Column. This allows me to check the data before moving on to the next piece of the data flow.
I was using this tactic when working with a master package that calls the child package I was working on. This was the source of the problem. When the master package calls the child package it identifies that components in the Data Flow are not used later in the package and skips them.
To summarize: If you are using a master package to call a child package make sure all pieces of the Data Flow are used later on in the package or they will be skipped during processing.
The starting item in my Control Flow needs to be something that will check what time it is and set a variable. Is there an item in the Control flow items that can do this?
Access your system variables. System::StartTime will be when the package started execution. If there is a significant amount of time between start of execution and the element that needs to evaluate the current time, you can look at the ContainerStartTime. If that isn't precise enough for you, look at using a script task.
SQL Server 2005 Run a stored procedure via SSIS using GETDATE() as a Parameter