How to set up SSIS parent package such that 4 child packages can run at the same time with different parameter values passed in? - ssis

Question

I have created a child SSIS package that executes according to the "ProcessName" variable value that is specified initially. Now, I wish to create a parent package such that I can execute 4 child package tasks with different ProcessName values passed in to be executed in parallel. How can I maintain my child package and pass in different values to each of the 4 execute packages task such that the ProcessNames variable values are different for each of them? I am new to SSIS and would deeply appreciate if someone could advice or give a direction on how I could go about doing so.

Answer 1

I would see this as a pattern like the following
The "trick" here is that within each Sequence Container, SEQC, I need to define my variable that holds my parameter value. That variable needs to be scoped to the container - otherwise, there is only one SSIS variable and the 4 processes that attempt to initialize that value will be in conflict.
In the SSIS Variables menu, there is a Move Variable icon (second one listed)
Here you can see that I have ParameterValue defined in both "SEQC Opt 1a" and "SEQC Opt 1b" and they're initialized with different values.
The first step within the Sequence container is an Execute SQL Task where I pull back the intended parameter value. Maybe that is not needed in your case but it can be helpful to have a repository of run-time values. In the case of 1b, this is much more what my execution pattern looks like. I have a query that pulls back any packages to be run within the scope of this container and the starting value. e.g.
ContainerName|PackageName|StartingValue
SEQC Opt 1a |Child0.dtsx|100
SEQC Opt 1a |Child1.dtsx|200
SEQC Opt 1a |Child2.dtsx|300
SEQC Opt 1b |Child5.dtsx|600
SEQC Opt 1b |Child6.dtsx|700
SEQC Opt 1b |Child7.dtsx|800
This table pattern allows me to dynamically run packages in both parallel and in serial. Assuming Child7 and Child2 in the above set are very slow but the other 4 packages are relatively fast. The fast ones would start up, do their work and complete and the next runs. There are limits to how many parallel operations can fire at once so you can't scale infinitely across processes so a balance of serial and parallel operations makes sense.
Once you have your pattern working for one sequence container: copy, paste, rename and assuming you lookup in a table based on the task name as I show above, it's ready to go.

Answer 2

NOTE for everyone reading this answer: This answer is not full/complete with examples/full steps. From comment above I am posting this now so requestor can see it and get started.
This was from notes I wrote myself a long while back on how to do this for myself. I am posting it as answer as it is helpful and too large to post as comment. Plus I have not rewritten anything in what wrote for myself and what I am posting.
Currently I can not find my full code yet to post full details/steps. If/when I do I will post here, but this should be good details on what/how to do it. Plus this gives info on how to handle child package error trapping as well.
-- my notes I saved for myself posting as answer:
Steps for creating child packages:
Create any variables needed in the child package
Create the coorisponding variable name in the parent package (the name doesnot have to be the same, and maybe want to name it something to identify it as a child package variable)
Child Package:
Need to set up: Package Configurations
a. Right click on package and click Package Paramaters
b. Click the checkox to Enable Package configurations
Click Add and set the paramaters:
a. Configuration Type: Pareknt Package variable
b. Specify the configuration setting directly: Put the parent variable name in here that the child package is going to access
c. Click "Next"
d. In "Objects" window scroll down to the variable you are setting from the parent variable name you selected above and click the "Value" option under Properties for that variable name
e. Click "Next"
f. Under Configuration Name: Set a detailed name for what this variable is/does.
Error Handling (NOTE: This is not required but you wont capture the child error messages if you dont do this):
a. Go to Event Handlers Tab
b. Under drop down (on top right) select OnError
c. Add a Scrit Task
d. Pass as read only variables:
System::ErrorDescription
System::SourceName
System::PackageName
e. Copy/paste the code below into the script task uin the Main() function.
----- this is for the error handling
public void Main()
{
// build our the error message
string ErrorMessageToPassToParent = "Package Name: " + Dts.Variables["System::PackageName"].Value.ToString() + Environment.NewLine + Environment.NewLine +
"Step Failed On: " + Dts.Variables["System::SourceName"].Value.ToString() + Environment.NewLine + Environment.NewLine +
"Error Description: " + Dts.Variables["System::ErrorDescription"].Value.ToString();
// have to do this FIRST so you can access variable without passing it into the script task from SSIS tool box
// Populate collection of variables. This will include parent package variables.
Variables vars = null;
Dts.VariableDispenser.GetVariables(ref vars);
// checks if this variable exists in parent first, and if so then will set it to the value of the child variable
// (do this so if parent package does not have the variable it will not error out when trying to set a non-existant varaible)
if (Dts.VariableDispenser.Contains("OnError_ErrorDescription_FromChild") == true)
{
// Lock the to and from variables.
// parent variable
Dts.VariableDispenser.LockForWrite("User::OnError_ErrorDescription_FromChild");
// Need to call GetVariables again after locking them. Not sure why - perhaps to get a clean post-lock set of values.
Dts.VariableDispenser.GetVariables(ref vars);
// Set parentvar = childvar
vars["User::OnError_ErrorDescription_FromChild"].Value = ErrorMessageToPassToParent;
vars.Unlock();
}
Dts.TaskResult = (int)ScriptResults.Success;
}
Parent Package:
Add this variable to propertly capture the child error messages (not required but you wont capture chidl error messages if you dont):
variable: OnError_ErrorDescription_FromChild
Error Handling(NOTE: This is not required but you wont capture the child error messages if you dont do this):
a. Go to Event Handlers Tab
b. Under drop down (on top right) select OnError
c. Add a Scrit Task
d. Pass as read only variables:
User::OnError_ErrorDescription_FromChild
e. Copy/paste the code below into the script task uin the Main() function.
----- this is for the error handling
public void Main()
{
// get the varaible from the parent package for the error
string ErrorFromChildPackage = Dts.Variables["User::OnError_ErrorDescription_FromChild"].Value.ToString();
// do a check if the value is empty or not (so we knwo if the error came from the child package or the occurent in the parent package itself
if (ErrorFromChildPackage.Length > 0)
{
// Then raise the error that was created in the child package
Dts.Events.FireError(0, "Capture Error From Child Package Failure",
ErrorFromChildPackage
, String.Empty, 0);
//Dts.TaskResult = (int)ScriptResults.Failure;
} // end if the error length of variable is > 0
Dts.TaskResult = (int)ScriptResults.Success;
}
NOTES:
For error handling:
a. The child package error handling is written so it wont fail if the variables or error handling does not exist in parent package.
b. If you include the error handling (and variable) in the parent package it MUST exist in the child package though.