I need to elaborate the GUID of every subpackage which is executed by a main package.
In order to do that, I've handled the event Pre-Execute, to record every source of Pre-Execute event. The problem is that in PreExecute event handler, it isnt possible to know if the source of the event, represented by SourceID and SourceName, is a subpackage or a task.
I would code only in the main package, in order to centralize the code.
Other attemps:
I've tried with other events, like PreValidate, and the issue is the same.
Looking at the standard output (see below) I found a specific log row for subpackages, but I dont know how to read standard output log at runtime.
This is the standard output:
Log:
Name: User:PackageStart
Computer: COMPUTERNAME
Operator: username
Source Name: childpackage
Source GUID: {9E629F59-5DD1-44AD-BD64-9704353267E2}
Execution GUID: {4168C4C6-1EC7-41C2-8F2D-98AD3A4F4D82}
Message: Beginning of package execution.
Start Time: 2016-05-16 12:28:16
End Time: 2016-05-16 12:28:16
End Log
Thank you
Related
I have some long saltstack orchestration states and i want to add some informational messages to it (for example: Gonna apply some state on minion foo) ,and this messages must be printed immediately ( not after all actions completed).
Jinja log message {% do salt.log.error("Some message) %} is not suitable (they printed before state actually runs).
test.echo module also not suitable (prints message after all actions completed)
test_blabla:
salt.runner:
- name: salt.cmd
- arg:
- test.echo
- some_blabla
Is there any way to print messages during states execution? Maybe I'm missing something?
There are ways to cheat. As long as you are talking about orchestration AND you are running said orchestration through salt-run.
And you almost had it. except to conflated the answer into jinja and the wrong module into the state. the reason the log message comes before anything runs is because you are calling it in jinja. but you don't have to call it through jinja. you can call it in a state.
test_blabla:
module.run:
- name: log.error
- message: some_blabla
test_oh:
salt.runner:
- name: salt.cmd
- arg:
- test.sleep
- 10
This is all kind of hacky as salt was not meant to do this. salt runs async which means it doesn't return until it is finished. or may not return directly at all and you are meant to check the results else where. another way to do this is have a state fire an event and watch the event bus instead of the state run.
I have created a child SSIS package that executes according to the "ProcessName" variable value that is specified initially. Now, I wish to create a parent package such that I can execute 4 child package tasks with different ProcessName values passed in to be executed in parallel. How can I maintain my child package and pass in different values to each of the 4 execute packages task such that the ProcessNames variable values are different for each of them? I am new to SSIS and would deeply appreciate if someone could advice or give a direction on how I could go about doing so.
I would see this as a pattern like the following
The "trick" here is that within each Sequence Container, SEQC, I need to define my variable that holds my parameter value. That variable needs to be scoped to the container - otherwise, there is only one SSIS variable and the 4 processes that attempt to initialize that value will be in conflict.
In the SSIS Variables menu, there is a Move Variable icon (second one listed)
Here you can see that I have ParameterValue defined in both "SEQC Opt 1a" and "SEQC Opt 1b" and they're initialized with different values.
The first step within the Sequence container is an Execute SQL Task where I pull back the intended parameter value. Maybe that is not needed in your case but it can be helpful to have a repository of run-time values. In the case of 1b, this is much more what my execution pattern looks like. I have a query that pulls back any packages to be run within the scope of this container and the starting value. e.g.
ContainerName|PackageName|StartingValue
SEQC Opt 1a |Child0.dtsx|100
SEQC Opt 1a |Child1.dtsx|200
SEQC Opt 1a |Child2.dtsx|300
SEQC Opt 1b |Child5.dtsx|600
SEQC Opt 1b |Child6.dtsx|700
SEQC Opt 1b |Child7.dtsx|800
This table pattern allows me to dynamically run packages in both parallel and in serial. Assuming Child7 and Child2 in the above set are very slow but the other 4 packages are relatively fast. The fast ones would start up, do their work and complete and the next runs. There are limits to how many parallel operations can fire at once so you can't scale infinitely across processes so a balance of serial and parallel operations makes sense.
Once you have your pattern working for one sequence container: copy, paste, rename and assuming you lookup in a table based on the task name as I show above, it's ready to go.
NOTE for everyone reading this answer: This answer is not full/complete with examples/full steps. From comment above I am posting this now so requestor can see it and get started.
This was from notes I wrote myself a long while back on how to do this for myself. I am posting it as answer as it is helpful and too large to post as comment. Plus I have not rewritten anything in what wrote for myself and what I am posting.
Currently I can not find my full code yet to post full details/steps. If/when I do I will post here, but this should be good details on what/how to do it. Plus this gives info on how to handle child package error trapping as well.
-- my notes I saved for myself posting as answer:
Steps for creating child packages:
Create any variables needed in the child package
Create the coorisponding variable name in the parent package (the name doesnot have to be the same, and maybe want to name it something to identify it as a child package variable)
Child Package:
Need to set up: Package Configurations
a. Right click on package and click Package Paramaters
b. Click the checkox to Enable Package configurations
Click Add and set the paramaters:
a. Configuration Type: Pareknt Package variable
b. Specify the configuration setting directly: Put the parent variable name in here that the child package is going to access
c. Click "Next"
d. In "Objects" window scroll down to the variable you are setting from the parent variable name you selected above and click the "Value" option under Properties for that variable name
e. Click "Next"
f. Under Configuration Name: Set a detailed name for what this variable is/does.
Error Handling (NOTE: This is not required but you wont capture the child error messages if you dont do this):
a. Go to Event Handlers Tab
b. Under drop down (on top right) select OnError
c. Add a Scrit Task
d. Pass as read only variables:
System::ErrorDescription
System::SourceName
System::PackageName
e. Copy/paste the code below into the script task uin the Main() function.
----- this is for the error handling
public void Main()
{
// build our the error message
string ErrorMessageToPassToParent = "Package Name: " + Dts.Variables["System::PackageName"].Value.ToString() + Environment.NewLine + Environment.NewLine +
"Step Failed On: " + Dts.Variables["System::SourceName"].Value.ToString() + Environment.NewLine + Environment.NewLine +
"Error Description: " + Dts.Variables["System::ErrorDescription"].Value.ToString();
// have to do this FIRST so you can access variable without passing it into the script task from SSIS tool box
// Populate collection of variables. This will include parent package variables.
Variables vars = null;
Dts.VariableDispenser.GetVariables(ref vars);
// checks if this variable exists in parent first, and if so then will set it to the value of the child variable
// (do this so if parent package does not have the variable it will not error out when trying to set a non-existant varaible)
if (Dts.VariableDispenser.Contains("OnError_ErrorDescription_FromChild") == true)
{
// Lock the to and from variables.
// parent variable
Dts.VariableDispenser.LockForWrite("User::OnError_ErrorDescription_FromChild");
// Need to call GetVariables again after locking them. Not sure why - perhaps to get a clean post-lock set of values.
Dts.VariableDispenser.GetVariables(ref vars);
// Set parentvar = childvar
vars["User::OnError_ErrorDescription_FromChild"].Value = ErrorMessageToPassToParent;
vars.Unlock();
}
Dts.TaskResult = (int)ScriptResults.Success;
}
Parent Package:
Add this variable to propertly capture the child error messages (not required but you wont capture chidl error messages if you dont):
variable: OnError_ErrorDescription_FromChild
Error Handling(NOTE: This is not required but you wont capture the child error messages if you dont do this):
a. Go to Event Handlers Tab
b. Under drop down (on top right) select OnError
c. Add a Scrit Task
d. Pass as read only variables:
User::OnError_ErrorDescription_FromChild
e. Copy/paste the code below into the script task uin the Main() function.
----- this is for the error handling
public void Main()
{
// get the varaible from the parent package for the error
string ErrorFromChildPackage = Dts.Variables["User::OnError_ErrorDescription_FromChild"].Value.ToString();
// do a check if the value is empty or not (so we knwo if the error came from the child package or the occurent in the parent package itself
if (ErrorFromChildPackage.Length > 0)
{
// Then raise the error that was created in the child package
Dts.Events.FireError(0, "Capture Error From Child Package Failure",
ErrorFromChildPackage
, String.Empty, 0);
//Dts.TaskResult = (int)ScriptResults.Failure;
} // end if the error length of variable is > 0
Dts.TaskResult = (int)ScriptResults.Success;
}
NOTES:
For error handling:
a. The child package error handling is written so it wont fail if the variables or error handling does not exist in parent package.
b. If you include the error handling (and variable) in the parent package it MUST exist in the child package though.
In Fiware CEP's User Manual (pdf), page 12, it's mentioned you can create an event Producer of the type 'Timed', that will retrieve events from a file at intervals of time based on their 'OccurranceTime' property.
In my Fi-Lab's intance I don't find this 'Timed' type of producer in the dropdown list, only: File, JMS, Rest and Custom.
So I thought this feature could be implemented in the type 'File', but I can't get it to work, the property 'sendingDelay' in the Producer, always dictates the reading speed, not 'OccurrenceTime' in the event payload. Deleting 'sendingDelay' from the Producer makes it not send events at all.
OccurranceTime is said, in the manual, to be in milliseconds and in the authoring tool it has variable type of 'Date', so "OccurranceTime":"1000" should mean one second.
So, how can I get events produced at desired times? Is it just a matter of correct formating?
(BTW: in the manual OccurranceTime is spelled in two diferent ways: 'OccuranceTime' and 'OccurranceTime'. I believe the correct one is with double 'r', as it's what the authoring tools gives by default when creating a new event.)
Thank you,
Arthur
Event producer of type 'Timed' is a new feature that is part of release 4 of the CEP. It should be available in FIWARE Lab on October.
When available, you could choose it as the producer's type in the CEP Authoring tool. Then, the CEP will read events from an input file. In this file, you will write the expected occurrence time of each event.
For example, if the content of the input event file in JSON format is:
{"Name":"TrafficReport", "volume":"1000", "OccurrenceTime":"1000"}
{"Name":"TrafficReport", "volume":"1600", "OccurrenceTime":"6000"}
{"Name":"TrafficReport", "volume":"2500", "OccurrenceTime":"11000"}
The producer will process the second input event 5 seconds after the first input event, since it said to occur 5000 ms after the first one.
I have a DTS package that calls an executable via an Execute Process Task object. The path of executable can change based on where the product that this is contained in is installed. Is there some way to make the executable path dynamic?
I tried using an expression for the executable property. I set it to the a value that came out of a stored procedure, but it seems to only calculate the value when you save the package. I tried setting DelayValidation = true, but it doesn't seem to ever update it at runtime.
I believe you have something amiss with your package. Update your question with concrete details or compare away to my sample.
Setup
I create 7 subfolders from my base location and inside each, I placed a batch file
#echo off
REM N replaced with value 0-6
ECHO C:\ssisdata\EXEC\N\RunMe.bat
This led to a structure like
C:\ssisdata\EXEC\0\RunMe.bat
C:\ssisdata\EXEC\1\RunMe.bat
C:\ssisdata\EXEC\2\RunMe.bat
C:\ssisdata\EXEC\3\RunMe.bat
C:\ssisdata\EXEC\4\RunMe.bat
C:\ssisdata\EXEC\5\RunMe.bat
C:\ssisdata\EXEC\6\RunMe.bat
When I run them, it simply reports back the hard-coded location message
SSIS
I created an SSIS package that had a For Loop Container and inside was an Execute Process Task coupled to a Script Task
Variables
FolderBase: string - C:\ssisdata\EXEC Abstracts away the common path
FolderChoice: int - 0 Montonically increasing value from 0 to 6. Use by the loop to force change the location of the executable
Output: string - `` Captures the output from the executable to prove it works as expected
CurrentExecutable: string - C:\ssisdata\EXEC\0\RunMe.bat This is an Expression based on the above variables. Expression is #[User::FolderBase] + "\\" + (DT_WSTR, 1) #[User::FolderChoice] + "\\RunMe.bat"
Execute Process Task
I did nothing of interest here. I route standard out to an SSIS Variable and I actually used C:\ssisdata\Exec\RunMe.bat as my source but the next step updated this screenshot.
On the Expressions tab, I used my Variable #[User::CurrentExecutable] and assigned it to the Executable property.
Script Task
I passed in my #[User::Output] variable and call Dts.Events.FireInformation to make the output show up.
I have attached the screen shot
How to use the check points in SSIS 2012 package in order to restart the package from point of failure not from the beginning.
I think its not allowing me to attach the image...
You can use Checkpoints which basically executes the package from the point where it got failed.If your package has 5 to 6 steps and if the 2nd step failed then the next time when u execute the package it will start from 2nd step .
You can refer these blogs 31 days of SSIS and articles from Simple Talk
Update :-
Step1: Right click on Control Flow and select properties.
Step2: Specify the informations for CheckPointFileName, CheckpointUsage and SaveCheckPoints
CheckPointFileName = Points to the XML file location where checkpoint details will be stored by SSIS
CheckpointUsage =Specifies 3 properties (Never,IfExists,Always)
Never signifies that the checkpoints will never be used.
IfExists = Checkpoints will be used if a file exists
Always=A checkpoint file will always be used .If the file is missing then error will be thrown
SavecheckPoints =Set it true to save the checkpoints
Now for all the controls or tasks set FailPackageOnFailure property to True for the components which you want to participate in Checkpoints . This property indicates whether the package fails when the execution fails