This is driving me potty.
I'm trying to pass two SSIS variables to a command line executable using an Execute Process Task object.
No matter what I try, the values are not being passed through.
This is how it is set currently:
The variables are in scope and are of type string (representing integer values).
Sods law came into effect after posting this question. I tried evaluating my variables as an expression and assigning the resulting string to the arguments property.
I don't see why this would work over the example I gave, but it does, so I'm using that.
Related
I'm currently working on a PowerShell module, and I've come across something rather unusual that I cannot figure out how to duplicate. I'm using a module from Az PowerShell 3.2.0 as a reference.
I have the following example from Microsoft's Az.Dns Module:
https://learn.microsoft.com/en-us/powershell/module/az.dns/Add-AzDnsRecordConfig
$RecordSet = Get-AzDnsRecordSet -Name www -RecordType A -ResourceGroupName MyResourceGroup -ZoneName myzone.com
Add-AzDnsRecordConfig -RecordSet $RecordSet -Ipv4Address 1.2.3.4
Set-AzDnsRecordSet -RecordSet $RecordSet
The $RecordSet variable is being set locally, passed as a parameter to the Add-AzDnsRecordConfig command of this module, and the value of the $RecordSet local variable is then being automatically updated. When this variable is passed to the Set-AzDnsRecordSet command as a parameter, it contains the updated value it was assigned and not its initial value. Note that there is no additional assignment statement of the return value of Add-AzDnsRecordConfig.
How is this possible?
I know that I can define a function parameter as type [ref] or System.Management.Automation.PSReference and then pass by reference when it is called as function -param ([ref]$myVariable). I can then update the value using $myVariable.Value, but that is not what is happening here. Somehow, this variable is being passed by value, and the value is being updated back in the local scope as if it were passed by reference.
Changing the name of the local variable also does not break this functionality. I've also done a Show-Command -Name Add-AzDnsRecordConfig and I can confirm that the type is not System.Management.Automation.PSReference.
I have a need to duplicate this functionality as closely as possible, as I am building a wrapper of sorts around this, but I am not sure how Microsoft is making this magic happen within this command.
#zett42 Thank you for the concise answer. I definitely over-complicated this, and I did not realize that objects were automatically passed by reference without the need to specify it. As it turns out, I can simply reference the parameter within the function as $RecordSet.Property = "New Value".
Back in the local scope, that does update the initially defined variable.
https://johnfabry.azurewebsites.net/2015/06/26/powershell-reference-types-and-value-types/
This article also helped me to understand how this works.
This is what I'm trying to do in a script task:
long lngMaxRowsToPull = Convert.ToInt64(Dts.Variables["Project::MaxRowsPerPull"].Value);
I get an error message that the variable does not exist.
Yet Its defined as a ReadOnlyVariable to the script and it does exist as a project parameter.
So close. ;)
Your code is trying to access a variable/parameter named Project::MaxRowsPerPull
In fact, the $ is significant so you need to reference $Project::MaxRowsPerPull
Also note that you have the data type for the parameter as Int32 but are then pushing it into Int64. You can always put a smaller type into a larger container but if you tried to fill the parameter with too large a value your package will asplode.
You need to add $ to your parameter fetch name as per syntax.
long lngMaxRowsToPull = Convert.ToInt64(Dts.Variables["$Project::MaxRowsPerPull"].Value);
I have several SSIS packages that parse different files into their respective tables. Using SSIS File Watcher task, I want to create a master package that will check a folder for the files and then select the proper sub package based on the OutputVariableName using a precedence constraint. I'm not sure how to write the expression to set the variable correctly. I want to set it based on a FINDSTRING() in the file name.
Your question is about how to use the File Watcher Task from Konesans in an integration services packages.
OutputVariableName String The name of the variable into which the full file path found will be written on completion of the task. The variable specified should be of type string
Via http://www.sqlis.com/post/file-watcher-task.aspx
Create an SSIS Variable at the Package level, called CurrentFileName. Configure the File Watcher Task such that the property OutputVariableName is User::CurrentFileName. When you drop a file into the folder that the Task is watching, it will assign the full path to that variable CurrentFileName.
Your desire is to do something with that Variable with FindString function to help determine the package to fire. Since you don't specify the something I'm going to assume it's based on file name.
It's a pity you're forcing a person to use FindString to perform this task. I say that because the .NET library offers an excellent static method to determine base file names and I hate seeing people re-invent (and debug) the wheel.
I would create 3 Variables to support this endeavor.
BaseFileName - String - Evaluate as Expression = True
FileExtensionPosition - Int32 - Evaluate as Expression = True
LastSlashPosition - Int32 - Evaluate as Expression = True
The LastSlashPosition is going to use FindString to determine the last occurrence of \ in a string. The FileExtensionPosition is going to determine the last occurence of . in a string. BaseFileName will use these numbers to calculate where to slice the #[User::CurrentFileName]` string.
The lazy trick I use for finding the last X in a string is to reverse it. It's then the first element in the string and I can pass 1 as the final parameter to FindString.
The expression assigned to #[User::LastSlashPosition] is
FINDSTRING(REVERSE(#[User::CurrentFileName]), "\\", 1)
The expression assigned to #[User::FileExtensionPosition] is
FINDSTRING(REVERSE(#[User::CurrentFileName]), ".", 1)
The expression assigned to #[User::BaseFileName] then becomes
SUBSTRING((RIGHT(#[User::CurrentFileName], #[User::LastSlashPosition] -1 )), 1, LEN(RIGHT( #[User::CurrentFileName], #[User::LastSlashPosition] -1)) - #[User::FileExtensionPosition] )
Breaking that down, (RIGHT(#[User::CurrentFileName], #[User::LastSlashPosition] -1 )) translates to the base file name. If CurrentFileName equals J:\ssisdata\so\FileWatcher\CleverFile.txt then that evaluates to "CleverFile.txt" I substring that to strip out the end. You can reduce this to just a single substring operation but my brain hurts this late.
Now you are trying to use, I assume, a series of Execute Package Tasks based on a Precedence Constraint of Success and #[User::BaseFilename] == "SubPackage1 You can do this and it'll work fine, it's just that you'll need to set each one up and as you go to set up a new child, you'll have to repeat the work.
As an alternative to this approach, I'd use a ForEach enumerator and define all my key value pairs in there.
Column 0 is the value that BaseFileName could evaluate to.
Column 1 is the SSIS package I want to fire off in response.
I created two variables to support this and configured them as such
My resulting package looks like
Not shown is the File Watcher Task because I have no desire to install that on my machine. Assume that connects to the ForEach Loop.
The ForEach loop spins through the key value pairs shown above. The Script Task inside the container is there just to provide a base for the Precedence Constraint to work. I configure the precedence constraint as Success and #[User::KeyName] == #[User::BaseFileName]
I then simulate the Execute Package Task firing based on that. The actual package name would be driven by the value of #[User::KeyPackage]
And that's your over-engineered solution for the day ;)
I have a fairly straightforward SSIS package where I can't sucessfully pass the value of a package-scoped variable from the Control Flow to a Data Flow task. Consider the below diagram:
The Execute SQL task gets values from a list of "machines". This is used to control a ForEach Loop Container, which works very well. Next a script task performs some math and assigns a single number to a package scoped variable (integer type). I have added message boxes that pop up during the loop so that I can verify that the value of this variable is being set properly.
The last icon is a data flow where I want to use the variable value. I have a simple script task that contains just a message box showing me the current value of this same variable. Every time, the variable is the value that I initially set in the designer (BIDS). Therefore, the value is not being "passed" to the data flow. I have verified multiple times that the names of the variables are correct (including case sensitive values).
This should be pretty simple, and I am getting frustrated with this issue. I would greatly appreciate and suggestions or comments. Thank you!
How are you setting the package variable from your script task? It should look like this (c#):
DTS.Variables["testVariable"].Value = "some value";
Then to test it from the script component in your dataflow task:
public override void PostExecute()
{
base.PostExecute();
MessageBox.Show(Variables.testVariable, "test");
}
I did this in a test package and it worked fine.
EDIT
Also make sure that you added the variable to the ReadWriteVariables section of the properties for the script tasks.
i just wrote a m-file with some defined input in which a simulink file is called.
it worked correctly
but when I'm going to define a function based on the same m-file ( so i can give multiple inputs to it) it give's me this error :
""
Invalid matrix-format variable specified as workspace input in 'blocks/From Workspace'. The matrix
must have two dimensions and at least two columns. Complex signals of any data type and non-double
real signals must be in structure format. The first column must contain time values and the
remaining columns the data values.
""
but i'm pretty sure that variable has 2 dimension and has twoo coloumns.
i don't have any idea what to do here.
what can i do here ?
Are you saying the mfile that runs your Simulink simulation works when the mfile is a script, but not when the mfile is a function? If so, this answer may provide some insight. Despite a preference for functions, I use scripts to run Simulink parameter studies - it was just easier to set up.