How do you access the Package Variables Collection in biml? - ssis

Question

I'm just getting started with biml and bimlscript. I can see the power it holds, but so far digging through the language and API reference has been frustrating. I can't seem to find any reference online to access the package's variable collection.
I'm trying to set up this script so I can add more variables into my Variables section, and then automatically add those variables to a script task later in the process.
Here is the minimal code for my problem:
<Biml xmlns="http://schemas.varigence.com/biml.xsd" >
<Packages>
<Package Name="Load">
<Variables>
<Variable Name="ETLProcessStepID" DataType="Int32">0</Variable>
<Variable Name="TenantID" DataType="Int32">1</Variable>
</Variables>
<!-- more stuff going on in the biml -->
<# var package = RootNode.Packages.Where(loadPackage => loadPackage.Name.Contains("Load"));
foreach (var variable in package.Variables) { #>
<ReadWriteVariables VariableName="<#=variable.Name#>" />
<# }#>
</Package>
</Packages>
</Biml>
This seems to be the closest I've gotten. Unfortunately It results in:
Error 0 'System.Collections.Generic.IEnumerable<Varigence.Languages.Biml.Task.AstPackageNode>' does not contain a definition for 'Variables' and no extension method 'Variables' accepting a first argument of type 'System.Collections.Generic.IEnumerable<Varigence.Languages.Biml.Task.AstPackageNode>' could be found (are you missing a using directive or an assembly reference?).
If I'm reading the documentation right, there is a Variables collection in the Packages node. https://varigence.com/Documentation/Api/Type/AstPackageNode
If I'm not reading the documentation right, can anyone direct me to a reference on how I could access the package variables collection?

Answer 1

The first error you're running into is that your C# variable called package is going to return a collection from that Linq call. Since there should only be one element that matches it, we'll use First to just give us one of these things
var package = RootNode.Packages.Where(loadPackage => loadPackage.Name.Contains("Load")).First();
Now the tricky part and I'll actually have to check with some bigger guns on this, but I don't think you'd be able to access the current package's variable collection like that because it's not built yet. Well, at least using BIDS Helper/BimlExpress. The Biml first needs to get compiled into objects because, assuming a single select, you won't have anything in the RootNode.Packages collection. You certainly wouldn't have "Load" because you're compiling it right now.
In Mist, the paid for solution which is soon to be rebranded as BimlStudio, you could use a Transformer to accomplish this. You'd build out the Load package first and then a transformer fires off just prior to emission as a dtsx package and does whatever correction you were trying.
Consider the following test harness. It creates a simple package and then has some bimlscript immediately after it wherein I enumerate though all the packages and then for each package, I enumerate the root level Variables collection. You'll only see the "Test" message rendered. The inner calls won't fire because nothing exists yet.
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="so_38908470" >
<Variables>
<Variable Name="ETLProcessStepID" DataType="Int32">0</Variable>
<Variable Name="TenantID" DataType="Int32">1</Variable>
</Variables>
<#
string message = "<!-- {0} -->";
WriteLine(message, "test");
foreach (var package in RootNode.Packages)
{
WriteLine(message, package.Name);
foreach(var variable in package.Variables)
{
WriteLine(message, variable.Name);
}
}
#>
</Package>
</Packages>
</Biml>
The more I think about this, Tiering might be able to accomplish this with BIDS Helper/BimlExpress. Since it looks like you're trying to use the Variables defined within a package as inputs to a Script Task or Component, as long as you're using ScriptProjects type things which are parallel to a Packages collection, this might work.
Eureka
Add two Biml files to your project: Load.biml and Script.Biml. Use the following code in each. Select both and right click to generate SSIS package.
Load.biml
This is going to be your package. It is the package you started up above with a Script Task in there that is going to dump the name and value of all the user variables declared at the root of the package. But as you see, there isn't anything in the ScriptTask tag that specifies what variables or what the code is going to do.
<Biml xmlns="http://schemas.varigence.com/biml.xsd" >
<Packages>
<Package Name="Load">
<Variables>
<Variable Name="ETLProcessStepID" DataType="Int32">0</Variable>
<Variable Name="TenantID" DataType="Int32">1</Variable>
</Variables>
<Tasks>
<Script ProjectCoreName="ST_EchoBack" Name="SCR Echo Back">
<ScriptTaskProjectReference ScriptTaskProjectName="ST_EchoBack" />
</Script>
</Tasks>
</Package>
</Packages>
</Biml>
Script.biml
This biml looks like a lot but it's the same concepts as I was working with above where I enumerate though the packages collection and then work with the Variables collection. I use the biml nuggets to control the emission of the Namespace, Name and DataType properties.
<## template language="C#" tier="1" #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd" >
<ScriptProjects>
<ScriptTaskProject ProjectCoreName="ST_EchoBack" Name="ST_EchoBack" VstaMajorVersion="0">
<ReadOnlyVariables>
<!-- List all the variables you are interested in tracking -->
<#
string message = "<!-- {0} -->";
WriteLine(message, "test");
// ValidationReporter.Report(Severity.Error, "test");
foreach (var package in RootNode.Packages.Where(x=> x.Name == "Load"))
{
WriteLine(message, package.Name);
// ValidationReporter.Report(Severity.Error, package.Name);
foreach(var variable in package.Variables)
{
WriteLine(message, variable.Name);
// ValidationReporter.Report(Severity.Error, variable.Name);
#>
<Variable Namespace="<#=variable.Namespace#>" VariableName="<#=variable.Name#>" DataType="<#=variable.DataType#>" />
<#
}
}
#>
</ReadOnlyVariables>
<Files>
<File Path="ScriptMain.cs" BuildAction="Compile">using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
namespace ST_EchoBack
{
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public void Main()
{
bool fireAgain = false;
string message = "{0}::{1} : {2}";
foreach (var item in Dts.Variables)
{
Dts.Events.FireInformation(0, "SCR Echo Back", string.Format(message, item.Namespace, item.Name, item.Value), string.Empty, 0, ref fireAgain);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
}
} </File>
<File Path="Properties\AssemblyInfo.cs" BuildAction="Compile">
using System.Reflection;
using System.Runtime.CompilerServices;
[assembly: AssemblyVersion("1.0.*")]
</File>
</Files>
<AssemblyReferences>
<AssemblyReference AssemblyPath="System" />
<AssemblyReference AssemblyPath="System.Data" />
<AssemblyReference AssemblyPath="System.Windows.Forms" />
<AssemblyReference AssemblyPath="System.Xml" />
<AssemblyReference AssemblyPath="Microsoft.SqlServer.ManagedDTS.dll" />
<AssemblyReference AssemblyPath="Microsoft.SqlServer.ScriptTask.dll" />
</AssemblyReferences>
</ScriptTaskProject>
</ScriptProjects>
</Biml>
I thought I could simplify call GetBiml() the variable variable but that is going to emit the exact biml it was defined with
<Variable Name="ETLProcessStepID" DataType="Int32">0</Variable>
<Variable Name="TenantID" DataType="Int32">1</Variable>
and if that didn't have the actual value in there, it'd be legit syntax for the ReadOnly/ReadWrite variables collection. Que lástima.
Biml Script Component Source
Biml Test for echo aka Script Task