BIML - 'AstTableNode' does not contain a definition for 'GetDropAndCreateDdl' - ssis

I am working on a BIML project to generate SSIS packages. I have a separate static class for utility methods.
I am attempting to call GetDropAndCreateDdl() to get the DDL from the souce to dynamically create a table in the destination. This should work in theory as it is referenced in multiple posts: here and here as samples.
When generating the BIML, running the sample code below, I receive an error: Error: 'AstTableNode' does not contain a definition for 'GetDropAndCreateDdl' and no accessible extension method 'GetDropAndCreateDdl' accepting a first argument of type 'AstTableNode' could be found
public static string GetDropAndCreateDDL(string connectionStringSource, string sourceTableName)
{
var sourceConnection = SchemaManager.CreateConnectionNode("Source", connectionStringSource);
var sourceImportResults = sourceConnection.ImportTableNodes(Nomenclature.Schema(sourceTableName),Nomenclature.Table(sourceTableName));
return sourceImportResults.TableNodes.ToList()[0].GetDropAndCreateDdl();
}
(Let's ignore the possibility of getting no table back or multiples for the sake of simplicity)
Looking at the varigence documentation, I don't see any reference to this method. This makes me think that there is a utility library that I am missing in my includes.
using Varigence.Biml.Extensions;
using Varigence.Biml.CoreLowerer.SchemaManagement;
What say you?
Joe

GetDropAndCreateDdl is an extension method in Varigence.Biml.Extensions.SchemaManagement.TableExtensions
ImportTableNodes returns an instance of
Varigence.Biml.CoreLowerer.SchemaManagement.ImportResults and the TableNodes is an IEnumerable of AstTableNodes
So, nothing weird there (like the table nodes in the import results being a different type)
I am not running into an issue if I have the code in-line with BimlExpress.
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#
string connectionStringSource = #"Provider=SQLNCLI11;Data Source=localhost\dev2017;Integrated Security=SSPI;Initial Catalog=msdb";
var sourceConnection = SchemaManager.CreateConnectionNode("Source", connectionStringSource);
List<string> schemaList = new List<string>(){"dbo"};
var sourceImportResults = sourceConnection.ImportTableNodes("dbo", "");
WriteLine("<!-- {0} -->", sourceImportResults.TableNodes.Count());
//var sourceImportResults = sourceConnection.ImportTableNodes(schemaList,null);
var x = sourceImportResults.TableNodes.ToList()[0];
var ddl = x.GetDropAndCreateDdl();
WriteLine("<!-- {0} -->", sourceImportResults.TableNodes.FirstOrDefault().GetDropAndCreateDdl());
#>
</Biml>
The above code results in the following expanded Biml
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<!-- 221 -->
<!-- IF EXISTS (SELECT * from sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[autoadmin_backup_configuration_summary]') AND type IN (N'V'))
DROP VIEW [dbo].[autoadmin_backup_configuration_summary]
GO
CREATE VIEW [dbo].[autoadmin_backup_configuration_summary] AS
SELECT
ManagedBackupVersion,
IsAlwaysOn,
IsDropped,
IsEnabled,
RetentionPeriod,
EncryptionAlgorithm,
SchedulingOption,
DayOfWeek,
COUNT(*) AS DatabaseCount
FROM autoadmin_backup_configurations
GROUP BY
ManagedBackupVersion,
IsAlwaysOn,
IsDropped,
IsEnabled,
RetentionPeriod,
EncryptionAlgorithm,
SchedulingOption,
DayOfWeek
GO
-->
</Biml>

Related

LineageID is string or Integer

I'm trying to fetch lineageId of output columns in a SSIS Package. I'm traversing SSIS Package as XML Document in C# code.
No issue in traversing and fetching data from the nodes in the package, but facing issue when I'm trying to fetch LineageID.
I many forums and articles said that LineageId generated in SSIS Package is a Unique Integer value for that particular package, so I'm trying to convert LineageID to an integer value. But it seems to be a String value.
Kindly clarify my doubt. Is lineageid STRING OR INT?
If it is Int then please tell me how to find it.
foreach (var x in NodeCollection)
{
var outputCollection = x.Elements("outputColumn");
foreach (var output in outputCollection)
{
string Name = output.Attribute("name").Value;
long LineageId = Convert.ToInt64(output.Attribute("lineageId").Value);
}
}
By debugging with Breakpoint "output.Attribute("lineageId").Value" is returning following value
"Package\DFT\SRC.Outputs[OLE DB Source Output].Columns[TYPE]"
I have performed a very similar exercise and found that the lineage id values were definitely integer. This is an extract from my code:
var nsmgr = new XmlNamespaceManager(doc.NameTable);
nsmgr.AddNamespace("DTS", "www.microsoft.com/SqlServer/Dts");
foreach (XmlNode childnode in doc.SelectNodes("//*[#lineageId != '' and #name != '']"))
{
XmlNode ExecutableNode = childnode.SelectSingleNode("ancestor::DTS:Executable[1]", nsmgr);
var taskName = ExecutableNode.SelectSingleNode("DTS:Property[#DTS:Name='ObjectName']", nsmgr).InnerText;
var targetColumnName = childnode.Attributes["name"].Value;
var lineageID = Convert.ToInt32(childnode.Attributes["lineageId"].Value);
We are using this to map column names to lineageid's so that we can interpret error messages thrown by our SSIS packages.
Hope this helps!

Entity Framework 5 - T4 generated context class causing 'duplicate parameter name'

I'm using EF5.0 in an ASP.NET MVC app. My Entity Model is named 'DataModel'. Included in the model is a table-valued function that exists in my MSSQL database, named MatchingEntries. It returns a table of integer ids.
I've looked at the DataModel.Context.cs file, that gets generated via the .tt (T4) template file. It has the following code in it:
[EdmFunction("DataEntities", "MatchingEntries")]
public virtual IQueryable<Nullable<int>> MatchingEntries(string term)
{
var termParameter = term != null ?
new ObjectParameter("Term", term) :
new ObjectParameter("Term", typeof(string));
return ((IObjectContextAdapter)this).ObjectContext.CreateQuery<Nullable<int>>("[DataEntities].[MatchingEntries](#Term)", termParameter);
}
The error I am getting results from using this method twice within the one query, such as:
IQueryable<int> one = db.MatchingEntries("\"one*\"");
IQueryable<int> two = db.MatchingEntries("\"two*\"");
List<int> both = one.Intersect(two).ToList();
The error is:
A parameter named 'Term' already exists in the parameter collection. Parameter names must be unique in the parameter collection.
Parameter name: parameter
Is this a known limitation of the classes generated from an EDMX for table-valued functions? With LINQ2SQL I am able to execute this a a single query to the database (that does a JOIN between the 2 outputs from MatchingEntries) and it replaces the parameter name #Term with #p0 and #p1 for the two different instances of the call. I'd like to make Entity Framework do the same.
So, my question is, how can I get EF to work in the same manner and avoid the 'Duplicate parameter' error?
My fallback is to evaluate each call to db.MatchingEntries separately, by putting ToList() after them. My other idea has been to replace the ObjectParameter name in the T4 generated Context.cs class with something randomly generated each time. These feel like hacks that I should be able to avoid.
This answer is Linq to Entities specific. This doesn't have to be done in Linq to SQL (Linqpad).
Thanks to this question I got a pointer to a viable solution:
extend the autogenerated DBContext class (partial class)
add a method with two parameters in the partial class
at calling, pass an index as second parameter
Detailed Answer:
DataEntitys.my.cs:
[EdmFunction("DataEntities", "MatchingEntries")]
public virtual IQueryable<Nullable<int>> MatchingEntries(string term, int index)
{
string param_name = String.Format("k_{0}", index);
var termParameter = term != null ?
new ObjectParameter(param_name, term) :
new ObjectParameter(param_name, typeof(string));
return ((IObjectContextAdapter)this).
ObjectContext.CreateQuery<Nullable<int>>(
String.Format("[DataEntities].[MatchingEntries](#{0})", param_name),
termParameter);
}
Call the function:
foreach (string teil in such)
{
index++;
if (teil.Trim() != "")
res = res.Join(db.MatchingEntries("\"" + teil + "*\"", index), l => l.ID, s => s.KEY, (l, s) => l);
}

Pass a variable using Script task in SSIS

I have a C# script in the ssis package as mentioned below
SqlConnection importTab = new SqlConnection(#"Server=ServerName;
Integrated Security=true;user=;pwd=;database=DBname");
I need to pass the database name (DBName) inside a variable...
May be like this
SqlConnection importTab = new SqlConnection(#"Server=ServerName;
Integrated Security=true;user=;pwd=;database="+"User::Variable" +");"
But I know I am wrong...
To use a variable in a script, first ensure that the variable has been added to
either the list contained in the ReadOnlyVariables property or the list contained in
the ReadWriteVariables property of this script task, according to whether or not your
code needs to write to the variable.
//Example of reading from a variable:
DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
//Example of writing to a variable:
Dts.Variables["User::myStringVariable"].Value = "new value";
//Example of reading from a package parameter:
int batchId = (int) Dts.Variables["$Package::batchId"].Value;
//Example of reading from a project parameter:
int batchId = (int) Dts.Variables["$Project::batchId"].Value;
//Example of reading from a sensitive project parameter:
int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
I do it like this:
When opening the script task properties you have two fields, ReadOnlyVariables and ReadWriteVariables. Write your variable name into the according field based on your needs, in your case User::Variable.
In the code you can use it like this
Dts.Variables["User::Variable"].Value.ToString()
Following code in the Script task may help you
var dbServerName = Dts.Variables["yourVariableName"].Value.ToString();
var sqlConnString = string.Format("Server=ServerName;Integrated Security=true;user=;pwd=;database={0}", dbServerName);
SqlConnection sqlConn = new SqlConnection(sqlConnString);

Retrieving column mapping info in T4

I'm working on a T4 file that generates .cs classes based on an entity model, and one of the things I'm trying to get to is the mapping info in the model. Specifically, for each field in the model I'm trying retrieve the database field name it is mapped to.
I've found that the mapping info is apparently stored in StorageMappingItemCollection, but am having an impossible time figuring out how to query it and retrieve the data I need. Has anyone worked with this class and can maybe provide guidance?
The code I have so far goes something like this (I've pasted everything up to the problematic line):
<#
System.Diagnostics.Debugger.Launch();
System.Diagnostics.Debugger.Break();
#>
<## template language="C#" debug="true" hostspecific="true"#>
<## include file="EF.Utility.CS.ttinclude"#>
<## output extension=".cs"#><#
CodeGenerationTools code = new CodeGenerationTools(this);
MetadataLoader loader = new MetadataLoader(this);
CodeRegion region = new CodeRegion(this, 1);
MetadataTools ef = new MetadataTools(this);
string inputFile = #"MyModel.edmx";
EdmItemCollection ItemCollection = loader.CreateEdmItemCollection(inputFile);
StoreItemCollection storeItemCollection = null;
loader.TryCreateStoreItemCollection(inputFile, out storeItemCollection);
StorageMappingItemCollection storageMappingItemCollection = null;
loader.TryCreateStorageMappingItemCollection(
inputFile, ItemCollection, storeItemCollection, out storageMappingItemCollection);
var item = storageMappingItemCollection.First();
storageMappingItemCollection has methods like GetItem() and such, but I can't for the life of me get it to return data on fields that I know exist in the model.
Thx in advance!
Parsing the MSL isn't really that hard with Linq to XML
string mslManifestResourceName = GetMslName(ConfigurationManager.ConnectionStrings["Your Connection String"].ConnectionString);
var stream = Assembly.GetExecutingAssembly().GetManifestResourceStream(mslManifestResourceName);
XmlReader xreader = new XmlTextReader(stream);
XDocument doc = XDocument.Load(xreader);
XNamespace xmlns = "http://schemas.microsoft.com/ado/2009/11/mapping/cs";
var items = from entitySetMap in doc.Descendants(xmlns + "EntitySetMapping")
let entityTypeMap = entitySetMap.Element(xmlns + "EntityTypeMapping")
let mappingFragment = entityTypeMap.Element(xmlns + "MappingFragment")
select new
{
EntitySet = entitySetMap.Attribute("Name").Value,
TypeName = entityTypeMap.Attribute("TypeName").Value,
TableName = mappingFragment.Attribute("StoreEntitySet").Value
};
It may be easier to parse the EDMX file as XML rather than using the StorageMappingItemCollection.

Entity Framework 4.0 Code-First Dynamic Query

I would like to query a table based on a list of KeyValuePair. With a Model-First approach, I could do the following:
var context = new DataContext();
var whereClause = new StringBuilder();
var objectParameters = new List<ObjectParameter>();
foreach(KeyValuePair<string, object> pair in queryParameters)
{
if (whereClause.Length > 0)
whereClause.Append(" AND ");
whereClause.Append(string.Format("it.[{0}] = #{0}", pair.Key));
parameters.Add(new ObjectParameter(pair.Key, pair.Value));
}
var result = context.Nodes.Where(whereClause.ToString(), parameters.ToArray());
Now I'm using a Code-First approach and this Where method is not available anymore. Fortunately, I saw an article somewhere (I can't remember anymore) which suggested that I could convert the DbContext to a IObjectContextAdapter then call CreateQuery like this:
var result = ((IObjectContextAdapter)context)
.ObjectContext.CreateQuery<Node>(whereClause.ToString(), parameters.ToArray());
Unfortunately, this throws an error:
'{ColumnName}' could not be resolved in the current scope or context. Make sure that all referenced variables are in scope, that required schemas are loaded, and that namespaces are referenced correctly.
Where {ColumnName} is the column specified in the whereClause.
Any ideas how I can dynamically query a DbSet given a list of key/value pairs? All help will be greatly appreciated.
I think your very first problem is that in the first example you are using Where on the entity set but in the second example you are using CreateQuery so you must pass full ESQL query and not only where clause! Try something like:
...
.CreateQuery<Node>("SELECT VALUE it FROM ContextName.Nodes AS it WHERE " + yourWhere)
The most problematic is full entity set name in FROM part. I think it is defined as name of the context class and name of the DbSet exposed on the context. Another way to do it is creating ObjectSet:
...
.ObjectContext.CreateObjectSet<Node>().Where(yourWhere)