Script task knows there is a variable but crashes if I attempt to use it - ssis

setting up the container
Assigning the script variables
OK so no matter what I do with variables, my script task simply will not work with them. The error thrown is generic but as I'm working with SSDT I can't step through the code or do any reasonable debugging.
I have stripped this back to the point where I have uninstalled and reinstalled SSDT, and started with a new package from scratch, adding in a line of code at a time to the automatically generated code for the ScriptTask to ensure I'm not missing anything. Everything runs fine literally until I attempt to pull anything with a package variable, the code for which I've copied from the auto-help, and verified against SO and other sources.
"dts.Variables("User::variablename").value.toString()"
should work according to everything but I'm getting nowhere.
I've tried "User::FromFile", "FromFile", "Package::FromFile", "Project::FromFile" and "variables.item(1)" and none of the variations work either.
dts.variables.count shows that the script does know that there is a single variable there, and the variable does populate because I can get a subsequent data flow task to attempt opening files based on it.
Public Sub Main()
'
' Add your code here
'
Dim currFileName As String
Dim n As Integer
Dts.Events.FireInformation(0, "Script Task", "variables count: " & CType(Dts.Variables.Count, String), "", 0, True)
For n = 1 To Dts.Variables.Count
Dts.Events.FireInformation(0, "Script Task", CType(Dts.Variables.Item(n).Name.ToString, String) & " : " & CType(Dts.Variables.Item(n).Value, String), "", 0, True)
Next
' Debugging - assiging a file path explicity here does work so the loop and the code is fine
Dim filePath As String = "\\Path\File.csv"
Dts.Events.FireInformation(0, "Script Task", "Reading in file " & filePath, "", 0, True)
Try
currFileName = CType(Dts.Variables("FromFile").Value.ToString, String)
Catch
Dts.Events.FireError(999, "Reading Variable", "Can't read filename from variable name", "", 0)
End Try
Dts.Events.FireInformation(0, "Script Task", "Reading in file " & FilePath, "", 0, True)
Dim lines() As String = System.IO.File.ReadAllLines(filePath)
Dts.TaskResult = ScriptResults.Success
End Sub
Edit: Starting from scratch with the code below worked. However I am still not clear why previous attempts where I was using ".ToString()" or "CType(Dts.Variables("name of any variation").value,string)" weren't working.
Even when I wrapped all such attempts in Try-Catch loops I still got nowhere, where I would have expected VB to at least pick up the error but nothing.
Update #2
Currently on ... iteration 40(?) of "add one line/block of code in a try/catch and deal with the error". I'm still stuck why a try/catch would not prevent these invocation errors. Surely that's the whole point?
Update #3
Adding extra FireInformation events caused failures. Commenting them out didn't change the failures. Restarting VS and the script is completing in seconds ... having done nothing. Commenting out everything apart from #billinkc's code which works ... still does nothing. Losing the will to live here.
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj,Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()

I believe your root issue is that you're using the function ToString as a pointer and not invoking it. i.e. Value.ToString instead of Value.ToString() My VB is rusty so I could be off on that. Let's explore
I created an SSIS Variable, a package Parameter and a Project Parameter, all called FromFile and initialized them with strings to verify I am seeing expected values.
I added a Script Task, set the language to VB and added the above variables to my ReadOnlyVariables collection User::FromFile,$Package::FromFile,$Project::FromFile
I make great use of the following script within my SSIS packages. It enumerates through whatever variables it has access to and pushes the results to the OnInformation event stream. In Visual Studio, this will show in the Execution Results panel and the Output tab - which is way more useful. When running in the SSISDB, unless you've turned off Information events, you'll see it recorded in SSISDB.catalog.operation_messages
Public Sub Main()
Dim fireAgain As Boolean = False
For Each item As Variable In Dts.Variables
Dim template As String = "{0}::{1}->{2}"
Dim message As String = ""
message = String.Format(template, item.Namespace, item.Name, item.Value)
Dts.Events.FireInformation(0, "Scr Echo", message, "", 0, fireAgain)
Next
Dts.TaskResult = ScriptResults.Success
End Sub
In the Output tab, here's what I see
Information: 0x0 at SCR Echo Back, Scr Echo: $Package::FromFile->I am package parameter
Information: 0x0 at SCR Echo Back, Scr Echo: $Project::FromFile->I am project parameter
Information: 0x0 at SCR Echo Back, Scr Echo: User::FromFile->Hello world from Variable
As an added bonus, we see that the Parameters, Package:: and Project:: won't work because that's now how those parameters are addressed. The would be $Package::FromFile and $Project::FromFile
Now that I know I am correctly able to access the value of my variables or parameters, let's dig in an see if I can either reproduce your error or provide working code. I don't understand enough of what your code expects. You have two variables, filePath and currFileName. filePath is initialized to a fully qualified path, maybe? You're either going with a UNC but no share specified or a local path where you've half double escaped the slashes. We assign a value to currFileName which is from our Variable.
However, we use the filePath variable as the source for our ReadAllLines.
Public Sub Main()
Dim filePath As String = "C:\ssisdata\Input\SO_67660730.DoesNotExist.txt"
Dim fireAgain As Boolean = False
Dim lines() As String
If (False) Then
For Each item As Variable In Dts.Variables
Dim template As String = "{0}::{1}->{2}"
Dim message As String = ""
message = String.Format(template, item.Namespace, item.Name, item.Value)
Dts.Events.FireInformation(0, "Scr Echo", message, "", 0, fireAgain)
Next
End If
' Populate our filePath local variable with the value from the Variable collection
' Option A Blindly assume the first element in our collection is what we want
filePath = Dts.Variables(0).Value.ToString()
Dts.Events.FireInformation(0, "Option A", filePath, "", 0, fireAgain)
' Fully qualified named value
filePath = Dts.Variables("User::FromFile").Value.ToString()
Dts.Events.FireInformation(0, "Option B", filePath, "", 0, fireAgain)
' Either way, we have the file path available to us, let's read the data
If (System.IO.File.Exists(filePath)) Then
lines = System.IO.File.ReadAllLines(filePath)
Dts.Events.FireInformation(0, "Rows Read", lines.Length.ToString(), "", 0, fireAgain)
Else
Dts.Events.FireInformation(0, "No file found", filePath, "", 0, fireAgain)
End If
Dts.TaskResult = ScriptResults.Success
End Sub
Results being
Information: 0x0 at SCR Read File, Option A: C:\ssisdata\Input\SO_67660730.txt
Information: 0x0 at SCR Read File, Option B: C:\ssisdata\Input\SO_67660730.txt
Information: 0x0 at SCR Read File, Rows Read: 3
If you persist in using VB.NET as your language, you're going to be need to find blogs and samples that were built on 2005/2008 version of SSIS as that was the only language available to us. Most of the world has adopted C# as their primary language for SSIS Script and Tasks but Ben Weissman with Solisyon uses VB in his examples.

Related

How adding ticket attachments with the Zendesk API using vba in Access [duplicate]

Overview
I have a Microsoft Word Add-In, written in VBA (Visual Basic for Applications), that compresses a document and all of it's related contents (embedded media) into a zip archive. After creating the zip archive it then turns the file into a byte array and posts it to an ASMX web service. This mostly works.
Issues
The main issue I have is transferring large files to the web site. I can successfully upload a file that is around 40MB, but not one that is 140MB (timeout/general failure).
A secondary issue is that building the byte array in the VBScript Word Add-In can fail by running out of memory on the client machine if the zip archive is too large.
Potential Solutions
I am considering the following options and am looking for feedback on either option or any other suggestions.
Option One
Opening a file stream on the client (MS Word VBA) and reading one "chunk" at a time and transmitting to ASMX web service which assembles the "chunks" into a file on the server.
This has the benefit of not adding any additional dependencies or components to the application, I would only be modifying existing functionality. (Fewer dependencies is better as this solution should work in a variety of server environments and be relatively easy to set up.)
Question:
Are there examples of doing this or any recommended techniques (either on the client in VBA or in the web service in C#/VB.NET)?
Option Two
I understand WCF may provide a solution to the issue of transferring large files by "chunking" or streaming data. However, I am not very familiar with WCF, and am not sure what exactly it is capable of or if I can communicate with a WCF service from VBA. This has the downside of adding another dependency (.NET 3.0). But if using WCF is definitely a better solution I may not mind taking that dependency.
Questions:
Does WCF reliably support large file transfers of this nature? If so, what does this involve? Any resources or examples?
Are you able to call a WCF service from VBA? Any examples?
I ended up implementing option one referenced in the original question.
I "chunk" the file in VBA and transfer each "chunk" to the web service. I based the VBA portion of the solution on the code found here: Copy Large File by Chunk with Progress Notification. Instead of copying to the file system, however, I send it to the server.
The Code: VBA Land
Here is the (fugly) VBA code that creates the file chunks:
Function CopyFileByChunk(fileName As String, sSource As String) As Boolean
Dim FileSize As Long, OddSize As Long, SoFar As Long
Dim Buffer() As Byte, f1 As Integer, ChunkSize As Long
On Error GoTo CopyFileByChunk_Error
f1 = FreeFile: Open sSource For Binary Access Read As #f1
FileSize = LOF(f1)
If FileSize = 0 Then GoTo Exit_CopyFileByChunk ' -- done!
ChunkSize = 5505024 '5.25MB
OddSize = FileSize Mod ChunkSize
Dim index As Integer
index = 0
If OddSize Then
ReDim Buffer(1 To OddSize)
Get #f1, , Buffer
index = index + 1
SoFar = OddSize
If UploadFileViaWebService(Buffer, fileName, index, SoFar = FileSize) Then
g_frmProgress.lblProgress = "Percent uploaded: " & Format(SoFar / FileSize, "0.0%")
Debug.Print SoFar, Format(SoFar / FileSize, "0.0%")
DoEvents
Else
GoTo CopyFileByChunk_Error
End If
End If
If ChunkSize Then
ReDim Buffer(1 To ChunkSize)
Do While SoFar < FileSize
Get #f1, , Buffer
index = index + 1
SoFar = SoFar + ChunkSize
If UploadFileViaWebService(Buffer, fileName, index, SoFar = FileSize) Then
g_frmProgress.lblProgress = "Percent uploaded: " & Format(SoFar / FileSize, "0.0%")
Debug.Print SoFar, Format(SoFar / FileSize, "0.0%")
DoEvents
Else
GoTo CopyFileByChunk_Error
End If
Loop
End If
CopyFileByChunk = True
Exit_CopyFileByChunk:
Close #f1
Exit Function
CopyFileByChunk_Error:
CopyFileByChunk = False
Resume Exit_CopyFileByChunk
End Function
Here is the referenced VBA method that uploads the chunks to the server:
Public Function UploadFileViaWebService(dataChunk() As Byte, fileName As String, index As Integer, lastChunk As Boolean) As Boolean
On Error GoTo ErrHand
Dim blnResult As Boolean
blnResult = False
'mdlConvert.SetProgressInfo "Connecting to the web server:" & vbNewLine & _
DQUOT & server_title() & DQUOT
If InternetAttemptConnect(0) = 0 Then
On Error Resume Next
Dim strSoapAction As String
Dim strXml As String
strXml = "<?xml version=""1.0"" encoding=""utf-8""?>" & _
"<soap:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">" & _
"<soap:Body>" & _
"<UploadZipFile xmlns=""http://something.com/"">" & _
"<zipBytes></zipBytes>" & _
"<index>" & index & "</index>" & _
"<isLastChunk>" & IIf(lastChunk, 1, 0) & "</isLastChunk>" & _
"</UploadZipFile>" & _
"</soap:Body>" & _
"</soap:Envelope>"
Dim objXmlhttp As Object
Dim objDom As Object
Set objXmlhttp = New MSXML2.xmlhttp
' Load XML
Set objDom = CreateObject("MSXML2.DOMDocument")
objDom.LoadXML strXml
'insert data chunk into XML doc
objDom.SelectSingleNode("//zipBytes").dataType = "bin.base64"
objDom.SelectSingleNode("//zipBytes").nodeTypedValue = dataChunk
' Open the webservice
objXmlhttp.Open "POST", webServiceUrl, False
' Create headings
strSoapAction = "http://something.com/UploadZipFile"
objXmlhttp.setRequestHeader "Content-Type", "text/xml; charset=utf-8"
objXmlhttp.setRequestHeader "SOAPAction", strSoapAction
' Send XML command
objXmlhttp.send objDom.XML
' Get all response text from webservice
Dim strRet
strRet = objXmlhttp.responseText
' Close object
Set objXmlhttp = Nothing
Set objDom = Nothing
'get the error if any
Set objDom = CreateObject("MSXML2.DOMDocument")
objDom.LoadXML strRet
Dim isSoapResponse As Boolean
isSoapResponse = Not (objDom.SelectSingleNode("//soap:Envelope") Is Nothing)
Dim error As String
If Not isSoapResponse Then
error = "Woops"
Else
error = objDom.SelectSingleNode("//soap:Envelope/soap:Body/soap:Fault/faultstring").text
End If
If error <> "" Then
ShowServerError error, True
blnResult = False
Else
Err.Clear 'clear the error caused in the XPath query above
blnResult = True
End If
'close dom object
Set objDom = Nothing
Else
GetErrorInfo "UploadFileViaWebService:InternetCheckConnection"
End If
ErrHand:
If Err.Number <> 0 Then
ShowError Err, "UploadFileViaWebService"
blnResult = False
End If
UploadFileViaWebService = blnResult
End Function
The Code: C# ASMX Web Service
Now, on the server side, the web service method accepts a few important parameters.
string fileName: The name of the
file (each chunk has the same file
name)
byte[] zipBytes: The contents
of each chunk
int index: The index
(used in conjunction with fileName
to provide unique ordered partial
files on the file system)
bool isLastChunk: This is the "i'm done -
go ahead and merge all the "chunks"
and clean up after yourself" flag.
int index and bool isLastChunk. With this context provided from the VBA world, I know enough to save each of these file chunks and then combine them when the isLastChunk flag is true.
/// <summary>
/// Accepts a chunk of a zip file. Once all chunks have been received, combines the chunks into a zip file that is processed.
/// </summary>
/// <param name="fileName">Name of the file.</param>
/// <param name="zipBytes">The collection of bytes in this chunk.</param>
/// <param name="index">The index of this chunk.</param>
/// <param name="isLastChunk">if set to <c>true</c> this is the last chunk.</param>
/// <returns>Whether the file was successfully read and parsed</returns>
/// <exception cref="ParserException">An error occurred while trying to upload your file. The details have been written to the system log.</exception>
[WebMethod]
public bool UploadZipFile(string fileName, byte[] zipBytes, int index, bool isLastChunk)
{
try
{
const string ModuleRootUrl = "/Somewhere/";
string folderName = HostingEnvironment.MapPath("~" + ModuleRootUrl);
string fullDirectoryName = Path.Combine(folderName, Path.GetFileNameWithoutExtension(fileName));
try
{
if (!Directory.Exists(fullDirectoryName))
{
Directory.CreateDirectory(fullDirectoryName);
}
string pathAndFileName = Path.Combine(fullDirectoryName, AddIndexToFileName(fileName, index));
using (var stream = new MemoryStream(zipBytes))
{
WriteStreamToFile(stream, pathAndFileName);
}
if (isLastChunk)
{
try
{
MergeFiles(fullDirectoryName, fileName, index);
// file transfer is done.
// extract the zip file
// and do whatever you need to do with its contents
// we'll assume that it works - but your "parsing" should return true or false
return true;
}
finally
{
DeleteDirectoryAndAllContents(fullDirectoryName);
}
}
}
catch
{
DeleteDirectoryAndAllContents(fullDirectoryName);
throw;
}
}
return false;
}
Here is the C# code that writes each incoming chunk the the file system:
/// <summary>
/// Writes the contents of the given <paramref name="stream"/> into a file at <paramref name="newFilePath"/>.
/// </summary>
/// <param name="stream">The stream to write to the given file</param>
/// <param name="newFilePath">The full path to the new file which should contain the contents of the <paramref name="stream"/></param>
public static void WriteStreamToFile(Stream stream, string newFilePath)
{
using (FileStream fs = File.OpenWrite(newFilePath))
{
const int BlockSize = 1024;
var buffer = new byte[BlockSize];
int numBytes;
while ((numBytes = stream.Read(buffer, 0, BlockSize)) > 0)
{
fs.Write(buffer, 0, numBytes);
}
}
}
Here is the C# code to merge all of the zip file "chunks":
/// <summary>
/// Merges each file chunk into one complete zip archive.
/// </summary>
/// <param name="directoryPath">The full path to the directory.</param>
/// <param name="fileName">Name of the file.</param>
/// <param name="finalChunkIndex">The index of the last file chunk.</param>
private static void MergeFiles(string directoryPath, string fileName, int finalChunkIndex)
{
var fullNewFilePath = Path.Combine(directoryPath, fileName);
using (var newFileStream = File.Create(fullNewFilePath))
{
for (int i = 1; i <= finalChunkIndex; i++)
{
using (var chunkFileStream = new FileStream(AddIndexToFileName(fullNewFilePath, i), FileMode.Open))
{
var buffer = new byte[chunkFileStream.Length];
chunkFileStream.Read(buffer, 0, (int)chunkFileStream.Length);
newFileStream.Write(buffer, 0, (int)chunkFileStream.Length);
}
}
}
}
I've transmitted large files like this using MTOM encoding.
More info on MTOM here: http://msdn.microsoft.com/en-us/library/aa395209.aspx
You can download an MTOM sample here: http://msdn.microsoft.com/en-us/library/ms751514.aspx
Check out Bustamante's book on WCF if you want more on MTOM.
As for the VBA call, I'm not an expert in that area therefore I don't have any info in regards to it.

Read whole appsettings.json file as a string

I wish to read the entire contents of my appsettings.json file into NewtonSoft.JSON so I can parse it using NewtonSoft.
This is due to being able to use full JSON paths with NewtonSoft (filtering etc.)
I basically want to just read my entire appsettings.json file as a string.
I already have it working in regardings to a Configuration.
Private Shared Function InitConfig() As IConfigurationRoot
Try
Dim builder = New ConfigurationBuilder() _
.AddJsonFile("appsettings.json", True, True) _
.AddEnvironmentVariables()
Return builder.Build
Catch ex As Exception
Console.WriteLine(ex.Message)
Return Nothing
End Try
End Function
However I don't want to choose specifics i.e. config("test1:test:bot_token"). I wish to just read the entire string, however I don't appear to be able to get this from the ConfigurationRoot.
Cheers
Went about this all the wrong way - just used a streamreader to read the appsettings.json file.
Complete brain fart moment.
Dim tr As TextReader = New StreamReader("appsettings.json")
Dim stream = tr.ReadToEnd

JSON to DataTable net with varing roots

This is my first dealing with JSON and unfortunately I have to do it in VB.Net, company policy. So I have searched high and low and tried a lot of examples even converted some of the C# code to VB.Net but I can't seem to find a complete solution.
I receive a JSON string like this:
{
"65080007":{
"partNo":"ATD000007",
"description":"Swingarm Hat Platform",
"quantity":4,
"assemblyseq":""
},
"65080143":{
"partNo":"ATD000143",
"description":"ASY Gas Spring Bracket",
"quantity":2,
"assemblyseq":""
},
"65080071":{
"partNo":"ATD000071",
"description":"TT Gas Spring",
"quantity":2,
"assemblyseq":""
},
"65080147":{
"partNo":"ATD000147",
"description":"ASY Lateral Hinge",
"quantity":8,
"assemblyseq":""
},
"65085181":{
"partNo":"RD0181",
"description":"ASY KIT Bolt, Carriage, 0.375 in x 16, 1.5 in (x45) & Nut, Flange, 0.375 in x 16 (x45)",
"quantity":1,
"assemblyseq":""
},
"65080796":{
"partNo":"ATD000796",
"description":"Decal, TT Equipped, Rectangular, 5 in x 10 in",
"quantity":1,
"assemblyseq":""
},
"65080797":{
"partNo":"ATD000797",
"description":"Decal, TT Open/Close, Triangular, 12 in x 8 in",
"quantity":1,
"assemblyseq":""
},
"65080745":{
"partNo":"ATD000745",
"description":"",
"quantity":1,
"assemblyseq":""
}
}
What I need to do bind or assign this data to a DataGridView.DataSource.
I've seen some examples but I can't set it as a DataSource.
I've tried this example:
Sub Main()
Dim json_result = GetJson()
Dim table = JsonConvert.DeserializeAnonymousType(json_result, (DataTable))
Dim newJString = Newtonsoft.Json.JsonConvert.SerializeObject(table, Newtonsoft.Json.Formatting.Indented)
Console.WriteLine("Re-serialized JSON: ")
Console.WriteLine(newJString)
Console.WriteLine("")
End Sub
Public Function GetJson() As String
Dim json_result As String = <![CDATA[
' I used the above json string
Return json_result
End Function
I have made my JSON classes to deserialise the JSON and tried JsonConvert.Deserialize keep getting an error it epected a array and found an object.
Public Class Jobs
'<JsonProperty("partno")>
Public Property PartNo As String
' <JsonProperty("description")>
Public Property Description As String
'<JsonProperty("quantity")>
Public Property Quantity As String
'<JsonProperty("assemblyseq")>
Public Property Assemblyseq As String
End Class
The issue I think is the root properties "65080797" these numbers will not be the same every time we get the JSON back from NetSuite.
So I tried to:
Dim obj = JsonConvert.DeserializeObject(Of Jobs)(result)
Console.WriteLine(obj.PartNo) it comes out PartNo = nothing
So I've tried this:
Dim resultA = JsonUtil.Deserialize(Of Jobs)(result, ignoreRoot:=True)
Module JsonUtil
Function Deserialize(Of T As Class)(ByVal json As String, ByVal ignoreRoot As Boolean) As T
Return If(ignoreRoot, JObject.Parse(json)?.Properties()?.First()?.Value?.ToObject(Of T)(), JObject.Parse(json)?.ToObject(Of T)())
End Function
End Module
This gives me the first group:
ATD000007
Swingarm Hat Platform
4
The assembly number was blank.
I'm open for any suggestions on how I can get the above JSON into a data table or DataGridView or how to make a list without the root "65080797" this number will be unique with every response.
The people that designed this response string refuses to remove the root properties.
Thank you for taking the time to read this mess.
All comments/suggestions are appreciated.
Yes, Json array looks like [something] instead of {somthing}, you could convert it to array if you want but you can also do it in different ways and even without any external library, you could create a datatable then bind it to datagridview or you could add data directly to datagridview.
Anyway, I made a datatable for you.
Imports System.Web.Script.Serialization 'for reading of JSON (+add the reference to System.Web.Extensions library)
Dim JSONC = New JavaScriptSerializer().Deserialize(Of Dictionary(Of String, Dictionary(Of String, String)))(JSON) 'you could do it in different ways too
Dim NewDT As New DataTable
'Create Columns
For Each key In JSONC.First.Value.Keys
'"65080007" <first
'{ "partNo":"ATD000007", "description":"Swingarm Hat Platform", "quantity":4, "assemblyseq":"" } <first.value
' "partNo" <first.value.key(s) :"ATD000007" <first.value.value(s)
NewDT.Columns.Add(key)
Next
'Add Rows
For Each item In JSONC
NewDT.Rows.Add(item.Value.Values.ToArray)
Next
DataGridView1.DataSource = NewDT
If you have unusually structured JSON like that, your best bet is probably to resort to using raw JObject and JToken handling instead of trying to get JSON.NET to deserialize into structured data automatically.
Create a method that will convert an individual JToken into the Job class you described in your question. I've added a Key property to identify those unique keys in the JSON that were problematic for you; if you don't need these, just remove the .Key bit from the example below.
Public Function CreateJob(data As JToken)
Return New Job With
{
.Key = data.Path,
.PartNo = data("partNo"),
.Description = data("description"),
.Quantity = data("quantity"),
.Assemblyseq = data("assemblyseq")
}
End Function
Once you have Job objects being created, you can walk the entire structure and turn it into an array of jobs with the sample code below.
Dim result As JObject = JsonConvert.DeserializeObject(json)
Dim jobs As Job() = result.Values().Select(Of Job)(AddressOf CreateJob).ToArray()
Once you have an array of Job objects, binding to a DataSource should be trivial.

In Excel VBA on Windows, for parsed JSON variables what is this JScriptTypeInfo anyway?

answering my own question here.
I have done some work with JSON in Excel VBA and lots of findings to post which I will do so in Q & A format
https://stackoverflow.com/help/self-answer https://blog.stackoverflow.com/2011/07/its-ok-to-ask-and-answer-your-own-questions/
So elsewhere on stackoverflow one can see questions about parsing JSON in VBA but they seem to miss a trick or two.
To begin with, I resile from using custom JSON parsing libraries and instead use the ScriptControl's Eval method as the basis of all my JSON code.
And also we express a preference from native Microsoft solutions.
Here is a prior question In Excel VBA on Windows, how to mitigate issue of dot syntax traversal of parsed JSON broken by IDE's capitalisation behaviour? upon which this question builds. It shows how using VBA.CallByName is more robust
than using the dot syntax to traverse a parsed JSON object. Also another prior question In Excel VBA on Windows, how to loop through a JSON array parsed? shows how it also can
be used to access array elements. But CallByName returns a curious variable type that appears in Watch window as Object/JScriptTypeInfo
and if one type Debug.Print in the immediate window (or hovers over the variable) one gets the uninformative "[object Object]". In another question
in the series In Excel VBA on Windows, how to get stringified JSON respresentation instead of “[object Object]” for parsed JSON variables? I present some debugging "sugar" that allows nice inspection of variables. In a fourth question In Windows Excel VBA,how to get JSON keys to pre-empt “Run-time error '438': Object doesn't support this property or method”?, whilst investigating
how to query a JSON object for a member I discover a hasOwnProperty() method that seems attached to a JScriptTypeInfo object.
So in this question I ask, what exactly is this JScriptTypeInfo anyway?
This is Question 5 of series of 5. Here is the full series
Q1 In Excel VBA on Windows, how to mitigate issue of dot syntax traversal of parsed JSON broken by IDE's capitalisation behaviour?
Q2 In Excel VBA on Windows, how to loop through a JSON array parsed?
Q3 In Excel VBA on Windows, how to get stringified JSON respresentation instead of “[object Object]” for parsed JSON variables?
Q4 In Windows Excel VBA,how to get JSON keys to pre-empt “Run-time error '438': Object doesn't support this property or method”?
Q5 In Excel VBA on Windows, for parsed JSON variables what is this JScriptTypeInfo anyway?
One possible place to look is in the type library for the ScriptControl as this is the library which emanates this type.
The full details of this type on my machine are
Libary Name: Microsoft Script Control 1.0 (Ver 1.0)
LIBID: {0E59F1D2-1FBE-11D0-8FF2-00A0D10038BC}
Location: C:\wINDOWS\SysWOW64\msscript.ocx
Using both VBA IDE's Object Browser and OLEVIEW.exe which disassembles type library I can no trace of the interface JScriptTypeInfo or the method hasOwnProperty.
But isn't it the case that the script engine hosts language implementations, such as VBScript and JScript (Microsoft name for Javascript).
So perhaps we should hunt for a JScript implementation DLL and indeed there is one here are details
Libary Name: Microsoft JScript Globals
LIBID: {3EEF9759-35FC-11D1-8CE4-00C04FC2B085}
Location: C:\wINDOWS\SysWOW64\jscript.dll
which on my machine is not registered and so not in my list of Tools->References libraries or in OLEVIEW.exe. I was lucky to find whilst poking around.
Here is some output from OLEVIEW giving an exceprt of the type library
[
uuid(3EEF9758-35FC-11D1-8CE4-00C04FC2B097)
]
dispinterface ObjectInstance {
properties:
methods:
[id(0x0000044c)]
StringInstance* toString();
[id(0x0000044d)]
StringInstance* toLocaleString();
[id(0x0000044e)]
VARIANT hasOwnProperty(VARIANT propertyName);
[id(0x0000044f)]
VARIANT propertyIsEnumerable(VARIANT propertyName);
[id(0x00000450)]
VARIANT isPrototypeOf(VARIANT obj);
[id(0x00000451)]
ObjectInstance* valueOf();
};
This above shows the hasOwnProperty to be a method of a IDispatch interface (or dispinterface) necessary to work with VBA object's declared of type Object (e.g. Dim foo as Object)
Registering the type library with regsvr32 appears to do nothing. One must browse to the file in Tools References to view in VBA's object browser.
We can be pretty sure about this JScript.dll file because using Process Explorer we can see the dll being loaded when executing the line oScriptEngine.Language = "JScript"
In the lack of a registered type library I loaded the file JScript.dll into Notepad++ and searched for .J.S.c.r.i.p.t.T.y.p.e.I.n.f.o as a regular expression and found a hit. Bingo!
Not only is there an ObjectInstance which would describe most of the variables a VBA program encounters but also there is an ArrayInstance which is intriguing, perhaps we can use Javascript's own array functions
or at least a subset as documented in JScript.dll's type library. Here is some sample code
'Tools->References->
'Microsoft Script Control 1.0; {0E59F1D2-1FBE-11D0-8FF2-00A0D10038BC}; C:\Windows\SysWOW64\msscript.ocx
'and FYI/browsing capabilities Microsoft JScript Globals; C:\wINDOWS\SysWOW64\jscript.dll
Option Explicit
Private Sub TestJSONParsingWithCallByName5()
Dim oScriptEngine As ScriptControl
Set oScriptEngine = New ScriptControl
oScriptEngine.Language = "JScript"
Dim sJsonString(0 To 1) As String
sJsonString(0) = "{'key1': 'value1' ,'key2': { 'key3': 'value3' } }"
sJsonString(1) = "[ 1234, 2345, 3456, 4567, 5678, 6789 ]"
Dim objJSON(0 To 1) As Object
Set objJSON(0) = oScriptEngine.Eval("(" + sJsonString(0) + ")")
Set objJSON(1) = oScriptEngine.Eval("(" + sJsonString(1) + ")")
Debug.Assert objJSON(0).hasOwnProperty("key1")
Debug.Assert objJSON(0).hasOwnProperty("key2")
Debug.Assert CallByName(objJSON(1), "length", VbGet) = 6
Debug.Assert CallByName(objJSON(1), "0", VbGet) = "1234"
'* Is objJSON(1) an ArrayInstance?
'* does it support the reverse method of the ArrayInstance object?
'Call objJSON(1).Reverse '* reverse gets capitalised into Reverse ... grrrr
Call CallByName(objJSON(1), "reverse", VbMethod) '* so use CallByName as solution to "helpful" capitalisation
'* Yes, the elements are reversed!
Debug.Assert CallByName(objJSON(1), "length", VbGet) = 6
Debug.Assert CallByName(objJSON(1), "0", VbGet) = "6789"
Stop
'** And now we know objJSON(1) is an ArrayInstance we can have some fun with array operations
Dim objSplice As Object
Set objSplice = CallByName(objJSON(1), "splice", VbMethod, 2, 1)
Debug.Assert CallByName(objJSON(1), "length", VbGet) = 5
Debug.Assert CallByName(objSplice, "length", VbGet) = 1
Dim objSlice As Object
Set objSlice = CallByName(objJSON(1), "slice", VbMethod, 2)
Debug.Assert CallByName(objJSON(1), "length", VbGet) = 5
Debug.Assert CallByName(objSlice, "length", VbGet) = 3
Stop
Call CallByName(objJSON(1), "sort", VbMethod)
Debug.Assert CallByName(objJSON(1), "join", VbMethod) = "1234,2345,3456,5678,6789"
Debug.Assert CallByName(objJSON(1), "join", VbMethod, " ") = "1234 2345 3456 5678 6789"
Stop
Debug.Assert CallByName(objJSON(1), "pop", VbMethod) = "6789"
Debug.Assert CallByName(objJSON(1), "length", VbGet) = 4
Stop
End Sub
SUMMARY: JScriptTypeInfo is something to show in the VBA IDE watch window and the return of the VBA function TypeName() but which really hides a number of objects that can be found in JScript.dll.
I suppose it can be described as polymorphic, perhaps better to describe it as late binding. To view capabilities use Tools-References and browse to JScript.dll.

SSIS Script Component or Task to check File Line Terminators and Fail if NOT CRLF

I'm a little new to using scripts for my ETL work and I couldn't find anything related to this other than to use a script to replace LF or CRLF with a value. Is it possible to use a script or something else to validate that my file uses CRLF line terminators only, and if it is anything but CRLF it fails the job.
I'm looking to fail this job so then I can report to the agency sending files that they need to follow specific format and so the only files loaded are CRLF files.
Thanks,
Found out a way to handle what I was asking. I ended up creating a script task before my Data flow to check the file to see if it contained "\r\n". With this I used two package variables and passed them through my script. Those variables were "FileName" (could be file path but I used the same name as what was being used in the package), "ErrorMessage" and "IsCrLf". The "IsCrLf" variable is a boolean variable which basically just checks to see if "\r\n" exists in the file. If not, the ErrorMessage will get populated and passed through to an e-mail alert.
Here is my code for my task:
public void Main()
{
using (StreamReader r = new StreamReader(Dts.Variables["User::FileName"].Value.ToString()))
{
string s1 = r.ReadToEnd();
string s2 = "\r\n";
bool b = s1.Contains(s2);
if (b)
{
Dts.Variables["User::IsCrLf"].Value = true;
}
else
{
Dts.Variables["User::ErrorMessage"].Value = Dts.Variables["User::FileName"].Value.ToString()+Environment.NewLine+"File does not contain the expected CRLF format.";
Dts.Events.FireError(0, "Error", "File does not contain the expected CRLF format.", string.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
}
}