SSIS package : Store value of for loop variable in an array - ssis

I am creating an SSIS package where I have to iterate through some of the selected folder and I want to store the folder names in an array to keep track of folders that I have processed. Can I keep an array in SSIS package and keep appending the value in that array?

You can store the value of a for loop variable in an array. Doing this is a little messy IMO. There's likely a cleaner approach using "out of the box" SSIS functionality as #billinkc suggested. However, here are some pointers ...
Let's go with your scenario where you have a for each loop that iterates over some files (using a Foreach File Enumerator) and you want to store the folder names in an array.
Here are some variables we'll use:
FolderList will be the array and CurrentFile will be the for loop variable. The package in its simplest form might look like this:
In the script task, the code might look like this. I've chosen to use a List<string> as my array type, but you could use something else, such as ArrayList. (Note: you'll need to add using statements for System.Collections.Generic and System.IO for the code below):
public void Main()
{
//get current directory
string directory = Path.GetDirectoryName(Dts.Variables["User::CurrentFile"].Value.ToString());
List<string> lst = new List<string>();
// if the FolderList is already a List<string> then set set it to lst
if ((Dts.Variables["User::FolderList"].Value is List<string>))
{
lst = (List<string>)Dts.Variables["User::FolderList"].Value;
}
// if the directory isn't in the list yet, then add it to the list
if(!lst.Contains(directory))
{
lst.Add(directory);
}
// update our variable with the List<string>
Dts.Variables["User::FolderList"].Value = lst;
Dts.TaskResult = (int)ScriptResults.Success;
}
Each time the Script Task is executed, you'll get a new folder added to the array. Once the for each loop is done, you may want to examine the values of the array. You can do this using a Script Task (similar to what we did above):
List<string> lst = (List<string>)Dts.Variables["User::FolderList"].Value;
// do stuff with lst
You can also iterate over the values in the array using a for each loop (use the Foreach From Variable Enumerator), which I just learned as I was walking through this (thanks!). Just set the variable to enumerate over to your array variable (here FolderList) and specify another variable (e.g. CurrentFolder) as index 0 in Variable Mappings. This worked for a List<string>, but I'm not sure what other collection types it would work with.

Related

Why are my promoted variables are not shared between Workshop modules in Carbon?

I have a carbon workspace with two workshop modules. One of them includes a button that opens another workshop module and should pass a promoted variable (an array) to the new module.
But when the second module is opened via the button it looks like the variable was never passed along.
In the network tab it should show an error like the below:
foundry.objects.workshop.app.workshopModuleParameters: Tried passing a list parameter when opening another Workshop module in Carbon, but list parameters are not currently supported by Carbon, so ignoring the parameter value
In that case there are two options, one is to move to non-array variables if possible:
The other one is to use a function that would take the array, convert it into a string with a specific delimiter and pass this string variable to the new module:
#Function()
public makeString(arrayToConvert: string[]): string{
var convertedString = arrayToConvert[0];
if(arrayToConvert.length===1){
return convertedString
}
for (let i =1; i<arrayToConvert.length; i++){
convertedString = convertedString.concat(",", arrayToConvert[i])
}
return convertedString;
}
Convert the array to a string with a variable:
And pass the string variable to the new module:
In the second module the string would be converted back into an array.
#Function()
public makeArray(stringToConvert: string): string[]{
var convertedArray = stringToConvert.split(",");
return convertedArray;
}

SSIS- How to import multiple JSON files in a folder to Oracle?

I am trying to import multiple JSON files in a folder to an Oracle database using SSIS. The code below is the JSON parser that is able to import a single file. I need this to loop through a folder and import all the files. Here is a the code in the script component to import the JSON file. Any ideas? Thank you!
public override void CreateNewOutputRows()
{
String jsonFileContent = File.ReadAllText(#"C:\Users\tngo\File\File1.json");
JavaScriptSerializer js = new JavaScriptSerializer();
List<IGData> igdatas = js.Deserialize<List<IGData>>(jsonFileContent);
foreach (IGData igdata in igdatas)
{
Output0Buffer.AddRow();
Output0Buffer.piececount = igdata.piececount;
Output0Buffer.wgt = igdata.wgt;
}
}
Since you are already in C# you can finish it off there with a foreach loop around your whole code.
string[] files = System.IO.Directory.GetFiles("C:\\Users\\tngo\File\\", "*.json");
foreach(string file in files)
{
String jsonFileContent = File.ReadAllText(file)
JavaScriptSerializer js = new JavaScriptSerializer();
List<IGData> igdatas = js.Deserialize<List<IGData>>(jsonFileContent);
foreach (IGData igdata in igdatas)
{
Output0Buffer.AddRow();
Output0Buffer.piececount = igdata.piececount;
Output0Buffer.wgt = igdata.wgt;
}
}
You'll need to use the Foreach Loop Task.
In the Forech Loop Editor do the following:
Use the Foreach File Enumerator type and point the Folder to C:\Users\tngo\File\. Your Files wildcard will be *.json, and you should check Fully qualified under Retrieve file name. After that, click on Variable Mapping on the left pane of the editor, and create a new string variable that will hold you fully qualified filename. We'll call ours ForEachLoop_StringVar for this example.
After you create the loop, drag your Script Task into the Foreach Loop, and then double click the Script Task to open the Script Task Editor. Add the string variable you created above as a ReadOnlyVariables, and then hit the Edit Script... button to pull up your script. You can then change the hard-coded filename with a reference to your variable. Your script code would then look something like this:
public override void CreateNewOutputRows()
{
String jsonFileContent = File.ReadAllText((string)Dts.Variables["User::ForEachLoop_StringVar"].Value);
JavaScriptSerializer js = new JavaScriptSerializer();
List<IGData> igdatas = js.Deserialize<List<IGData>>(jsonFileContent);
foreach (IGData igdata in igdatas)
{
Output0Buffer.AddRow();
Output0Buffer.piececount = igdata.piececount;
Output0Buffer.wgt = igdata.wgt;
}
}

AS3: How to Deep Copy an Object

i have an object that i need to copy to my SharedObject data.
The problem is that the data property of shared object is read-only, so i can't clone my 'source' object and assign it to the shared object data, i have to make a copy of them in this way:
var so: SharedObject = SharedObject.getLocal("appData");
copyObject(sourceObj, so.data);
so.flush();
and the copy method:
public static function copyObject(sourceObject:Object, destinationObject:Object):void{
// this would be the code that i need
}
Also have in mind that my object has properties that are objects, so it has inside n leves of objects. That is why i can't simply make a for each and assign all properties on the first level, so what i need is to make a DEEP copy, probably recursive. I tried for hours to make this copyObject method with no success. Also i've searched on the internet but i didn't find any object copy that suits me.
Can someone please help me with this method? I would really apreciate it!
Thank you for your help!
The solution is to write your object to a byte array, encoded it to a string(optional - you can probably save the byte array as well, haven't looked it up) and save it to your shared object.
This function will take an object and turn it into a string
public static function serializeToString(value:Object):String{
if(value==null){
throw new Error("null isn't a legal serialization candidate");
}
var bytes:ByteArray = new ByteArray();
bytes.writeObject(value);
bytes.position = 0;
return Base64.encodeByteArray(bytes);
}
This one will get your object back from a string.
public static function readObjectFromStringBytes(value:String):Object{
var result:ByteArray = Base64.decodeToByteArray( value) as ByteArray;
result.position = 0;
return result.readObject();
}
The Base 64 encoding class you can find here https://github.com/juancgarcia/screenshotify/blob/master/Downloadify-652377f/src/com/dynamicflash/util/Base64.as.
You need to implement IExternalizable on all objects you want to store this way. The implementation includes making writeExternal method called against a ByteArray when you do writeObject(), and readExternal methods, that's called against a newly created instance, so your class should write the necessary metadata in order to make your object deep-cloned, including writing property objects.
Manual on IExternalizable
And on a side note, you should not store one object in the entire so.data, you'd better assign a field in so.data and stuff your object copy in there.
For complex objects I would use RegisterClassAlias:
import flash.net.registerClassAlias;
registerClassAlias("YourClassName", YourClassName);
var so:SharedObject = SharedObject.getLocal("objectName");
so.data.yourData = YourClassName.instance;
so.flush();
For simple Object type with deep level of simple data (primitives including arrays) I would simply use JSON.stringify() and JSON.parse() when reading back the data.

SSIS: Get any flat file source from folder and cache the name as a super global variable

I'm working in SSIS and Visual Studio 2008. When executed, I need to have the SSIS package perform the following tasks:
Check a folder for a file
If a file exists take the file and use it as the source for the flat file
Store the name of the file into a global variable that I can access in other parts of my package
The package will be run by some other script. Thus we need it to check for the file every time the package runs. We are trying to prevent the scenario where we have to monitor the folder and execute the package manually when the file appears.
Any suggestions?
The easiest way would be to set up a Foreach Loop container that has all the "work" of your package inside of it (optionally, you can it as a precursor step and use a conditional expression off of it). Assuming you have 2 variables called FileName (which is what you will have the value assigned to) and an InputFolder variable that contains the "where" we should be looking
ForEach Loop Editor
Collection tab:
Enumerator = Foreach File Enumerators
Expression: Directory = #[User:InputFolder]
FileSpec: "YD.*"
Retrieve file name
* Fully qualified
Variable Mappings tab:
Variable: User::FileName
Index: 0
You can also do this via a script task, if you'd like to see that, let me know.
EDIT
This script again assumes you have the variables InputFolder and FileName defined. Create a Script Task Component and check InputFolder as a read only variable, FileName as a read/write variable.
using System;
using System.Data;
using System.IO; // this needs to be added
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
// namespace will vary
namespace ST_bc177fa7cb7d4faca15531cb700b7f11.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
#region VSTA generated code
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
public void Main()
{
string inputFolder;
string fileName;
inputFolder = Dts.Variables["InputFolder"].Value.ToString();
// File, if exists will look like YD.CCYYMMDD.hhmmss.done
string fileMask = "YD.*.done";
// this array will catch all the files matching a given pattern
string[] foundFiles = null;
foundFiles = System.IO.Directory.GetFiles(inputFolder, fileMask);
// Since there should be only one file, we will grab the zeroeth
// element, should it exist
if (foundFiles.Length > 0)
{
fileName = foundFiles[0];
// write the value to our global SSIS variable
Dts.Variables["FileName"].Value = fileName;
}
Dts.TaskResult = (int)ScriptResults.Success;
}
}
}
Here is a possible option. You can achieve this using the Foreach Loop container. Please find the example that I have provided below. Hopefully, that gives an idea.
Step-by-step process:
On the SSIS package, create 3 variables are shown in screenshot #1. Scope CheckFile represents the package name. Variable Folder will represent the folder that you would like to check for the file. Filename represents the file name to check for. Variable FilePath will be the global variable that you will need. It will be filled in with the file path value if the file exists, otherwise it will be empty.
On the package's Control Flow tab, place a Foreach Loop container and a Script Task. Script Task is to showcase that the variable retains the value after the Foreach Loop container execution is complete. Refer screenshot #2.
Configure ForEach Loop container as shown in screenshots #3 and #4.
Replace the Main() method within the Script Task with the code given under the Script task code section. This is to demonstrate the value retained by the variable FilePath.
Screenshots #5 shows no files exist in the path c:\temp\ and screenshot #6 shows the corresponding package execution.
Screenshots #7 shows the file TestFile.txt exists in the path c:\temp\ and screenshot #8 shows the corresponding package execution.
If you would like to process the file when it exists, you can place a Data Flow Task within the Foreach Loop container to do that.
Hope that helps.
Script task code:
C# code that can be used only in SSIS 2008 and above..
public void Main()
{
Variables varCollection = null;
Dts.VariableDispenser.LockForRead("User::FilePath");
Dts.VariableDispenser.GetVariables(ref varCollection);
if (String.IsNullOrEmpty(varCollection["User::FilePath"].Value.ToString()))
{
MessageBox.Show("File doesn't exist.");
}
else
{
MessageBox.Show("File " + varCollection["User::FilePath"].Value.ToString() + " exists.");
}
Dts.TaskResult = (int)ScriptResults.Success;
}
Screenshot #1:
Screenshot #2:
Screenshot #3:
Screenshot #4:
Screenshot #5:
Screenshot #6:
Screenshot #7:
Screenshot #8:

Is it possible to get all member variables in flash(AS3)?

I am trying grab all the member variables in AS3, and then foreach one i would like to process it in various ways. I would need the name and then if it is a collection of some type I would like to loop through that collection as well. I am attempting to essentially serialize in a somewhat custom fashion.
Thanks!
If you're looking to serialize an object, you will definitely want to use JSON.
JSON basically converts objects into strings and also the other way round using an encode()/serialize() and decode()/deserialize() function.
There is a built-in JSON class in AS3, and it's really easy to use.
Once you do something like:
var myObject:Object = {};
var myObjectString:String = JSON.serialize(myObject);
After getting the string, you can do all your switch logic to manipulate each of your different variables and convert it back into an object via the deserialize() function.
You could use describeType. That returns information about the object as XML. By default, you can iterate over public properties in objects. You could try something like...
// the object to iterate over
var someObj:Object = {};
for(var prop:String in someObj) {
// check to see if its something you want to iterate over
if (someObj[prop] is Array) {
// iterator over the property here
}
}
I hope this answers your question.