SSIS flat file loading by checking conditions - ssis

Question

I've 10 flat files(.dat format) in a folder which need to be uploaded into database everyday at a scheduled time.
All files related information are present in a database like File name, file path, table name, column names and delimiter.
We need to check each file exists or not, if not, need to log an entry, "File Not Found".
If the file exists, it needs to check for a trailer record(the last record in the file which says, Count=00001000, It has to be the count of number of records in that particular file).
If the trailer record not exists, then need to log an entry "No trailer record found".If the trailer record says zero count, then a log entry has to be made "Zero count" and also, if the counts of the file are not matching a log entry is needed, "Count mismatch".
If all the conditions are satisfied then data need to be loaded into database for each of the file.
Please suggest your ideas to implement the above scenario. Thanks!!!

Answer 1

Following solution may help you to resolve the issue.
Use the For each loop container with "Item" enumerator. Since you have 10 files and if something missing you need raise then you should use this. File enumerator just iterate through the files, not raises any error.
Following are Steps.
Create following SSIS package with variables.
FileFullPath
IsValidated
For each loop enumerator should be configured as following screenshots.
Configuartion in collection:
configuration in Variable section
Inside the container have a script task. you have to mention the FileFullPath as readonly variable and IsValidate as read and write like the following screen.
Click Edit script and insert the following code.
public void Main()
{
Dts.Variables["IsValidated"].Value = true;
string fileFullPath = Dts.Variables["FileFullPath"].Value.ToString();
if (!File.Exists(fileFullPath))
{
var msg = String.Format("File is not available in location : {0}", fileFullPath);
Dts.Events.FireError(0, "Dat file loading", msg, string.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
//Read last line
String lstLine = File.ReadLines(fileFullPath).Last();
int totalCount = 0;
bool talierExists = int.TryParse(lstLine, out totalCount);
if (!talierExists)
{
var msg = String.Format("No tailer row found and last line is : {0}", lstLine);
Dts.Events.FireError(0, "Dat file loading", msg, string.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
//Total count
int fullCount = File.ReadLines(fileFullPath).Count();
if (fullCount != totalCount)
{
var msg = String.Format("No of count is not matching, tailer count = {0} and full count={1}");
Dts.Events.FireError(0, "Dat file loading", msg, string.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
Dts.Variables["IsValidated"].Value = true;
Dts.TaskResult = (int)ScriptResults.Success;
}
After that have your Data flow. Connect the script task with your data flow and right click on the connector and go to edit and configure as follows.
Your SSIS package will looks like follows.
Hope this helps!