i have a ssis project, to start with, i have a Foreach File Enumerator and inside it are multiple tasks. It works flawlessly under normal circumstances, however if there is no file present in the enumerator directory, an error occurs, the question is, how can i make a validation or some sort for it to avoid it throwing an error on runtime? thanks
img links
ssis
http://s15.postimage.org/l41py15aj/ssis.png
error
http://s15.postimage.org/rj0qupc0b/ssiserror.png
You can have a script task before Foreach loop which basically checks for files and directory .Then have a precedence constraint to halt the package if either directory or files is not present .
Create 3 variables
Name DataType
Directory String
Files String
Exists int
In the script task just check if the directory is present along with the files .You can modify the code if you need to even enumerate the subfolders or check if specific files are present in the folder
if (Directory.Exists(Dts.Variables["User::Files"].Value.ToString()))
{
if (Directory.GetFiles(Dts.Variables["User::Files"].Value.ToString()).Length != 0)
{
Dts.Variables["User::Exists"].Value = 1;
}
else
{
Dts.Variables["User::Exists"].Value = 0;
}
}
else
{
Dts.Variables["User::Exists"].Value = 0;
}
In the precedence constraint check the value for the variable Exists
Evaluation operation : Expression
Expression : #Exists==1
Update :
In the script task editor you need to add the variables in the ReadOnlyVariable section in the script tag
I would set the expression on the Disable of the Sql Task assert whether the a file name was valid.
I haven't got SSIS here but I'll look it up.
You are aware of this particular case when the error occurs and you want to handle it. What if it fails for some other unforeseen error?
Always add an 'OnError' Event Handler.
Now, after handling your error condition using the event handler, you can choose to propogate this error to fail the task (which is the default behavior) or you can set the system variable, Propogate, in your 'OnError' event handler to false. This will allow the rest of the code after the For Each to continue as normal.
Related
I have a parent child package situation where the parent is reading a SQL Server task table. Loops through the tasks and gets the connection string needed for the the task and passes it to the child package. I have created and set a variable in the parent package. I have also created and set a variable n the child package in a Script Task. The Variable is set to the Connection String of my OLE_DB connection in an expression (Package.DelayValidation = True). After the Script task runs(and works..I displayed a message box with the correct value) I run a Data Flow Task which trys to read the database using the connection string. This is where the "error code dts_e_cannotacquireconnectionfromconnectionmanager" happens. I know I have to be missing something, just can not put my finger on it.
Since both packages are available in a single project, try to select Project Reference ,in the parent package, as the ReferenceType and in the PackageNameFromProjectReference enter your child .
I am new to the Business Objects Data services.
I have to run a dataflow reading from a file. Filename should be read based on wild chars like Platform. And I want to run the dataflow only if the file exists, if file is not present , it should not error out or should not do anything but it should just move on to the next dataflow or workflow in the job.
I tried below code to check if the file exists as built_in function File_Exists cannot check the file based on wild chars.
*$FILEEXISTSFLAG= exec('/bin/ksh',' "ls xxxxxx/Platform.csv',8);*
My intention is based on the value assigned to $FILEEXISTSFLAG from above code, I will decide whether to execute the data flow or not (if $FILEEXISTSFLAG is null do nothing otherwise execute the data flow ) but its retrieving below output.
*ls: cannot access /xxxxxx/Platform.csv: No such file*
Is there any other way to achieve this?
I was able to solve the above problem by using the index function.
$FILEEXISTSFLAG is containing a value like "ls: cannot access Platform: No such file or directory ". So, I have used the index function as below. So if the output is not null for below index function, it will execute the dataflow, otherwise it will do nothing.
index( $FILEEXISTSFLAG , 'No such file',1)
Thanks,
Phani.
I need some help.
I am importing some data in .csv file from an oledb source. I don't want the headers to appear twice in the destination. If i Uncheck the "Column names in first data row" property , the headers don't get populated in the first execution as well.
Output as of now.
Col1,Col2
A,B
Col1,Col2
C,D
How can I make the package run in such a way that if the file is empty , the headers get inserted. Then if the execution happens again, headers are not included,just the data.
there was a similar thread, but wasn't able to apply the solution as how to use expressions to get the number of rows of destination itself. It was long back , so I created a new.
Your help is deeply appreciated.
-Akshay
Perhaps I'm missing something but this works for me. I am not having the read only trouble with ColumnNamesInFirstDataRow
I created a package level variable named AddHeader, type Boolean and set it to True. I added a Flat File Connection Manager, named FFCM and configured it to use a CSV output of 2 columns HeadCount (int), AddHeader (boolean). In the properties for the Connection Manager, I added an Expression for the property 'ColumnNamesInFirstDataRow' and assigned it a value of #[User::AddHeader]
I added a script task to test the size of the file. It has read/write access to the Variable AddHeader. I then used this script to determine whether the file was empty. If your definition of "empty" is that it has a header row, then I'd adjust the logic in the if check to match that length.
public void Main()
{
string path = Dts.Connections["FFCM"].ConnectionString;
System.IO.FileInfo stats = null;
try
{
stats = new System.IO.FileInfo(path);
// checking length isn't bulletproof based on how the disk is configured
// but should be good enough
// http://stackoverflow.com/questions/3750590/get-size-of-file-on-disk
if (stats != null && stats.Length != 0)
{
this.Dts.Variables["AddHeader"].Value = false;
}
}
catch
{
// no harm, no foul
}
Dts.TaskResult = (int)ScriptResults.Success;
}
I looped through twice to ensure I'd generate the append scenario
I deleted my file and ran the package and only had a header once.
The property that controls whether the column names will be included in the output file or not is ColumnNamesInFirstDataRow. This is a readonly property.
One way to achieve what you are trying to do it would be to have two data flow tasks on the control flow surface preceded by a script task. these two data flow tasks will be identical except that they will be referring to two different flat file connection managers. Again, the only difference between these two would be the different values for the ColumnsInTheFirstDataRow; one true, another false.
Use this Script task to decide whether this is the first run or subsequent runs. Persist this information and check it within the script. Either you can have a separate table for this information, or use some log table to infer it.
Following solution is worked for me.You can also try the following.
Create three variables.
IsHeaderRequired
RowCount
TargetFilePath
Get the source row counts using Execute SQL task and save it in
RowCount variable.
Have script task. Add readonly variables TargetFilePath and
RowCount. Add read and write variable IsHeaderRequired.
Edit the script and add the following line of code.
string targetFilePath = Dts.Variables["TargetFilePath"].Value.ToString();
int rowCount = (int)Dts.Variables["RowCount"].Value;
System.IO.FileInfo targetFileInfo = new System.IO.FileInfo(targetFilePath);
if (rowCount > 0)
{
if (targetFileInfo.Length == 0)
{
Dts.Variables["IsHeaderRequired"].Value = true;
}
else
{
Dts.Variables["IsHeaderRequired"].Value = false;
}
}
Dts.TaskResult = (int)ScriptResults.Success;
Connect your script component to your database
Click connection manager of flat file[i.e your target file] and go
to properties. In the expression, mention the following as shown in
the screenshot.
Map the connectionString to variable "TargetFilePath".
Map the ColumnNamesInFirstDataRow to "IsHeaderRequired".
Expression for Flat file connection Manager.
Final package[screenshot]:
Hope this helps
A solution ....
First, add an SSIS integer variable in the scope of the Foreach Loop or higher - I'll call this RowCount - and make its default value negative (this is important!). Next, add a Row Count to your Data Flow, and assign the result to the RowCount SSIS variable we just made. Third, select your Connection Manager (don't double-click) and open the Properties window (F4). Find the Expressions property, select it, and hit the ellipsis (...) button. Select the ColumnNamesInFirstDataRow property, and use an expression like this:
[#User::RowCount] < 0
Now, when your package starts, RowCount has the static value of -1 or another negative number. When the data flow starts for the first time in your loop, the ColumnNamesInFirstDataRow property will have a value of TRUE. When the first data flow completes, the row count (even if it's zero) is written to the RowCount variable. On the second interation of the loop, the Connection Manager is then reconfigured to NOT write column names...
Can anyone help:
Required: SSIS Package to loop through a folder (containing 100 files) and check whether required files (which are 5/6) are present in that folder.
Does anyone already has code for this - where we are checking for multiple files existence in the destination folder
Regards
Add a Foreach loop container to your Control Flow
Double click it and select Collection. On Enumerator, select Foreach
File Enumerator
Select your folder and the type of file
Select the return type when a file is found. The options are the
whole filename including extension and path, the name and extension
or simply the name of the file found
Select the checkbox if you want the subfolders
Click on the variables option on the left and then new variable or
select an existing variable.
At this point you have each file name on the folder. To prove it, add a script component, double click it, and your variable on the read Only Variable and click on Edit Script. Make your Main like this:
public void Main()
{
System.Windows.Forms.MessageBox.Show(Dts.Variables["FileName"].Value.ToString());
Dts.TaskResult = (int)ScriptResults.Success;
}
now, the comparison you can do several ways. I dont know where do you have the "required files" list, but assuming it is on a database, you can add a data flow task and inside of it send the filename to the DB to do the comparisson.
How do you check to see if a file does not exist in SQL Server Integration Services 2005?
Is there a native SSIS component which will just do this for you?
I have checked for file existence this using the Script Task and then branch accordingly.
You can do something like
If System.IO.File.Exists("\\Server\Share\Folder\File.Ext") Then
Dts.TaskResult = Dts.Results.Success
Else
Dts.TaskResult = Dts.Results.Failure
End If
Although there are no native components for this, there are several third party components for SSIS that you can use for this purpose.
The File System Task in SSIS is basically for move, copy, delete, etc., but does not support file existence checks.
#Raj More gave a good solution. Another way that I have used before is to create a Foreach Loop Container that loops over the file system for a file spec. If you know the name of the file you want, then you can set the name in a variable and set the spec to equal the variable in the expression tab for the Foreach Loop Container. You could also just specify or a directory or a partial file name if you don't know the exact name but know the naming convention or know there will be no other files in the folder.
If you want to take a specific action based on whether or not there is a file, then you could create a variable with a default value of 0 and create a script task in the Foreach Loop Container that increments the variable. You could also just put the commands in the Foreach Loop Container that you want to execute if you want to execute it for the existence of each individual file. If you want to take an action based on the absence of the file, then you could restrict your precedence constraint after the Foreach Loop Container so that it is restricted on constraint and expression and make it check if the counter variable is > 0.
#Raj's solution could also be used to increment the variable. Instead of using an If Else to raise an error or success result, you could do this:
C#
if (System.IO.File.Exists("\\Server\Share\Folder\File.Ext"))
{ Dts.Variables["my_case_sensitive_variable_name"].Value = Dts.Variables["my_case_sensitive_variable_name"].Value + 1;
}
VB.NET
If System.IO.File.Exists("\\Server\Share\Folder\File.Ext") Then
Dts.Variables["my_case_sensitive_variable_name"].Value = Dts.Variables["my_case_sensitive_variable_name"].Value + 1
End If
The advantage of this approach is that the package may not need to fail in the absence of a file. You could also use a variable name if the file changes that you could define either as a variable in the package or just solely created in the script task. The only short-coming of #Raj's approach is that you have to know the file name you want to check.
Another possibility is to execute a File System Task to rename the file to its existing name or copy the file to its existing location. If the file doesn't exist, then you can route the error to an action. I don't recommend this solution, but I remember using it years ago in one instance where it actually made sense. But in that particular instance, I was actually copying it to a real location.
Good luck!