I have two files that are named like this:
CustomerReport(08022021-08032021)
ComparingReport(08022021-08032021)
I need to load the CustomerReport to a table and the ComparingReport to another table.
I tried for each loop container but I cant think of how the expression will be to pickup the file.
Im thinking of something like Customer*.csv where the * acts like a wild card but that didnt work. What can I do in this case?
Here key of the answer is to use Foreach Loop and Conditional Split.
Don't pay attention on errors, because I don't have your CSV files and tables in DB!
Create a new variable FileName - string data type
Add foreach loop and set like on screenshot
Collection Tab:
Variable Mappings Tab:
Add Data Flow Task into Foreach Loop container
Drag elements from toolbox like on a image
Flat File Source connect to one of your CSV file using Flat File Connection Manager and on connection manager, in properties > expressions, set for ConnectionString a variable FileName
Set Conditional Split like on a image
Expression for Customer is:
LEFT(
SUBSTRING(#[User::FileName],FINDSTRING((#[User::FileName]),"Customer",1),100),
FINDSTRING(SUBSTRING(#[User::FileName],FINDSTRING((#[User::FileName]),"Customer",1),100),"Report",1) - 1
) == "Customer"
Connect Conditional Split to OLE DB Destination's
NOTE: I can't run package as I said on top of this answer, Pay attention to Conditional Split, but this is the way how you need to find if part of a string is into whole string.
This is a follow-on to my previous question about FOR EACH SMO Enumerator. I now have a working FOR EACH SMO Enumberator. I set it up to loop through the tables in my data base and put the name of the table in a variable User::TableName.
Now comes the problem. I want to use each table name as the Source in a Data Flow Control Task that simply uses the table Name as the Source and an Excel Spreadsheet as the Destination. In setting up the Source for the Data Flow Task I can't find a place to say that the Source is a Variable and that it can be found in User::TableName.
Can you tell me how to do this. I tried going into the properties of the Destination Connection Manager and setting the Connection String to User::TableName, but this doesn't seem to work.
Thanks,
Dick
Any help is much appreciated. I am trying to create an SSIS package to loop through files in the folder and get the Path + filename and finally execute the stored proc with parameter as path+filename. I am not sure how to get the path+filename and insert the into the Stored proc as parameter. I have attached the screenshot for your reference:
Looks like you have the right idea in general and the link #Speedbirt186 provided has some good details but it sounds like there are a couple of nuances that I thought I might point out in regards to flow and variables.
The foreach loop can assign the entire path or the file name or file name & extension to a variable. The latter will be the most help in your case if you don't want to add a script task to split the Filename from the path. If you start by adding 5 variables to your project it will make it a little easier. 1 will be the Source Directory Path, another the Destination (Archive) Directory Path, and then 1 to hold the File Name and Extension assigned by the for each loop. Then 2 additional dynamic variables that simply combine the source directory and file name to get the source full path and the destination with file name to get the destination full path.
Next make sure you set up your database and Excel file connections. In your Excel file connection after setting it up go to Expressions in the properties window and set the "Connection String" property to SourceFullPath. This will tell the connection to change the file path at every iteration of your loop.
Now you just need to setup your loop etc. Add the fore each loop container setting a directory, filter, and choose File Name and Extension.
Now in the expression box on the collection page set the directory property to be that of your Source Directory variable.
The last part of the Fore each loop is to set your variable mappings to store the file name in your variable. so go to that tab choose your file name variable and set index to 0.
At this point you can add your data flow and setup your import just like you would with a normal file (note your default value for your file name parameter should be to an actual file with the structure you will want to import).
After your data flow drop in your Execute SQL task and set it up how you need. here is an example of direct input and you can see an easy way to reference a parameter is simply a question mark (?).
Next in your sql task setup your parameter mapping by adding in the details you need such as:
Now you are on to your file task. Drop your file task and setup as you desire, but choose your destination and source full path variables to tell the task which file to move.
that's it your are done. there is 1 more thing to note though. The way you have your precedence set in the image you posted you show going from your data flow to your sql and to your file task simultaneously. If your stored procedure relies on your file you may want to put it after your sql task. You can always change the constraint options to "completion" if you want to move the file even if your stored proc fails.
What you want to do is to create a variable in your package, call it something like Filename. In the Edit window of the Foreach you can configure that variable to be set (on the Variable Mappings page- set index to 0).
To create a variable, you will need to have the Variables window showing. Use the View menu to show it if it's not currently open.
Then when calling your stored procedure you can pass the then current value of the variable as a parameter.
This link might help: https://www.simple-talk.com/sql/ssis/ssis-basics-introducing-the-foreach-loop-container/
In SSIS, i want to read data from a flat file(Flat File Source) and assign certain column values to a variable in SSIS (like IDNo). I need this ID to retrive other information like email and other details to send email dynamically later in the process.
I did some reasearch i found that Script Component is one of the way to assign values to variables, but in our environment script component is not available. Is it possible to do that if so, can some one please provide me a sample. thanks.
You don't necessarily need script component to achieve this.
Read your IDs to a recordset (that would be a variable of type Object):
Create a variable of type Object.
In Data Flow Task create a Recordset Destination.
Pick your variable in VariableName dropdown.
Create Foreach Loop Containter:
Choose Foreach ADO Enumerator in Enumerator setting.
As ADO object source variable pick your recordset variable from the dropdown list
In Variable Mappings assign your email variable with index 0
Send the emails from the loop using email variable as an expression in Send Email Task.
Why is script component not available? It'd make things easier and help to avoid some problems; I don't use mail task anymore in favour of script task, because I ran to some problems with it in the past.
Hope this helps.
In Reporting Services I would like to add a parameter that contains data from a custom code block. Ideally, I would be able to run the following code (this is a simple testing example):
Function GetPeriods() As String()
Dim values As System.Collections.ArrayList =
New System.Collections.ArrayList()
For i as integer = 1 to 24
values.Add(i)
Next
Return values.ToArray()
End Function
and put the following in the "Text Field" of the parameter:
=Code.GetPeriods()
However, when I run the report, the parameter I apply this to is disabled and empty. Is there a different technique that should be used? Or am I doing something wrong?
If you're using SQL 2008 Reporting Services then you can have a look at this page which introduces the concept of using custom assemblies.
If you're using SQL 2005 Reporting Services then this link is the one you want.
It's a mostly trivial thing, simply compile your code into a class library and follow the instructions provided to allow your report to reference it.
You are returning an array item (an array of strings) into a text field. Instead, try returning a plain string. That should work. If you would still like to return an array list, you must basically bind it to a list control in your RDL. You can definitely do that with dataset extensions. However, I am not sure if there is any other easy way. Check the proprties of the list control and see if it allows you to directly bind to an array list.
You can create the same stored procedure on SQL Server and load parameter values from that procedure.
To access your members/functions implemented in custom code of SSRS report you should set the access modifier to "Public":
Public Function GetPeriods() As String
...
see article Writing Custom Code in SQL Server Reporting Services
I've been trying to do this same thing, set a simple list of parameter values from report code. None of the links in any of these answers shows how to do this and after quite a bit of digging around I don't think it's even possible. Yes it is possible to get the values from a database query, from a web service, or from a custom assembly, but each of these creates a lot of overhead compared to getting the list from a simple function call like =Code.GetValues(), where the function uses a For loop to create the values.
msvcyc is correct in pointing out that the parameter is expecting a string value, but the function is returning an array. I changed the return type to Array as suggested by prashant sable, but the select list is still grayed out, it does not work. And coldice is correct in saying that the access modifier should be Public.
In my digging around I found an article by James Kovac from 2005 that pointed out why this is not possible. The Parameters class has a get method, but no set method. In the VS 2008 object browser for SSRS 2008 the object name has changed, but it still does not contain a set method (see Microsoft.ReportingServices.Interfaces.IParameter.Name or .Value).
My current workaround is to just hard code the list of values, but if your value list needs to be dynamic then your only choices are database queries, web services, or custom assemblies. I think the easiest workaround of these three is to get the values from the database engine, as suggested by oleksiy.t, as long as you can write a query to return the value list you want. Your list of integers, or my list of time intervals, would both be easy queries to write. Otherwise you will need to use one of the other two workarounds.
I checked your code. The only thing that's wrong is your function returns String(). When I changed your method signature to return Array, it worked fine, in my report.
Change the signature to Function GetPeriods() As Array
Everything I've seen requires parameters and their respective settings to be part of the RDL.
That being said, if you're going to "hardcode" the values, you could create a dataset just for the report, perhaps in XML, or if it needs to be programmatically driven, do it in a web service.