I have an Excel Source which has got 1000 rows with some 10 columns and one of the column is a Date Field ,We have to retrieve the minimum date value and assign it to a variable in ssis .Could you guys provide me a script or steps to map that value to the variable...So that i can use it in control flow task to perform truncate operation using the variable value.
please adviiiise
your help in this regard is appreciated.
Rosh..
It's fairly simple: you use an Execute SQL Task to retrieve the value and store it in a variable.
Basic steps:
A. Create an Excel Connection Manager, point it at your file
B. Create a variable to store the value
C. Add an Execute SQL Task
Connection type: EXCEL
Specify connection manager
ResultSet: single row
SQLSourceType: Direct input
SQLStatement: select max(fieldname) as fieldname from [sheetname$]
In the result set tab, add a row with the the ResultName set to fieldname, and the earlier created variable in the Variable Name column.
Note that the sheetname qualification (square brackets) is necessary because of the required $. If your field (column) contains a space in the name, you have to also qualify it: [field name]
Related
This is a two-part question. Is there a best practice approach/workaround to dynamically SET the return value from a query into an SSIS Variable using Expressions? I've researched and read the docs and it seems SSIS Variable Expressions store the value of the Expression and not the returned/runtime value.
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
If the above is not possible, has anyone leveraged the Execute SQL Task to set values for multiple variables? The idea here is to avoid using an Execute SQL Task for each dynamic variable initailization.
My requirements are to SET initial values for 10 variables on the main control flow. I am trying to avoid using 10 Execute SQL Tasks to accomplish this.
Without using the Script Task or 3rd Party plugins is the above possible in SSIS (Version 2019)?
Thank you
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
You are correct, you are looking to store values in an SSIS variable. A Variable with the EvaluateAsExpression property set to true means it will by dynamic but the scope of the dynamicism is the SSIS package itself.
In your case, you need to ask an external provider for information so you couldn't build an SSIS expression to satisfy that.
If you can get the 10 values you want to store in SSIS variables in a single query, then yes, you can get by with a single Execute SQL Task.
If the 10 columns are all in a single table, like a control table, then you could have a query like
SELECT
MAX(C.SalesFactDate) AS MaxSalesDate
, MAX(C.EmployeeDimDate) AS MaxEmployeeDate
-- etc
FROM
dbo.Control AS C;
If you need to get dates from the actual tables, that too can be a "single" query
SELECT
(SELECT MAX(C.SalesFactDate) FROM dbo.FactSales AS C) AS MaxSalesDate
, (SELECT MAX(C.EmployeeDimDate) FROM dbo.DimEmployees AS C) AS MaxEmployeeDate
-- etc
Either way, you will have a single row of data with 1 to N columns of data you want to associate with SSIS Variables.
On the Main task Execute SQL Task screen, change the Result Set type to Single row from None.
On the Results tab, you'll associate an SSIS variable per column. If you used OLE DB provider, the column name is the zero based ordinal. ODBC is 1 based ordinal and ADO.NET uses named entities.
Assuming I had created a variety of SSIS variables of type Date/db_date/etc and I used an OLE DB Connection manager, the screen would look like
0|#[User::SalesDate]
1|#[User::EmployeeDate]
etc
i want to read first line of the flat-file containing date value and compare with a user variable defined in 2008 SSIS package without using sript task.
This solution is a bit long for what the simple task you require but since you don't you want to use script task, you can try below:
Create variable to store the date value from your flat file
Create a data flow task to import the flat file into a SQL table.
Add an Execute SQL task to get the first line from the SQL table and map the result
set to the variable created at step 1.
You can now compare the variable created and the one you already have.
e.g #[User::NewVariableCreated] == #[User::ExistingVariable]
This will return a Boolean data type result of True or False depending on the values supplied.
Hope this helps.
One option is to use a Conditional Split. Within this task, a condition can then be added comparing the date column of each row with the variable. A basic example of a condition for this is below, which checks to see if the date column of each row is equivalent to or more recent than the date variable. From here, the rows be directed based off whichever condition they match.
FlatFileDateColumn >= #[User::DateVariable]
Good Day All,
I have a select query where i pull the data from sql ie
select invno , date_received from sales
What i want to do is split the file into multiple files using conditional split.
I don't know how to set the condition i tried as below
But it just creates one file, how do i create the multiple files based on column value if i don't know what the column value would be?
Also i would like to assign the column value ie INVNO to filename as well to prevent overwriting of files
Using the out of of the box componentry, your best bet would be to have something like this
The Execute SQL Task will return a Full Result Set to an SSIS variable of type Object. Your query would generate the distinct set of INVNO. SELECT DISTINCT T.INVNO FROM dbo.Sales AS T;
The Foreach Loop Container is then going to "shred" that recordset into a single instance of our INVNO. This requires you to have a variable, probably of type String, to receive that number.
The Data Flow Task will have as the source query a parameterized query. Assume OLE DB Connection manager, that'd be select INVNO, date_received FROM dbo.Sales AS S WHERE S.INVNO = ?; and then you map in the current value of INVNO (assigned as part of the shredding from the FELC)
Finally, the Flat File Connection Manager will have an Expression on the ConnectionString property that factors in the full path to the output file. It could be something as simple as "C:\ssisdata\" + #[User::Invno] + ".csv"
Oh, and set DelayValidation = True for the Flat File Connection Manager and the Data Flow Task.
I have a report that I am working on that will do the following:
Return results based first on the community selected by the user.
Filter to find alike addresses within the community, based on the number of square feet at each address.
Set the end date (a column within the data table) to a user defined parameter for use in a WHERE at the end of the query.
The relevant information is stored in the following places:
Community: ub_subdivision.descr
Address: ub_serv_loc_addr.location_addr
SqFt: arp_ops.dbo.vw_ub_serv_loc_classifications.SqFt
I have setup the query with 3 parameters:
#Community
#Months
#Address
When the user is running the report, the following should happen (in this order):
The community parameter should populate the values stored in ub_subdivision.descr and allow the user to select the community they want from that list.
The address parameter should populate the values within the selected community from step 1, and allow the user to select the address they want from that list.
Based on the selected address, the query should store the value of the SqFt related to this address and use that in the WHERE statement as follows: WHERE (arp_ops.dbo.vw_ub_serv_loc_classifications.SqFt = #Address)
The months parameter should allow for user input to define how many months of data they want. This parameter is called in the query in the WHERE statement: WHERE (ub_bill_run.def_end_dt > DATEADD(m, -#Months, GETDATE())).
If I save the dataset and create a "table report" in Report Builder 3.0 it does the job of recognizing the various parameters and loading them into the Parameters folder and into the Datasets' parameters.
The problem I have is that I am not able to change the parameter properties to display Available Values and select "get from a query". If I go this route, and try to run the query I get an error that I am using "forward dependencies".
I need the #Address parameter to display the address field as the label, but store the sqft field as the value. This is the way I know how to do this and, unfortunately, it doesn't seem to work.
I would appreciate any insight anyone may have.
Thanks!
John
There is one way to solve this make sure the order should be in the order of
#Community
#Months
#Address
change order to:
#Community
#Address
#Months
just delete existing #month and again add it manually and save it.
i hope it will work for you.
You cannot have parameters based on your main data set.
The forward dependency error is caused because your data set is to be filtered by your parameter, yet it is depending on the same data set to find its' set of values. This is a sort of paradox.
When using queries to define the set of values for your parameters, make sure you create a new data set for each parameter.
Next, make sure the parameters are listed in the order you want them to run. Within the data sets for your parameters, you may use where clauses to make them dependent on one another in the order that they run.
In this example:
Parameter data set for Community:
SELECT DISTINCT ub_subdivision.descr
FROM [YOUR JOINED TABLES]
Parameter data set for addresses:
SELECT DISTINCT ub_serv_loc_addr.location_addr
FROM [YOUR JOINED TABLES]
WHERE ub_subdivision.descr IN (#Community)
Parameter data set for SqFt:
SELECT DISTINCT SqFt
FROM [YOUR JOINED TABLES]
WHERE ub_subdivision.descr IN (#Community)
AND ub_serv_loc_addr.location_addr IN (#Address)
You should also make a month data set for your #month parameter, however it is not dependent on the other parameters so I will leave that to you.
Hope this helps!
I created SSIS package. I have a Data Flow Task in here, where I have OLE DB DataSource, which loads records from some table from database. Table name is assigned programmatically, so different columns may be output of that DataSource. Also I have Conditional Split connected to DataSource output, where I want to split records. I want to set condition in Conditional Split and I want to do something like that:
#[User::ConditionColumnName] >= #[User::SomeValue]
where #[User::SomeValue] is variable with some value to compare, but #[User::ConditionColumnName] is variable with name of some column from DataSource output. This value I will assign programmatically.
How can I do that? Or may be is there some other way to split data with unknown at compile time columns?
This sounds like a row-based conditional split. Perhaps you could add the variable value into your select list (so that you have a column you can compare on) something like
"SELECT '" + (DT_STR,50,1252)#[User::ConditionColumnName] + "' as MyConditionColumnName, .... FROM ... "
That way you have the column MyConditionColumnName per row that you can compare on in your Conditional Split.
(You could put this select into a variable and run the sql from variable, maybe easier to maintain) - either way you need to parse the query as an expression in order to evaluate your variable before it's run.