I created SSIS package. I have a Data Flow Task in here, where I have OLE DB DataSource, which loads records from some table from database. Table name is assigned programmatically, so different columns may be output of that DataSource. Also I have Conditional Split connected to DataSource output, where I want to split records. I want to set condition in Conditional Split and I want to do something like that:
#[User::ConditionColumnName] >= #[User::SomeValue]
where #[User::SomeValue] is variable with some value to compare, but #[User::ConditionColumnName] is variable with name of some column from DataSource output. This value I will assign programmatically.
How can I do that? Or may be is there some other way to split data with unknown at compile time columns?
This sounds like a row-based conditional split. Perhaps you could add the variable value into your select list (so that you have a column you can compare on) something like
"SELECT '" + (DT_STR,50,1252)#[User::ConditionColumnName] + "' as MyConditionColumnName, .... FROM ... "
That way you have the column MyConditionColumnName per row that you can compare on in your Conditional Split.
(You could put this select into a variable and run the sql from variable, maybe easier to maintain) - either way you need to parse the query as an expression in order to evaluate your variable before it's run.
Related
This is a two-part question. Is there a best practice approach/workaround to dynamically SET the return value from a query into an SSIS Variable using Expressions? I've researched and read the docs and it seems SSIS Variable Expressions store the value of the Expression and not the returned/runtime value.
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
If the above is not possible, has anyone leveraged the Execute SQL Task to set values for multiple variables? The idea here is to avoid using an Execute SQL Task for each dynamic variable initailization.
My requirements are to SET initial values for 10 variables on the main control flow. I am trying to avoid using 10 Execute SQL Tasks to accomplish this.
Without using the Script Task or 3rd Party plugins is the above possible in SSIS (Version 2019)?
Thank you
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
You are correct, you are looking to store values in an SSIS variable. A Variable with the EvaluateAsExpression property set to true means it will by dynamic but the scope of the dynamicism is the SSIS package itself.
In your case, you need to ask an external provider for information so you couldn't build an SSIS expression to satisfy that.
If you can get the 10 values you want to store in SSIS variables in a single query, then yes, you can get by with a single Execute SQL Task.
If the 10 columns are all in a single table, like a control table, then you could have a query like
SELECT
MAX(C.SalesFactDate) AS MaxSalesDate
, MAX(C.EmployeeDimDate) AS MaxEmployeeDate
-- etc
FROM
dbo.Control AS C;
If you need to get dates from the actual tables, that too can be a "single" query
SELECT
(SELECT MAX(C.SalesFactDate) FROM dbo.FactSales AS C) AS MaxSalesDate
, (SELECT MAX(C.EmployeeDimDate) FROM dbo.DimEmployees AS C) AS MaxEmployeeDate
-- etc
Either way, you will have a single row of data with 1 to N columns of data you want to associate with SSIS Variables.
On the Main task Execute SQL Task screen, change the Result Set type to Single row from None.
On the Results tab, you'll associate an SSIS variable per column. If you used OLE DB provider, the column name is the zero based ordinal. ODBC is 1 based ordinal and ADO.NET uses named entities.
Assuming I had created a variety of SSIS variables of type Date/db_date/etc and I used an OLE DB Connection manager, the screen would look like
0|#[User::SalesDate]
1|#[User::EmployeeDate]
etc
I am in report builder and I have my primary dataset that is from a SQL database, I also then created a second dataset (enter data). I need to compare 2 fields from each dataset to retrieve the correct value from the 2nd dataset and populate a column on my report. I have tried the IIF statements and Lookup statements but I keep getting the error "report item expressions can only refer to fields within the current dataset".
I have a attached a screenshot of what I am trying to do....
The IIF statement I tried to use.. If Acctnum and prodid = each other return IncodeNumber
=IIF((Fields!AcctNum.Value=Fields!AcctNum.Value, "IncodeAccount") AND
(Fields!ProdId.Value =Fields!ProdId.Value, "IncodeAccount")),(Fields!IncodeNumber.Value, "IncodeAccount"),"True")
See code in my problem.
You need to use LOOKUP(). The problem with LOOKUP() is that is can only compare a single value from each dataset. However, we can easily get around this issue by concatenating the two values you need to compare.
Note: This assumes the expression will be in a tablix that is bound to your first dataset and that IncodeAccount is your second dataset - the values you want to lookup. If this is not the case just adjust the expression accordingly
So for you, you probably need to do something like this..
=LOOKUP(
Fields!AcctNum.Value & "||" & Fields!ProdId.Value,
Fields!AcctNum.Value & "||" & Fields!ProdId.Value,
Fields!IncodeNumber.Value,
"IncodeAccount"
)
I've used two pipe symbols to join the values to avoid incorrect matches being found. e.g. Account 123 and product ID 4567 would incorrectly match to Account 1234 and product ID 567 as they would both be 1234567 when joined. By using the || the match would be 123||4567 and 1234||567 respectively.
You may need to convert the values to string using CStr()
Alternative approach
If you are going to do this 'join' multiple times in the same dataset then you could add a calculated column to the dataset that concatenates the two columns. Then you can use this single field in the lookup which will make things a little simpler.
Or, you could do this concatenation in a database view which would make things even easier.
I have a table that I am building a report for. There are about 10 fields, but 4 of those fields may contain value(s) that users would want to filter the report by.
I was thinking I could create a parameter with a list of label/value pairs and the value portion of the parameter item would be an actual where clause for the underlying dataset like:
#filter
label/value
exceptions/where error_field like '%exception%'
counts/where count_field > 100
2016/where year_field = 2016
I tried dataset:
select error_field, count_field, year_field from mytable
#filter
I also tried(leaving where out of parameter value):
select error_field, count_field, year_field from mytable
where #filter
Both dataset queries failed to save. I am thinking I could include all the varying where clauses inside the dataset query statement, but it may require different parameters but how can they be empty unless I used 1=1 as default value. I only wanted to use a single parameter tho.
Any other ideas?
Thank you.
Your dataset query needs to be valid SQL. You can use an expression for the dataset, and work with the SSRS expression language to generate the SQL you need.
Something like:
="SELECT * FROM TABLE " + IIF(Value = True," WHERE 'A' = 'B'","")
However, I think they are a pain to work with. They are harder to understand, and take longer to maintain, and much easier to contain a mistake.
I find it easer and safer just to pass the parameters to a SQL Stored Procedure on the server, especially if you are using a TEXT BOX entry field.
Good Day All,
I have a select query where i pull the data from sql ie
select invno , date_received from sales
What i want to do is split the file into multiple files using conditional split.
I don't know how to set the condition i tried as below
But it just creates one file, how do i create the multiple files based on column value if i don't know what the column value would be?
Also i would like to assign the column value ie INVNO to filename as well to prevent overwriting of files
Using the out of of the box componentry, your best bet would be to have something like this
The Execute SQL Task will return a Full Result Set to an SSIS variable of type Object. Your query would generate the distinct set of INVNO. SELECT DISTINCT T.INVNO FROM dbo.Sales AS T;
The Foreach Loop Container is then going to "shred" that recordset into a single instance of our INVNO. This requires you to have a variable, probably of type String, to receive that number.
The Data Flow Task will have as the source query a parameterized query. Assume OLE DB Connection manager, that'd be select INVNO, date_received FROM dbo.Sales AS S WHERE S.INVNO = ?; and then you map in the current value of INVNO (assigned as part of the shredding from the FELC)
Finally, the Flat File Connection Manager will have an Expression on the ConnectionString property that factors in the full path to the output file. It could be something as simple as "C:\ssisdata\" + #[User::Invno] + ".csv"
Oh, and set DelayValidation = True for the Flat File Connection Manager and the Data Flow Task.
I want to build a SSRS report that has column as week numbers - 8 weeks for 8 columns starting with current. This report is run every week and current week number is set then. So both column names and their values should change .Is it possible to build something like this in SSRS?
I tried doing this with a dynamic SQL based stored proc in dataset. However for every run I don't even see the columns values updating dynamically
Here's an example :
Also I am trying to avoid these week numbers as row values and then using matrices
My stored proc looks something like this
declare #n tinyint = datepart(wk, getdate())
declare #n1 tinyint = (#n+1), #n2 tinyint =(#n+2), #n3 tinyint =(#n+3), #n4 tinyint =(#n+4), #n5 tinyint =(#n+5), #n6 tinyint =(#n+6)
exec ('Select b.sku, b.['+#n+'], b.['+#n1+'], b.['+#n2+'], b.['+#n3+'], b.['+#n4+'], b.['+#n5+']...
Will appreciate any help in this direction.. many thanks!
When working with SSRS it's generally best to avoid dynamic SQL and pivoting the data in the SQL. Use the SQL to get the raw data you need and then let SSRS do the pivoting and aggregation. This way you take advantage of what they each do best. I know you said you want to avoid matrices, but it is the best way to make the report dynamic.
So you should either return all the data in one dataset and use filters on your matrices OR write two queries and have each one populate a matrix. BTW a matrix is just a table with a column group added, so don't be intimidated by them.
There are 2 ways to do this with a standard tablix.
Calculate the column headers as expressions using concatenation of Wk and some date math to find the correct week number and return the same sort of thing from your query (e.g. columns are current_week, week_minus_1, week_minus_2...)
Return the column headers as additional columns in your query that are the same value for every row (e.g. ColHeader0, ColHeader1...). Your data columns would still be relative weeks (e.g. ValueWeek0, ValueWeek1...). In your report the column header would have an expression like =First(Fields!ColHeader0.Value). This is a more flexible approach since it lets you pick 8 historical weeks instead of only the last 8 weeks if you add a parameter.
EDIT - Clarifications
The reason that you get the blank column Wk48 is approximately that you have created your report looking for that column that won't be there the next time. SSRS looks for exact columns. You should you use relative column names for either of the options I have specified:
exec ('Select b.sku, b.['+#n+'] as Wk0, b.['+#n1+'] as Wk1, b.['+#n2+'] as Wk2, b.['+#n3+'] as Wk3, b.['+#n4+'] as Wk4, b.['+#n5+'] as Wk5...
This will allow you to populate the aliased Wk0 column with the appropriate current week data and still make sure that it can be consistently referenced as the base week by SSRS.
To change the column headers you can:
Independently calculate the week numbers in SSRS in the column header expressions: ="Wk" + CStr(<correct week calculation>).
Return the column headers in the result set and access them in the column header expression:
exec ('Select b.sku, b.['+#n+'] as Wk0, b.['+#n1+'] as Wk1, b.['+#n2+'] as Wk2, b.['+#n3+'] as Wk3, b.['+#n4+'] as Wk4, b.['+#n5+'] as Wk5..., ''Wk'''+#n+' as ColHeader0, ''Wk'''+#n1+' as ColHeader1...
and reference the returned column headers from the SSRS column header expression as =First(Fields!ColHeader0.Value).
Here's a solution that worked for me:
Create parameters (say CurrWk, CurrWk1) ,set as hidden and store 'Default value' and 'Available value' equals to current week number (datepart(wk, now()) and any subsequent week by doing a +1, +2, +3.. etc.
Write a query expression . Click onto fx beside dataset query space and write the select query for your program embedding parameter values in the expression window. For eg ="Select SKU, [" & Parameter!CurrWk.Value & "] as Wk1,
[" & Parameter!CurrWk.Value & "] as Wk1 from Sales_Table"
Before passing this query as a 'command text expression' please ensure this query is working in sql ssms.
Save the expression. Now find 'Fields' tab on the left hand side panel.You need to map the fields manually from the query here. If this is not done, there is a very high chance you seean empty field list and wont be able to access them at all. This may be because ssrs do not store query metadata directly from expressions.
You can avoid part of the issue by having atleast the static fields , for example here SKU listed in the 'Fields' list by first running a sql query with static field(select SKU from Sales_Table ). You can then go back to update dataset- change query to expression and embed the parameterized field names.
Map field names. In this example I chose 'Query Type' fields and set Field names as SKU, CurrentWeek, NextWeek and mapped to source SKU, Wk and Wk1 respectively.
Click on 'Refresh Fields' at the bottom. Now you have a dataset with the complete field list. Use these in charts, tables . Run it every week and note the numbers changing as expected.
In case you are using this dataset in a table, make sure you set headers with Labels of Parameters (for eg here I did =Parameters!CurrWk.Label for col with current week data)
That's it!