I have an SSIS 2008 package with data flow task using Task Factory Salesforce.com source, and in the WHERE clause is the below statement, which works fine.
WHERE SystemModstamp > <#User::dt_last_success>
I'd like to pull off a conditional 'IF #load_all_data=True THEN do not include the above WHERE clause, and if False then include it. I've been told that this is not possible in Salesforce Object Query Language (SOQL) to include that logic in the WHERE clause of the above query* , which means that I'll need two separate data sources: one with the current WHERE.., and one without.
I'd prefer not to copy-paste each data flow task, and the only different being the WHERE clause, to pull this off.
Question: What's the easiest way to pull off two sources based on logic with a parameter in a data flow task, using the same destination? Preferably without a JOIN that requires sorting.
Thanks in advance.
Jim
https://salesforce.stackexchange.com/questions/57277/ssis-package-connecting-to-salesforce-with-or-parameter-y
I'm not familiar with the TaskFactory's Salesforce.com source, but most SSIS things allow for the concept of Expressions to be applied to them. Looking at the documentation for SFDC and knowing the PW folks, surely they allow for this.
Create a Variable, LoadAllData of type boolean and set it to False
Create a second variable, of type string, called Query. You will then need to set the EvaluateAsExpression property to True and then use an expression like the following (note the double quotes are part of the Expression.
"SELECT MyCols FROM MyTable "
+ (#[User::LoadAllData]) ? "" : "WHERE SystemModstamp > <#User::dt_last_success>"
That will build the query of SELECT MyCols FROM MyTable for the full load, SELECT MyCols FROM MyTable WHERE SystemModstamp > <#User::dt_last_success> for the incremental load.
Once you verify the strings are working, as expected with the variable toggle, you then go into your data flow, right click on the SFDC source and under Expressions, there ought to be something that corresponds to the SalesForceQuery and you then assign as its value #[User::Query]
Tying this all together, you'd then use the /SET property when you run the package to flip the value of LoadAllData to True whenever you need the full load (which I assume is the exception and not the standard case).
dtexec /file ./SFDC.dtsx /Set \Package.Variables[User::LoadAllData].Properties[Value];1
^^^ That's approximate. I haven't tried to set a boolean from the command line so it might expect True, true or possibly TRUE. Quick experimentation on your part should reveal the specific syntax.
Related
I am currently using SQL Server Report Builder 2012 and is connected to my MySQL Database via an ODBC Connector and as far as base report goes, all is well.
However, i can't seem to make the Query/Dataset Parameters to work the way its supposed to be. I have multiple parameters to my query as you can see below (obviously table and column names are removed):
Now the problem is, if i leave the parameters as is (#OfMonth, #OfDay, #OfYear) - SSRS does not seem to bind the actual values passed from the Report Builder's Parameter Object which i am confident to day that i have associated properly. Not even on the preview/query designer.
However, if i change all #XXXX parameters to simple ? placeholders, it magically works. This poses as issue specially with queries that have multiple parameters.
This is the Report Builder's screenshot of my Work in Progress:
i have no issues defining the 3 Parameter object under the Parameter Node. However, if i try to bind them under Dataset Properties with specific #XXXXX placeholders, it doesn't work, and the report fails to generate data. But if i replace all #XXXXX with ? (all of them are just ?, therefore duplicates), the parameter gets passed and the report loads.
For ODBC connections, you do need to use a ? instead of named variables.
dba.stackexchange | Pass Parameter - SSRS to MySQL
The Parameter Name field on the Dataset Properties should auto-fill with Parameter1, Parameter2,... to match your query but doesn't always seem to work. You can try adding them manually. Since it worked without the name for you, I assume the name doesn't actually matter.
When I would have a parameter used multiple times, I would declare a new one in the query and reuse the new one as #Bacon mentioned:
DECLARE #OfMonth INT
SET #OfMonth = ?
This way you only have to match them once at the beginning of your query.
Use ? as variable in your script, then remember specific order of '?' then using specific order/arrangement of '?' parameters, setup them in the parameter tab after you add the MySQL script.
Ex. Script.
Select * from table1
where column1 = ?
and column2 = ?
When you paste this on the dataset, each '?' will be mapped in the parameter tab.
? Parameter1
? Parameter2
Change this to your own parameters then you're good to go.
Good Day All,
I have a select query where i pull the data from sql ie
select invno , date_received from sales
What i want to do is split the file into multiple files using conditional split.
I don't know how to set the condition i tried as below
But it just creates one file, how do i create the multiple files based on column value if i don't know what the column value would be?
Also i would like to assign the column value ie INVNO to filename as well to prevent overwriting of files
Using the out of of the box componentry, your best bet would be to have something like this
The Execute SQL Task will return a Full Result Set to an SSIS variable of type Object. Your query would generate the distinct set of INVNO. SELECT DISTINCT T.INVNO FROM dbo.Sales AS T;
The Foreach Loop Container is then going to "shred" that recordset into a single instance of our INVNO. This requires you to have a variable, probably of type String, to receive that number.
The Data Flow Task will have as the source query a parameterized query. Assume OLE DB Connection manager, that'd be select INVNO, date_received FROM dbo.Sales AS S WHERE S.INVNO = ?; and then you map in the current value of INVNO (assigned as part of the shredding from the FELC)
Finally, the Flat File Connection Manager will have an Expression on the ConnectionString property that factors in the full path to the output file. It could be something as simple as "C:\ssisdata\" + #[User::Invno] + ".csv"
Oh, and set DelayValidation = True for the Flat File Connection Manager and the Data Flow Task.
I have a report in SSRS. My parameter allows multiple values. My query has in the WHERE statement:
WHERE AllDiag IN (#Diag)
My user should be allowed to enter something like Z34.83,R30.0,0000.
These are 3 different codes to search for, so technically it is looking for:
WHERE AllDiag IN ('Z34.83','R30.0','0000')
I've tried all kinds of things like making the parameter properties in the query properties an expression using =join(Parameters!Diag.Value,"','"), and even entering the list of codes with the quotes already, but nothing seems to allow this to work.
I even tried some split function to see if it searched for each separately but I'm not sure I even use it right since there seems there might be some function that should run before.
I'm out of ideas. Any help is greatly appreciated!
I'm assuming your dataset uses the WHERE clause as you stated
WHERE AllDiag IN (#Diag)
I'm also assuming that you cannot easily produce a list of available parameter values to choose from.
So to create a parameter that allows the user to manually enter a list of values simply set 'Allow multiple values' on #Diag parameter. The user then simply types each value and presses enter after each one.
Note there is no need for comma's just type them one by one pressing enter after each.
When SSRS sees a multi-value parameter being passed to a SQL statement using an IN clause, it converts this to dynamic SQL automatically including adding the comma's. If you trace the report using the SQL Profiler, you can see the SQL that is generated.
I thought I would share this with you all in case you may have had issues (like I did) with passing multiple values in a single parameter from your web page to a SSRS report.
NOTE: This is different from passing multiple parameters, each with its own value into a report. The later, there are plenty of examples on the web.
This is very usefull when you need to basically pass into your report's SQL command a list of values for your report's SQL command to use using a "special" function, and where you do not know the number of times the values may be required, as the user can choose anything from one value to 'n' values (but we will hit a limit, as I'll explain later). It's also useful for generating Excel row-by-row extracts from your website - say for Pivot table handling or charting later on.
Unfortunately using IN() on its own tricks a lot of people and they cannot figure out why it does not work. That's because if you define your report in SSRS to expect a parameter straight into the IN() function, the system literally places the value as a parameter in the function and tries to compare what is essentialy a parameter "data type" with your column's data type and you will get errors.
If your report has SQL similar to this ...
SELECT t.Col1, t.Col2, etc
FROM myTable t
WHERE t.myColumn IN (#myListOfValues)
where #myListOfValues is something like "'value1','value2','value3',..." it "may" work but I found passing such a string from ASP.net into SSRS did not work and there are technical issues with string handling from the ASP.net side plus a limit depending your system and browser.
To get around possible issues, create a function in your SQL Server database to receive a string of values delimited by a comma and allow the function to turn your list into a table. That way the table can easily be linked using SQL and passed as a sort of "parameter feeder" into your report's SQL or dataset.
So without babbling on too much lets start with code and an example:
Firstly lets create a special utility function that converts a list of values into a table, and by the way this function can be used within your projects to do exactly that - split strings delimited by something into a table for anything else.
Open SQL Server and create a new function using your normal right-click NEW function command. Something like this ...
CREATE FUNCTION [dbo].[fnMakeTableFromList](#List VARCHAR(MAX),#Delimiter VARCHAR(255))
RETURNS TABLE
AS
RETURN (SELECT Item = CONVERT(VARCHAR, Item)
FROM (SELECT Item = x.i.value('(./text())[1]', 'varchar(max)') FROM (SELECT [XML] = CONVERT(XML, '<i>' + REPLACE(#List, #Delimiter,'</i><i>')+ '</i>').query('.')) AS a
CROSS APPLY [XML].nodes('i') AS x(i)) AS y
WHERE Item IS NOT NULL);
NOTE: the delimiter does not have to be a single character! Again useful for delimiting using keywords, etc.
Note the XML logic and conversion in the function? That is because ASP.Net is going to literally pass some HTML into SQL Server and we're going to use it to strip off the data we need into a table.
Run the function with some values to test:
SELECT * FROM dbo.fnMakeTableFromList ('a,b,c,d', ',');
You should see 4 rows of data returned ...
a
b
c
d
That is the results in a table.
Now use this function in your SQL Reporting Services report:
Here is my report as an example:
SELECT DISTINCT s.StudentID
FROM tblStudents s
LEFT OUTER JOIN dbo.fnMakeTableFromList(#StudentList,',') AS list ON list.Item = s.StudentID
WHERE (#StudentList IS NULL
OR #StudentList='')
AND (l.Item IS NULL
OR l.Item = s.StudentID)
Note my example also caters for reporting every student ID if there was no value passed at all. So report every student found in tblStudents or report those based on the list of student IDs given, delimited by a comma. When you run this directly in SSRS, you'll be asked for a parameter #StudentList. In there type what ever values you need separated by a comma, and you should only get those student IDs. If not, make sure the report works "stand alone" first before going over to the ASP side.
Once you are happy your report works, and the function in SQL Server works, then we are ready to code the ASP.net side of things:
In your ASPX code behind page (C#) we need to control what the list is and how to pass it over to SSRS. Because we are dealing with a LIST<> here, I am only going to illustrate the way to do using a LIST<> to mimic an array. As you know C# does not have array terminology like you have with VB.
So in your ASP.net page paste this code in your PageLoad event ...
protected void Page_Load(object sender, EventArgs e)
{
//get parameters from the URL list
string strStudentList = Request.QueryString["StudentList"];
//create a List of parameters and pass it over to the report
List<ReportParameter> paramList = new List<ReportParameter>();
paramList.Add(new ReportParameter("StudentList", strStudentList));
//run the report
ReportViewer1.ServerReport.SetParameters(paramList);
}
Of course some objects in here have to be defined in your ASPx page.
For example I use a master page and as you can see, I did all of this to create a mailing list for printing on special sticky label paper.
<%# Page Language="C#" MasterPageFile="~/rptsStudentAdministration/StudentAdminReports.master" AutoEventWireup="true" CodeFile="rptStudentLabels.aspx.cs" Inherits="rptsStudentAdministration_rptStudentLabels" Title="Student Mailing Labels" %>
<asp:Content ID="Content1" ContentPlaceHolderID="ContentPlaceHolder1" Runat="Server">
<rsweb:reportviewer id="ReportViewer1" runat="server" font-names="Verdana" font-size="8pt"
height="800px" processingmode="Remote" width="900px">
<ServerReport ReportServerUrl="<%$ AppSettings:ReportServerURL %>" ReportPath="/rptsStudentAdministration/rptStudentLabels"></ServerReport>
</rsweb:reportviewer>
</asp:Content>
Make sure you are using these as well in your .cs file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Configuration;
using System.Collections;
using System.Collections.Generic;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;
using Microsoft.Reporting.WebForms;
And that's it folks!
CONCLUSION:
If you need to generate a report in SQL Server Reporting services that relies on users selecting none, one or many values to control the logic in the report, then think of passing them all as a single parameter and using a function to turn your values into a table for ease of SQL management. Once you have the SQL working, you should be able to generate the report easily in design mode and using the above ASPx logic, be able to pass all the values delimited by a comma into into your report. An added bonus is to HIDE the parameter in SSRS and that way the user does not have to see what values they chose, and you control the entire report being generated programmatically.
There are a lot of answers here which don't really point to the solution.
Problem: the #Parameter value is passed to SQL as a comma delimited NVARCHAR value, NOT a list of data values you can JOIN to or use WHERE clauses with. When passed to SQL Server via a procedure call the data type of your parameters is also lost.
Solution in SQL Server 2016+ is to use the build in "split_string" function which returns a list of values from a single delimited field.
DECLARE #Parameter nvarchar(max);
Select *
FROM [dbo].[MyTable] a
JOIN (Select [value] FROM string_split(#Parameter)) b
ON a.ID=b.Value
It may be necessary to CAST your value field depending on the data type you are expecting SSRS to pass through. For example DATE values may look like this:
DECLARE #Parameter nvarchar(max);
Select *
FROM [dbo].[MyTable] a
JOIN (Select CAST([value] as DATE) as [value] FROM string_split(#Parameter)) b
ON a.SomeDateValue=b.Value
In Sql Server 2014 or lower, you can use custom functions to separate delimited list into table rows. Many examples exists on MSDN and StackOverflow, here's very detailed blog post detailing the pros and cons of many methods.
Either method would work with Command Text and Stored Procedure data sets.
I second the answer from Canadean_AS that you should setup a multi-select parameter.
However, if for some reason you have a hard requirement to accept a single comma delimited string into #Diag, you can try the following in your query:
WHERE CHARINDEX(','+AllDiag+',' , ','+#Diag+',') > 0
Be aware that you may encounter performance issues if your where clause is filtering a large dataset with this function.
A more efficient option is to parse #Diag into a table of its own and join that table to the dataset in the FROM clause.
I need to store a large query into a string variable.
When I paste the sql query into the Value property of the variable, I see only one line.
I am using VS 2008 , BIDS to design the package.
I think this is a bug as stated here,
what I am actualy intersted is a workaround to bypass this.
We use a parameters table in a SQL database, then load long strings into variables using a simple SQL query. This has the additional advantage that you can refactor the query code without opening SSIS.
It also allows multiple packages to easily share common query code.
Edit: Of course, if you only want to do this once...
Manually edit the XML. Put a placeholder value in the variable, then open the .dtsx in a text editor, locate the placeholder value, and replace it with the value you need.
Please back up your package prior to doing this.
Another work around - you can set the EvaluateAsExpression to True and put the SQL query in the Expression. The Expression will have a ... button which you can click to edit a multi-line value.
Item in the recordset rstImportData("Flat Size") is = Null
With that, given the following statement:
IIF(IsNull(rstImportData("Flat Size")), Null, cstr(rstImportData("Flat Size")))
Result: Throws error 94: Invalid use of Null
If I change the statement by removing the type conversion upon a false comparison:
IIF(IsNull(rstImportData("Flat Size")), Null, 0)
Result: Null
It returns Null as it should have the first time. It appears that I cannot do a type conversion in an IIF if the value passed in should ever be null even if it passes an IIF test, it still attempts to evaluate it at both the true and false answer. The only reason I'm using IIF like this is because I have a 25 line comparison to compare data from an Import against a matching record in a database to see if I need to append the prior to history.
Any thoughts? The way data is imported there will be null dates and where the spreadsheet import is in a string format I must convert either side to the other to compare the values properly but if either side is null this exception occurs :(
EDIT
Example of why I was using IIF (and considering using a universal function)
If master("one") <> import("one") Or _
master("two") <> import("two") Or _
master("date") <> import("date") Or _ //import("date") comes from a spreadsheet, it comes in as string, CAN be a null value
master("qty") <> import("qty") Or _ //import("qty") comes from spreadsheet, comes in as a string can CAN be null
master("etc") <> import("etc") Then
....stuff....
End If
This code expands for roughly 20 columns to compare in the database. I would prefer to check as part of the statement. I can think of a bunch of solutions but they involve adding much more code. If that is the case power to it, however I'm not one to give in so easily.
Options I see are
Creating temp vars to do the work prior to comparing and using these new vars instead of the recordset
Creating an object to pass the record into to preformat and work with, though extra work would provide this functionality to each import type since there are different files with similar fields
I'm here for ideas, and I'm open to any interesting pieces that can be thrown my way as I get to decide how to do it I'm looking for the most reusable approach.
The simple expedient of changing the value to a string helps tremendously. The trick is that trimming a string which is NULL will get a null string. Which can then be operated on as if it wasn't a database null.
I frequently use the form:
CInt("0" & Trim(SomeVariant & " "))
To get a valid number without having to go through a bunch of hijinks. The null is a nonentity for this problem.
The behavior you described is the standard way IIf operates under VBA. This is part of what Access 2003 Help says about it:
"IIf always evaluates both truepart and falsepart, even though it returns only one of them. Because of this, you should watch for undesirable side effects. For example, if evaluating falsepart results in a division by zero error, an error occurs even if expr is True."
However, if you use an IIf statement in a query, evaluation short circuits after truepart when expr is True --- falsepart is not evaluated in that case. Unfortunately this information is not useful for you ... unless you can incorporate a query into your comparison.
I don't know of any other way to avoid your error with IIf. I would try appending the Excel data into a table whose structure matches that of the table you will compare against, thereby eliminating the need to do a string conversion at the same time you do the comparison.