Query 2 will run based on the Query1 output - ssis

The Execute SQL Task will provide us database list. Based on the list Query2 should run on all the database.
Query1 result :
databasename1
databasename2
databasename3
Query 2:
step 1
select * from databasename1.dbo.tablename
step 2
select * from databasename2.dbo.tablename
step 3
select * from databasename3.dbo.tablename
How can I use the results of a query to power subsequent queries? In my case, change the catalog name,

You can do this by using a Foreach Loop Container.
You'll need the following control flow:
Execute SQL Task -> Foreach Loop Container and another Execute SQL Task inside the container.
You'll need two variables:
DatabaseList (Object)
DatabaseName (String)
List the databases with a SQL Task
Create a user variable with Object type (DatabaseList)
On the General pane, set the ResultSet property to Full result set
On the Result Set pane set the add a new line (Result Name = 0, Variable Name = User::DatabaseList)
This will query the databases and stores the list in an object.
Note There should be one database name in one record in the first Execute SQL Task's query. Each record will be processed by the Foreach Loop.
Add a Foreach Loop Container after the SQL Command Task
On the Collection pane set the Enumerator to Foreach ADO Enumerator
Select User::DatabaseList as the ADO object source variable
On the Variable Mappings pane add a new entry (Variable = User::DatabaseName, Index = 0)
Add an Execute SQL Task inside the Foreach Loop Container and set up as normally.
Set the ResultSet property to None on the General pane.
Copy the Query to the clipoard.
Highlight the SQL Task inside the Container, then press F4 (opens the property tab).
Add a new expression which sets the SqlStatementSource property. As the Expression add an expression which returns the query with the database (based on the User::DatabaseName).
You can use expressions to dynamically set some the properties of an object. (This method also works with a Data Flow Task just the property to set (with the expression) could be different)
Note If you want to use DFT, then the tables should have the same structure (the meta data of the columns returned by the query should be the same in SSIS)

Related

Dynamically store values from a sql query into SSIS variables

This is a two-part question. Is there a best practice approach/workaround to dynamically SET the return value from a query into an SSIS Variable using Expressions? I've researched and read the docs and it seems SSIS Variable Expressions store the value of the Expression and not the returned/runtime value.
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
If the above is not possible, has anyone leveraged the Execute SQL Task to set values for multiple variables? The idea here is to avoid using an Execute SQL Task for each dynamic variable initailization.
My requirements are to SET initial values for 10 variables on the main control flow. I am trying to avoid using 10 Execute SQL Tasks to accomplish this.
Without using the Script Task or 3rd Party plugins is the above possible in SSIS (Version 2019)?
Thank you
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
You are correct, you are looking to store values in an SSIS variable. A Variable with the EvaluateAsExpression property set to true means it will by dynamic but the scope of the dynamicism is the SSIS package itself.
In your case, you need to ask an external provider for information so you couldn't build an SSIS expression to satisfy that.
If you can get the 10 values you want to store in SSIS variables in a single query, then yes, you can get by with a single Execute SQL Task.
If the 10 columns are all in a single table, like a control table, then you could have a query like
SELECT
MAX(C.SalesFactDate) AS MaxSalesDate
, MAX(C.EmployeeDimDate) AS MaxEmployeeDate
-- etc
FROM
dbo.Control AS C;
If you need to get dates from the actual tables, that too can be a "single" query
SELECT
(SELECT MAX(C.SalesFactDate) FROM dbo.FactSales AS C) AS MaxSalesDate
, (SELECT MAX(C.EmployeeDimDate) FROM dbo.DimEmployees AS C) AS MaxEmployeeDate
-- etc
Either way, you will have a single row of data with 1 to N columns of data you want to associate with SSIS Variables.
On the Main task Execute SQL Task screen, change the Result Set type to Single row from None.
On the Results tab, you'll associate an SSIS variable per column. If you used OLE DB provider, the column name is the zero based ordinal. ODBC is 1 based ordinal and ADO.NET uses named entities.
Assuming I had created a variety of SSIS variables of type Date/db_date/etc and I used an OLE DB Connection manager, the screen would look like
0|#[User::SalesDate]
1|#[User::EmployeeDate]
etc

SSIS OLE DB Data Source Input

I have an OLE DB Data Source in my SSIS package where I'm wanting to run this query:
SELECT * FROM vw_DimWorker WHERE AsOfSourceDtm > ?
and that variable is a DateTime. The value comes from a different data source where I just want to run this query:
SELECT last_update from Last_Updates WHERE table_name = 'worker'
I can create the source item and the sql task item but I'm not sure how to hook up the date from the second query shown so that I can then use it as the variable on the first query.
You would use variables in your package. When you run the first query have result set = single row/value. Then on the result set tab for your first query, set the results of the query to populate your variable.
Then in your next SQL task, setup a parameter (separate tab in the SQL task) and set the parameter to your variable you just populated.
Link to Example in MSDN:
https://msdn.microsoft.com/en-us/library/ms140355%28v=sql.120%29.aspx?f=255&MSPPError=-2147217396
But this may be better step by step instructions:
https://www.red-gate.com/simple-talk/sql/ssis/passing-variables-to-and-from-an-ssis-task/

SSIS Condition split based on column value

Good Day All,
I have a select query where i pull the data from sql ie
select invno , date_received from sales
What i want to do is split the file into multiple files using conditional split.
I don't know how to set the condition i tried as below
But it just creates one file, how do i create the multiple files based on column value if i don't know what the column value would be?
Also i would like to assign the column value ie INVNO to filename as well to prevent overwriting of files
Using the out of of the box componentry, your best bet would be to have something like this
The Execute SQL Task will return a Full Result Set to an SSIS variable of type Object. Your query would generate the distinct set of INVNO. SELECT DISTINCT T.INVNO FROM dbo.Sales AS T;
The Foreach Loop Container is then going to "shred" that recordset into a single instance of our INVNO. This requires you to have a variable, probably of type String, to receive that number.
The Data Flow Task will have as the source query a parameterized query. Assume OLE DB Connection manager, that'd be select INVNO, date_received FROM dbo.Sales AS S WHERE S.INVNO = ?; and then you map in the current value of INVNO (assigned as part of the shredding from the FELC)
Finally, the Flat File Connection Manager will have an Expression on the ConnectionString property that factors in the full path to the output file. It could be something as simple as "C:\ssisdata\" + #[User::Invno] + ".csv"
Oh, and set DelayValidation = True for the Flat File Connection Manager and the Data Flow Task.

SSIS Excel -Retrieving minimum date value and storing it into a variable

I have an Excel Source which has got 1000 rows with some 10 columns and one of the column is a Date Field ,We have to retrieve the minimum date value and assign it to a variable in ssis .Could you guys provide me a script or steps to map that value to the variable...So that i can use it in control flow task to perform truncate operation using the variable value.
please adviiiise
your help in this regard is appreciated.
Rosh..
It's fairly simple: you use an Execute SQL Task to retrieve the value and store it in a variable.
Basic steps:
A. Create an Excel Connection Manager, point it at your file
B. Create a variable to store the value
C. Add an Execute SQL Task
Connection type: EXCEL
Specify connection manager
ResultSet: single row
SQLSourceType: Direct input
SQLStatement: select max(fieldname) as fieldname from [sheetname$]
In the result set tab, add a row with the the ResultName set to fieldname, and the earlier created variable in the Variable Name column.
Note that the sheetname qualification (square brackets) is necessary because of the required $. If your field (column) contains a space in the name, you have to also qualify it: [field name]

Using an SSIS variable in a DataFlow task

I have and SQL Execute task that gets me a list of years and save them into a an object variable, let's say "Years".
Then I have a ForEach Loop Counter that recieves thah variable, and inside that Counter I have a DataFlow task, what I want to do is use that "Years" variable in the Dataflow task as I need to use that list of years in my query.
Any suggestions on how to do that?
Oh my query in my DataFlow task is an MDX query.
Thanks
Since you need to build the MDX query dynamically and append the specific year in the counter to it, you need to:
First create a new string variable called MDX_Query.
Then esnure that the "Evaluate as expression" option is set to True
Now create an expression for that variable that is basically your MDX query and append your #Years variable to it. Something alongs the lines of
"MDX query text goes here where SalesYear.["+ #[User::Years_Variable]+"]"
Now inside your Foreach Loop task, if you have created a SQL query task, ensure that your "SQL Source Type" is set to variable and set the "Source Variable" to MDX_Query