I have a ssis package, which runs on date parameter. If we dont specify the date it always load data for yesterday's date. And if we give any specific date like '2015-05-10', It should load for that date. How can we achieve this dynamically (using package configuration)? Once we load for any specific date, package should be set for yesterday's date dynamivally. Please guide me to achieve this as I am new to SSIS.
Thanks in advance
Add a parameter to Package ParamDate. This parameter has to be manually provided a value(e.g. 03-21-2015). Leave it blank(NULL value) if yesterday's date has to be considered.
Now define a variable inside your package VarDate
Have an expression for VarDate like this -
ISNULL(#[$Project::ParamDate])?DATEADD("day", -1, getdate()):#[$Project::ParamDate]
I am assuming you are loading data in a DFT.
So in the source query for that, you just need to add an extra condition in the where clause
SELECT ...... FROM SomeTable WHERE LoadDate = ?
In the "Parameters..." section of the source, assign the variable VarDate to the 0. For further details on how to map parameters see here.
Hope it helps.
EDIT
As this was tagged under SSIS 2012,my solution involved project parameter, which is a feature under project deployment model. Package configurations are on the hand under package deployment model.
If you want to use SQL Server package configuration table, then you can follow the steps below:
Lets say the table's name is SSISConfiguration
Have an Execute SQL task where you read this date from this table.
SELECT ISNULL(TstampUpdateDate, DATEADD(dd, -1, GETDATE()) FROM SSISConfiguration
This script will fetch the value of TstampUpdateDate from the SSISConfiguration table and if it is NULL(set below), it will return yesterday's date.
Map the result set to the variable #VarDate.
Have one more Execute SQL task after this step which will update this field to NULL.
Rest of the steps will be same as above(starting from the DFT bit).
Next time the package runs, if the date field is not updated, it will query as per yesterday's date.
Related
This is a two-part question. Is there a best practice approach/workaround to dynamically SET the return value from a query into an SSIS Variable using Expressions? I've researched and read the docs and it seems SSIS Variable Expressions store the value of the Expression and not the returned/runtime value.
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
If the above is not possible, has anyone leveraged the Execute SQL Task to set values for multiple variables? The idea here is to avoid using an Execute SQL Task for each dynamic variable initailization.
My requirements are to SET initial values for 10 variables on the main control flow. I am trying to avoid using 10 Execute SQL Tasks to accomplish this.
Without using the Script Task or 3rd Party plugins is the above possible in SSIS (Version 2019)?
Thank you
For instance, I'd like a the retuned value of MAX date stored in the Variable called [User::MaxDate] not the query string.
Variable: [User::MaxDate]
Expression: (SELECT MAX(dateCol) AS dt FROM tblDate)
You are correct, you are looking to store values in an SSIS variable. A Variable with the EvaluateAsExpression property set to true means it will by dynamic but the scope of the dynamicism is the SSIS package itself.
In your case, you need to ask an external provider for information so you couldn't build an SSIS expression to satisfy that.
If you can get the 10 values you want to store in SSIS variables in a single query, then yes, you can get by with a single Execute SQL Task.
If the 10 columns are all in a single table, like a control table, then you could have a query like
SELECT
MAX(C.SalesFactDate) AS MaxSalesDate
, MAX(C.EmployeeDimDate) AS MaxEmployeeDate
-- etc
FROM
dbo.Control AS C;
If you need to get dates from the actual tables, that too can be a "single" query
SELECT
(SELECT MAX(C.SalesFactDate) FROM dbo.FactSales AS C) AS MaxSalesDate
, (SELECT MAX(C.EmployeeDimDate) FROM dbo.DimEmployees AS C) AS MaxEmployeeDate
-- etc
Either way, you will have a single row of data with 1 to N columns of data you want to associate with SSIS Variables.
On the Main task Execute SQL Task screen, change the Result Set type to Single row from None.
On the Results tab, you'll associate an SSIS variable per column. If you used OLE DB provider, the column name is the zero based ordinal. ODBC is 1 based ordinal and ADO.NET uses named entities.
Assuming I had created a variety of SSIS variables of type Date/db_date/etc and I used an OLE DB Connection manager, the screen would look like
0|#[User::SalesDate]
1|#[User::EmployeeDate]
etc
I have a problem in a small SSIS package that I'm trying to do for storing a query result into an excel file
I want the file to have a dynamic name of Missing_Timecards_#DATETIME#.xlsx
for example: "Missing_Timecards_20220808_131321.xlsx"
for this I have created a template file that has the columns and sheet name I want.
and I have set a system task to copy this template file into a new one with the dynamic name I want to have:
for the variables I have set a combination of a few fields to get my dynamic filename with the date:
the expression for getting the date is
REPLACE(REPLACE( REPLACE(SUBSTRING((DT_WSTR,50)GETDATE(),1,19),"-",""),":","")," ","_")
so far so good, no errors here, when the process starts the variable gets calculated, the filetask creates a copy with the freshly calculated field and goes to the dataflow that retrieves the data and saves it into the excel file path set with the variable that was calculated originally for the filename+datetime
However here is where the issue appears, it seems that the variable is calculated again, so a new file gets created with a "fresh" datetime part of the name, and as the sheet name doesn't match it gives an error.
I think the issue is that is calculating the variable again, how do I stop this from happening? (I have set delay validation = true in for the excel connection and the dataFlow)
As you've identified, GETDATE() is calculated each time it is evaluated. Instead, I favor using a System scoped variable like #[System::StartTime] as it is the time the package started execution but remains constant for the duration of the package.
Literally, swap reference to getdate() with #[System::StartTime] and you're set.
The other option is to
Copy the existing expression to your clipboard
Clear the expression from the Variable
Add an Expression Task to the Control Flow and re-use the expression in the clipboard to assign the value to your #[User::DateTime] variable
Personally, I favor the former approach as a consultant because I still run into SQL Server 2008/R2 packages and the Expression Task was not available for the product.
i want to read first line of the flat-file containing date value and compare with a user variable defined in 2008 SSIS package without using sript task.
This solution is a bit long for what the simple task you require but since you don't you want to use script task, you can try below:
Create variable to store the date value from your flat file
Create a data flow task to import the flat file into a SQL table.
Add an Execute SQL task to get the first line from the SQL table and map the result
set to the variable created at step 1.
You can now compare the variable created and the one you already have.
e.g #[User::NewVariableCreated] == #[User::ExistingVariable]
This will return a Boolean data type result of True or False depending on the values supplied.
Hope this helps.
One option is to use a Conditional Split. Within this task, a condition can then be added comparing the date column of each row with the variable. A basic example of a condition for this is below, which checks to see if the date column of each row is equivalent to or more recent than the date variable. From here, the rows be directed based off whichever condition they match.
FlatFileDateColumn >= #[User::DateVariable]
I've built an SSIS package that runs on schedule weekly on Mondays. It looks for a .csv file that has a date at the end of the file name, which is 2 Saturdays ago from the date it runs (on Mondays). I need the variable to default to two Saturdays ago. Example: SSIS job runs on 7/9/2018, it needs to set it to 6/30/2018. SSIS needs to be runnable in case of failure, so if it has to be run again on, for example, 7/11/2018, it needs to default to 6/30/2018. This is a weekly job scheduled to run on Mondays.
I found a question posted similar to my need and the link is below for additional reference.
SSIS expression previous date without DateAdd()
I just figured out how to format the date extension as 'mmddyyyy' inside Expression Builder for the variable. Below is my code.
RIGHT("0" + REPLACE((DT_WSTR, 30) (DT_DATE) DATEADD("Day", 7*(DATEDIFF("Day", (DT_DATE)0, GETDATE())/7-1), (DT_DATE)0),"/",""),8)
I created an SSIS package composed of OLE DB Source connected to an SCD. Inside the SCD, I ticked the radio button for "Use start and end dates to identify current and expired records" I choose
Start Date column: RecordStartDate
End Date column: RecordEndDate
variable to set date values : System::CreationDate
If I run the ETL today (November 5, 2013) and check the destination table after running the ETL. The RecordStartDate uses the date when I created the SSIS package (October 18, 2013). Shouldn't the package use the date when the ETL was run? correct me if I'm wrong.
System::CreationDate - The date that the package was created.
Shouldn't the package use the date when the ETL was run?
No, package should use the date then the package was created.
SQL Server Integration Services System variables
The dimension wizard provides a choice of variables for you to use. By default, the package developer can choose to use one of the following
System::StartTime
System::CreationDate
System::ContainerStartTime
(you can augment this list with your own variable that could be preset from an expressions etc)
If you want to use the StartTime for this column then you just have to change the appropriate derived column transform in the package.
If you decide to use System::StartTime, in the created transformation, you will get 2 Derived Columns transformation (one for ValidTo and second for ValidFrom)
update the expressions to match your desired datatype e.g, let's assume in the db, you column type is "date". (DT_DBDATE)(#[System::StartTime]). be aware that it can cause an error: 0xC0202009 to Data Flow Task, Insertion Destination [61]: Error Code SSIS DTS_E_OLEDBERROR. An OLE DB error has occurred. Error Code: 0x80004005.