I've built an SSIS package that runs on schedule weekly on Mondays. It looks for a .csv file that has a date at the end of the file name, which is 2 Saturdays ago from the date it runs (on Mondays). I need the variable to default to two Saturdays ago. Example: SSIS job runs on 7/9/2018, it needs to set it to 6/30/2018. SSIS needs to be runnable in case of failure, so if it has to be run again on, for example, 7/11/2018, it needs to default to 6/30/2018. This is a weekly job scheduled to run on Mondays.
I found a question posted similar to my need and the link is below for additional reference.
SSIS expression previous date without DateAdd()
I just figured out how to format the date extension as 'mmddyyyy' inside Expression Builder for the variable. Below is my code.
RIGHT("0" + REPLACE((DT_WSTR, 30) (DT_DATE) DATEADD("Day", 7*(DATEDIFF("Day", (DT_DATE)0, GETDATE())/7-1), (DT_DATE)0),"/",""),8)
Related
I have an SSIS package that runs each morning to pull the previous days file from an FTP server. I am using the code below to create the file name using the previous date. Everything works great with this except when today's date is the first day of the month. for example, if ran today (3/1/2021) this returns name_of_file_20210328.xml.gz, however yesterday's date is 2/28/2021 not 3. How do i update this to say if today's date is beginning of month return mm - 1?
"name_of_file_" + (DT_STR,4,1252)(DATEPART("yyyy",GETDATE())) + (LEN((DT_STR,2,1252)(DATEPART("MM",GETDATE()))) == 2 ? (DT_STR,2,1252)(DATEPART("MM",GETDATE())) : "0" + (DT_STR,2,1252)(DATEPART("MM",GETDATE()))) + (LEN((DT_STR,2,1252)(DATEPART("dd",DATEADD( "day",-1, GETDATE())))) == 2 ? (DT_STR,2,1252)(DATEPART("dd",DATEADD( "day",-1, GETDATE()))) : "0" + (DT_STR,2,1252)(DATEPART("dd",DATEADD( "day",-1, GETDATE())))) +
".xml.gz"
Create a variable, Yesterday of type DateTime. Specify that it uses an expression and use the following expression. This provides a consistent reference point you can test against and if you disable the expression, allows you to specify a date for boundary/special case checking (like a leap year 2020-03-01)
DATEADD("DAY", -1, #[System::StartTime])
The next steps, especially if you're starting out, is to build the date parts in separate variables. It doesn't cost any extra to use lots of variables in your package and makes troubleshooting so much easier.
Add a new variable, YearString of type String.
(DT_WSTR, 4)datepart("YYYY", #[User::Yesterday])
That was easy.
Now we need to deal with create a zero, left padded string. Right now, your expression looks like it's trying to determine if day or month has 2 digits. I have a cleaner expression.
We're going to convert the day/month to a string and then prepend a zero to it. For Jan-Sep, we'll end up with a 2 character expression, Oct-Dec, we'll have a three character expression e.g. 011. For Day, similar bit where it's 01-09 or 010-031. We will then take the last two characters from the string. For the two character strings, it's a no-operation and for the three character variant, it gets us what we want.
Add Variable MonthString, as type string, to your package
RIGHT("0" + (DT_WSTR, 2)datepart("MONTH", #[User::Yesterday]), 2)
Add Variable DayString, as type string, to your package
RIGHT("0" + (DT_WSTR, 2)datepart("DAY", #[User::Yesterday]), 2)
Take a moment and look at your Variable collection. You can see that you have all the building blocks needed to properly construct your YYYYMMDD string. If something is wrong, it's small enough snippet to play with it. If not, break it up into smaller Variables.
Now that we have defined #Yesterday and then built YearString, MonthString and DayString off of Yesterday, all we have to do is bring it all together with the concatenation + operator
Back to the Variable well, creating #CurrentFileName of type string
"name_of_file_" + #[User::YearString] + #[User::MonthString] + #[User::DayString] + ".xml.gz"
Results in a value of name_of_file_20210216.xml.gz
Not addressed in this answer but things you should think about
What happens when the job doesn't run at all today (server fails horrifically, the data source is down, etc)? To pick up and process 2 days ago file, you would need to edit this package, run it through whatever change review process is applicable, deploy to prod, run it and then go back to the process yesterday file package.
That's not fun to type much less consider. You certainly aren't going to change the server time to trick the expression into being yesterday.
I am an advocate for passing the run date to the package (mechanism depends on how you deploy/run packages). In that situation, it's been my experience that it's a far easier bureaucracy fight to change the calling parameter (because no "code" has changed) than to get an emergency code change run through.
How can I write an SSIS expression that determines whether a date is a weekend day?
I have this expression, but it seems it only 'works on my machine'
DATEPART( "Weekday", #[User::CurrentDate] ) ==6 || DATEPART( "Weekday", #[User::CurrentDate] ) ==7
After I deploy SSIS package to server, is suddenly changes order of numbers and
Monday became 2.
When on my computer Monday is 1, exactly as I expected.
I checked SQL Server itself, but it seems is something inside SSIS decide if Sunday or Monday is first day.
I cannot find exact question related to this issue.
Well, if you must do this in an SSIS expression, here is one way to work around the problem: you would compare the DW of your variable to the DW of a known constant (like August 3, 2019, which is a Saturday in any locale that I know of).
In pseudocode, IF ( DW(DateVariable) = DW("20190803") OR DW(DateVariable) = (DW("20190803")+1)%7 ) THEN {DateVariable is a weekend}
Personally, I would look for a way to do this in TSQL. Import the raw data "as is" into a staging table, and then do the transformation while moving the data to the destination table with a stored procedure.
DATEPART("DW",GETDATE()) or DATEPART("WEEKDAY",GETDATE()) works perfectly fine on both SQL server and local.
How are you passing value to #[User::CurrentDate] variable(Please check if same date is set when packages is executed on SQL server and on local)
I changed DateTime on my local and tested your expression as part Of Execute SQL task and Derived Column Transformation which gives me 1 for Sunday, 2 for Monday and so on and the result matches with SQL.
SQL query to get day of week:
SELECT GETDATE() Today,DATENAME("DW", GETDATE()) DayofWeekToday,
DATEPART("DW",GETDATE()) DayNumberToday,GETDATE()-1 Yesterday,
DATENAME("DW", GETDATE()-1) DayofWeekYesterday,DATEPART("DW",GETDATE()-1) DayNumberYesterday
Adding more to my answer : There is a difference between DateFirst and DatePart. DateFirst sets first day of the week while Datepart gives you day of the week. use this sql function to check what's your first day of the week
Set Datefirst 7 -- To set first day of week (7 = Sunday)
SELECT ##DATEFIRST -- To check first day of week
**Update your system settings to match first day of the week with sql server and you should get same values when you evaluate expression.
Your expression looks good its only your first day of week on local and sql server are not matching.
Once I updated first day of week on SQL, I was able to replicate the issue you are facing.
I have a ssis package, which runs on date parameter. If we dont specify the date it always load data for yesterday's date. And if we give any specific date like '2015-05-10', It should load for that date. How can we achieve this dynamically (using package configuration)? Once we load for any specific date, package should be set for yesterday's date dynamivally. Please guide me to achieve this as I am new to SSIS.
Thanks in advance
Add a parameter to Package ParamDate. This parameter has to be manually provided a value(e.g. 03-21-2015). Leave it blank(NULL value) if yesterday's date has to be considered.
Now define a variable inside your package VarDate
Have an expression for VarDate like this -
ISNULL(#[$Project::ParamDate])?DATEADD("day", -1, getdate()):#[$Project::ParamDate]
I am assuming you are loading data in a DFT.
So in the source query for that, you just need to add an extra condition in the where clause
SELECT ...... FROM SomeTable WHERE LoadDate = ?
In the "Parameters..." section of the source, assign the variable VarDate to the 0. For further details on how to map parameters see here.
Hope it helps.
EDIT
As this was tagged under SSIS 2012,my solution involved project parameter, which is a feature under project deployment model. Package configurations are on the hand under package deployment model.
If you want to use SQL Server package configuration table, then you can follow the steps below:
Lets say the table's name is SSISConfiguration
Have an Execute SQL task where you read this date from this table.
SELECT ISNULL(TstampUpdateDate, DATEADD(dd, -1, GETDATE()) FROM SSISConfiguration
This script will fetch the value of TstampUpdateDate from the SSISConfiguration table and if it is NULL(set below), it will return yesterday's date.
Map the result set to the variable #VarDate.
Have one more Execute SQL task after this step which will update this field to NULL.
Rest of the steps will be same as above(starting from the DFT bit).
Next time the package runs, if the date field is not updated, it will query as per yesterday's date.
I have an SSRS 2005 report that has a dataset for to an Oracle database. The report that i have essentially just pulls all data back from an audit log. This report works perfectly fine and i have scheduled thi to run and send an email using the new subscription method within SSRS.
The only issue i have is i want the report to run on the last day of the month (its set up to do this alreday) and run the report based on that month only. Getting the report to run specifically for a month is what im unsure on? I though i would be able to set parameters within the report but these only enable my user to select a calendar date?
Is there a DATEADD or DATEPART type of function i can use to complete this task?
Regards
B
Best bet would be to set up a dynamic date default for the date patameter at the report level, and set up the subscription to use this default value, so whenever the subscription runs it would automatically have the appropriate date.
Probably the easiest thing to do is set the parameter as the current date, if that would work, e.g. something like =Today(). So that will be calculated whenever the subscription runs.
You can set the default value for whatever you need; first day of the current month, etc. You should able to set whatever value you need without too much trouble.
So for the first day of the month something like:
=DateSerial(Year(Today()), Month(Today()), 1).
Users running the report ad-hoc could just reset the parameter as required.
I'm assuming the report handles the amount of data to return correctly assuming the right data parameter is sent.
2 Parameters #DateFrom and #DateTo
Default Value for #DateFrom that outputs the first day of the current month:
=DateAdd("d",-Day(Today())+1,Today())
Default Value for #DateTo that outputs the last day of the current month:
=DateAdd("d",-Day(Today())+1,DateAdd("m",1,Today()))
Scenario:
I have a lookup table which has a date column, I need to look at this date column and check if its today's date, if not then wait for 5 mins and check the same thing again, and if the date is current send an email and exit the loop, and if after 6 retries if the date is not current execute a SQL task.
I have a ForLoop Container with the following settings:
InitExpression : #[User::Counter] = 0
EvalExpression : #[User::Counter] < 6
AssignExpression : #[User::Counter] = #[User::Counter] + 1
How / Where do I check the date :
SELECT ControlTimeStamp from LOOKUPTABLE
WHERE ControlTimeStamp = DATEADD(dd, 0, DATEDIFF(dd, 0, GETDATE()))
Note:
I'm using Business Intelligence Development Studio (BIDS) 2008 for SSIS package development.
I think you'll want an approach like this. Execute your SQL Task to determine whether today is your date. From there you'll need to either sleep for N minutes or you'll want to send an email. The trick is to use an Expression on the Precedence Constraint between the Execute SQL Task the children.
My implementation differs slightly from what yours but the concept remains the same. I created two variables, #ActualDate and #ReferenceDate. #ActualDate is today and #ReferenceDate gets set from the Execute SQL Task. I then see whether they are equivalent. For what you have, if you get a result, then you know the send mail condition has been met so change your Expressions to meet that.
What isn't shown is how to terminate the loop early as I'm not quite certain how to do that.