I am in the process of uploading the delta load to a CSV file. I have set up a conditional split in the package which will compare the last modified date with the below expression:
RIGHT("0" + (DT_STR,2,1252)DATEPART("dd",GETDATE()),2)
I need to replace above with:
(DT_STR,2,1252)DATEADD("dd",-1,GETDATE())
But it is throwing a warning error message. The package is failing.
The only possible thing you can replace with that second expression is your GETDATE() method. More precisely, you are probably trying to retrieve the previous day based on your second expression.
From:
RIGHT("0" + (DT_STR,2,1252)DATEPART("dd",GETDATE()),2)
Replaced:
RIGHT("0" + (DT_STR,2,1252)DATEPART("dd",DATEADD("dd",-1,GETDATE())),2)
Example:
Related
I'm trying to construct an SSIS package where one of the columns is a DT_DBDataTimeStamp format.
My problems is that I have to convert this to a DD/MM/YYYY format and I can't use a C# script.
I'm trying to use a derived column by having difficulty in getting the correct format.
I've tried Convert(varchar, Date_Column, 103) - doesn't work.
I've tried DAY(Date_Column) + MONTH(Date_Column) + YEAR(Date_Column) - doesn't work.
Anybody know how I can do this?
It would need to be varchar if you want this (actually, I will use nvarchar for simplicity):
right("0" + (DT_WSTR,2) day(Date_Column), 2) + "/" +
right("0" + (DT_WSTR,2) month(Date_Column) , 2) + "/" +
(DT_WSTR,4)year(Date_Column)
I have a date 20130131 in csv which I'm trying to convert to 2013-13-01 using Derived Column in SSIS. I have used this expression but it does not seem to work.
(DT_DBTIMESTAMP)(SUBSTRING(DATE_DECISION_TO_REFER,1,2) + "-" + SUBSTRING(DATE_DECISION_TO_REFER,3,2) + "-" + SUBSTRING(DATE_DECISION_TO_REFER,5,2))
How do I rewrite this expression to produce the correct output with a value of data type DT_DBTIMESTAMP?
Fast Parse is a much more elegant solution
Add a Data Conversion Data Flow Component.
Right click, Show Advanced Editor...
Goto Input and Output Properties
Expand Data Conversion Output and click on the date column
Set FastParse to True in the Custom Properties
Make sure the data type is set to database date [DT_DBDATE]
Run the ssis package again the yyyymmdd dates should flow smoothly from nvarchar to date
As per Kyle Hale suggestion, below is my answer
SUBSTRING([DATE_DECISION_TO_REFER],1,4) + "-" +
SUBSTRING([DATE_DECISION_TO_REFER],5,2) + "-" + SUBSTRING([DATE_DECISION_TO_REFER],7,2)
To educate you so that you will never face this issue again, this is how expression works
Get 4 characters from DATE_DECISION_TO_REFER starting at position 1, add a dash,
get 2 characters from DATE_DECISION_TO_REFER starting at position 5,
add a dash then add 2 characters from DATE_DECISION_TO_REFER starting at position 7.
Hope this will help.
Expression:
You should use the expression to format the date to yyyy-MM-dd like shown below. This expression will format the incoming value 20130131 to 2013-01-31 and then will convert to appropriate data type. The expression uses the incoming column named Column0.
(DT_DBTIMESTAMP)(SUBSTRING(Column0,1,4) + "-" + SUBSTRING(Column0,5,2) + "-" + SUBSTRING(Column0,7,2))
Issue:
Your expression is taking the value 20130131 and then incorrectly converting it to the value 20-13-01. This is an invalid format for dates and hence the expression is failing to type cast it to DT_DBTIMESTAMP data type.
Reading from a flat file that has a column containing the year and month in the following format --> "201212".
I need to insert this into a DATETIME2 column but first I must parse it with "/" and add a "01" as the day. Such that 201212 would become 2012/12/01
My expression in my Derived Column Transformation looks like this:
(DT_DBTIMESTAMP2,0)((DT_WSTR,4)SUBSTRING(RptMthDTM,1,4) + "/" + (DT_WSTR,2)SUBSTRING(RptMthDTM,5,2) + "/" + "01")
This seems like it should work and SSIS accepts this(as in it can parse the expression) but when running the package it throws a completely useless error "An error occurred while attempting to perform a type cast." along with the column it had the error on.
I didn't build this package, it was pawned off on me and I was told to get it working.
Thanks in advance interwebs family.
DT_DBTIMESTAMP2 can only be converted from a specific string format: yyyy-mm-dd hh:mm:ss[.fffffff]
You can use this expression instead:
(DT_DBTIMESTAMP2,0)((DT_WSTR,4)SUBSTRING(RptMthDTM,1,4) + "-" + (DT_WSTR,2)SUBSTRING(RptMthDTM,5,2) + "-" + "01 00:00:00")
More details:
[http://msdn.microsoft.com/en-us/library/ms141036.aspx][1]
I would like to create random unique filename in SSIS using expression builder for Flat file connection Manager - ConnectionString.
Should be like: "Custom name" + "unique part" + "csv"
Actualy a timestamp in miliseconds would do.
Any idea?
I suggest you using the expression design to do that.
Here Im using the ExecutionGuid to generate the value. It may not be the best idea because the result is quite big but you can use datetime functions as well to create something on the format you desire:
It's a litte convoluted thanks to all the REPLACEs, but you can use:
"Custom Name " + SUBSTRING(REPLACE(REPLACE(REPLACE((DT_WSTR, 50)(GETDATE()), "-",""), ".", ""), ":",""), 1, 18) + ".csv"
as a starting point. This will return:
Custom Name 20120529 133526359.csv
as a possible filename. It's not 100% guaranteed unique, and it's obviously not random, but I think it's all that can be done just in an Expression. Alternatively, you can use a Script Task to generate a more random string using C#/VB.NET, write that into a variable and use the variable in your Expression.
string.Format("Custom name{0}.csv", Path.GetRandomFileName());
or
string.Format("Custom name{0}.csv", DateTime.Now.ToString("yyyyMMddhhmmssfff"));
This will be to get unique date time..
" + (DT_WSTR, 30)(DT_DBDATE)GETDATE() + "_" +
(DT_WSTR, 05) DATEPART("Hh",GETDATE()) +
(DT_WSTR, 05) DATEPART("Mi",GETDATE()) +
(DT_WSTR, 05) DATEPART("Ss",GETDATE()) +
".csv"
I found that it was easy to find how to create the expression but difficult to find instructions on how to set your output filename to use this expression. It is not a straightforward process. Here is an answer to this problem. I hope this helps somebody.
https://blogs.msdn.microsoft.com/sqlgardner/2015/06/18/ssis-tip-using-a-file-path-parameter-for-multiple-flat-files/#comment-1755
I need to assign a formatted date to a column in a data flow. I have added a Derived shape and entered the following expression for a NEW column - Derived Column = "add as new column":
"BBD" + SUBSTRING((DT_WSTR,4)DATEADD("Day",30,GETDATE()),1,4) +
SUBSTRING((DT_WSTR,2)DATEADD("Day",30,GETDATE()),6,2) +
SUBSTRING((DT_WSTR,2)DATEADD("Day",30,GETDATE()),9,2)
The problem is that the Derived Column Transformation Editor automatically assigns a Data Type of Unicode string[DT_WSTR] and a length of "7". Howver, the length of a string is 11, therefore the following exception is thrown each time:
[Best Before Date [112]] Error: The "component "Best Before Date" (112)" failed
because truncation occurred, and the truncation row disposition on "output column
"Comments" (132)" specifies failure on truncation. A truncation error occurred
on the specified object of the specified component.
Does anyone know why the edit is insisting on a length of 7? I don't seem to be able to change this.
Many thanks,
Rob.
I can't understand why SSIS is measuring that column as only resulting in a seven character field - but to force it to provide an 11 character column for it, modify your expression slightly to this:
(DT_WSTR, 11)("BBD" + SUBSTRING((DT_WSTR,4)DATEADD("Day",30,GETDATE()),1,4) + SUBSTRING((DT_WSTR,2)DATEADD("Day",30,GETDATE()),6,2) + SUBSTRING((DT_WSTR,2)DATEADD("Day",30,GETDATE()),9,2))
What you want is:
"BBD" + (DT_WSTR,4)YEAR(DATEADD("Day",30,GETDATE()))
+ RIGHT("0" + (DT_WSTR,2)MONTH(DATEADD("Day",30,GETDATE())),2)
+ RIGHT("0" + (DT_WSTR,2)DAY(DATEADD("Day",30,GETDATE())),2)
The issue is in how you are converting your dates to a string. The calls to DATEADD return a full date & time. Next, you then have either (DT_WSTR,4) or (DT_WSTR,2) to convert that date into either a 4 or 2 character string. On my system, converting a datetime to a string defaults to "Aug 24 2011 4:18PM". So the first 4 characters gets you "Aug " and the first 2 characters gets you "Au". Then, you are extracting substrings using SUBSTRING. For your last two calls to SUBSTRING, you are starting the substring past the end of the 2 character string you converted the date into. This is why SSIS displays 7 characters:
"BBD" + "Aug " + "" + ""
3 + 4 + 0 + 0 = 7
It is better to use the built in functions to extract the Year, Month and Day from a datetime rather than converting to a string and then grabbing substrings. If you really wanted to use substrings, you would need to add a call to CONVERT to get the datetime to a specific string format, otherwise you will get whatever the default is for your locale setting in Windows. This could be different on each PC.
What release and service pack of SQL are you using?
I just tried this on my machine and had no problems changing the result size from 7 to 11. Is it possible that you have not installed all the service packs?
Are you replacing your existing field, and is that field possibly 7 chars long? The thing with the Derived Column Transform is that you can't change the field types (including length) of the existing fields.
Try to add a new field instead.
If that's not working, try adding an explicit cast around the whole expression.
(DT_WSTR,11)("BBD" + SUBSTRING((DT_WSTR,4)DATEADD("Day",30,GETDATE()),1,4) + SUBSTRING((DT_WSTR,2)DATEADD("Day",30,GETDATE()),6,2) + SUBSTRING((DT_WSTR,2)DATEADD("Day",30,GETDATE()),9,2))
Right click on "Derived Column" open "Show Advanced Editor" Select "Input and output Properties" tab.
Got to "Derived column output" => "Output Columns" => "Derived Column 1" (added by you)
In right side panel go to "Data type Properties" section=> DataType=>
Select "String [DT_STR]
click OKImage showing steps
This will solve your problem.