I want to insert a value "5/27/2013 16:42:37.490000" (Read from a flat file (DT_STR)) into a column(datetime) of SQL Server table . If I try to cast it with (DT_DBDATE) or DT_DBTIMESTAMP in a derived column , it gives an error .
[Derived Column [130]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "component "Derived Column" (130)" failed because error code 0xC0049064 occurred, and the error row disposition on "output column "Derived Column 1" (155)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
How can I go about it ?
Thanks
The value is a datetime2 type .AFAIK SSIS doesn't support datetime2 .You need to store it in database as string and then update the column by converting it to datetime2.
Here is Microsoft Connect Issue
Update: Using DT_DBTIMESTAMP2 you can convert the string to proper datetime format
The below code works perfectly fine in Derived Column
(DT_DBTIMESTAMP2,7)"2013-5-27 16:42:37.490000"
7 is the precession here .The above code won't work if the format of datetime is different.For example MM/DD/YYYY HH:mm:ss.ffffff .
However you can use Script component and pass array of different datetime formats to Datetime.ParseExact function
Step1: Drag a Script component and create a new output column of DT_DBTIMESTAMP datatype and name it as NewDate.
Step2: Write the below C# code
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
string[] format = new string[] { #"M/dd/yyyy HH:mm:ss.ffffff",
#"MM/dd/yyyy HH:mm:ss",
#"M/d/yyyy HH:mm:ss" ,
#"M/dd/yyyy HH:mm:ss.ffffff",
#"MM/dd/yyyy HH:mm:ss.ffffff",
#"M/d/yyyy HH:mm:ss.ffffff"};
DateTime dt = DateTime.ParseExact(Row.Date,
format ,
CultureInfo.InvariantCulture,
DateTimeStyles.None);
Row.newDate = dt;
}
I won't say it's the perfect solution, but just a workaround.
Format the input and then insert. The formatted data after using the below expression works well with either datetime or datetime2.
SUBSTRING(TMS_CREAT,(FINDSTRING(TMS_CREAT,"/",2) + 1),4) + "-" + SUBSTRING(TMS_CREAT,1,(FINDSTRING(TMS_CREAT,"/",1) - 1)) + "-" + SUBSTRING(TMS_CREAT,(FINDSTRING(TMS_CREAT,"/",1) + 1),2) + SUBSTRING(TMS_CREAT,(FINDSTRING(TMS_CREAT,"/",2) + 5),16)
Related
I am using AnyPoint studio and Im trying to insert a expression into db like this:
insert into table_name (exitdate)
values ('now() as Date{format: "yyyy-mm-dd"} + |P27D|')
But I'm getting an error: Data truncation: Incorrect date value: 'now() as Date{format: "yyyy-mm-dd"} + |P27D|' for column 'exitdate' at row 1.
What am I doing wrong?
Thank you.
You are using a DataWeave expression inside the SQL query. You should assign the result of the expression to a parameter and use the parameter instead.
Note that dates don't have a format. The format applies when converting to/from other types, like a String.
I have the following simple SSIS Package to process a comma delimited file:
I'm having problems with date columns - specifically MoveInDate which the input column has data in the format '20090731000000'.
It goes into a database column MoveInDt which has a Datetime data type and data ends up in the format:
'2009-07-31 00:00:00.000'
In SSIS I have specified the input column to be DT_DBTIMESTAMP. I'm unable to figure out how to do a simple 'if MoveInDate is blank, load null, else change data to 'YYYY-MM-DD 00:00:00.000'
Right now I can't even seem to figure out how to load a hardcoded date
This fails
As does
"2019-01-01"
Or
(DT_DBTIMESTAMP)"2019-01-01"
(DT_DBTIMESTAMP)(DT_DATE)"2019-01-01"
Error: 0xC02020A1 at Data Flow Task, Flat File Source [11]: Data conversion failed. The data conversion for column "Move In Date" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
Error: 0xC0209029 at Data Flow Task, Flat File Source [11]:
Looking up that message I've found posts about the data types not matching on the input to output column, but looking at it ...
Checking the flat file input column, it is defined as DT_DBTIMESTAMP
And the destination column is also defined as DT_DBTIMESTAMP when looking at input output properties.
I've looked at many questions here and elsewhere and have tried reducing the input to one simple row in the input file. I've tried recreating the package several times.
If the data is stored in the following format in the flat file
20090731000000
Then leave the source column data type as DT_STR and add a dervied column with the following expression:
(ISNULL([MoveInDate]) || [MoveInDate] == "") ? NULL(DT_DBTIMESTAMP) :
(DT_DBTIMESTAMP)(LEFT([MoveInDate],4) + "-" +
SUBSTRING([MoveInDate],5,2) + "-" +
SUBSTRING([MoveInDate],7,2) + " " +
SUBSTRING([MoveInDate],9,2) + ":" +
SUBSTRING([MoveInDate],11,2) + ":" +
SUBSTRING([MoveInDate],13,2) )
If data is stored in the following format:
2009-07-31 00:00:00
Use the following expression:
(ISNULL([MoveInDate]) || [MoveInDate] == "")
? NULL(DT_DBTIMESTAMP)
: (DT_DBTIMESTAMP)[MoveInDate]
If data is stored in the following format:
2009-07-31
Use the following expression:
(ISNULL([MoveInDate]) || [MoveInDate] == "")
? NULL(DT_DBTIMESTAMP)
: (DT_DBTIMESTAMP)([MoveInDate] + " 00:00:00")
I have a column in a text-file with dates like this: 20131207.
I and trying to convert the date like this but everything I have tried has failed.
Here is my code: I have modified it many times to fix this problem.
(LEN(TRIM(MyDate)) < 8) || (TRIM(MyDate) == " ") ||
(ISNULL(MyDate)) ? NULL(DT_DBDATE) :
(DT_DBDATE)(SUBSTRING(TRIM(MyDate),1,4) + "-" +
SUBSTRING(TRIM(MyDate),5,2) + "-" + SUBSTRING(TRIM(MyDate),7,2))
Where am I going wrong?
I checked Retain Null Values in the Flat Source Task.
Update: Here is the error message:
The conditional operation failed.
The "Rename Columns (My_Table)" failed because error code 0xC0049063 occurred,
and the error row disposition on "Rename Columns (My_Table).Outputs[Derived
Column Output].Columns[MyDate]" specifies failure on error. An error occurred
on the specified object of the specified component. There may be error
messages posted before this with more information about the failure.
I need to convert the data in the text file to date, from 20131207 to 2013-12-07 but I want to insert null if the data in the text file is less then 8 in length or cannot be converted to date.
try this:
add a new flat file source then add a derived column then in expression do this
`LEFT(date,4) + "-" + SUBSTRING(date,5,2) + "-" + RIGHT(date,2)`
the package :
with output like this :
Trim(MyDate)==" " should be Trim(MyDate)==""
I have the following line:
command.Parameters.Add("#date", SqlDbType.DateTime, 8).Value = date_of_trip;
where date_of_trip is a string containing 25-9-2013 (i.e. day-month-year)
This gives me an error message:
System.ArgumentException: Cannot convert 25-9-2013 to System.DateTime.
Parameter name: type ---> System.FormatException: String was not
recognized as a valid DateTime.
However, if date_of_trip is 1-1-2013, it seems to work perfectly!
You should parse the string to datetime before you send it to the db.
For example:
DateTime dt;
if(DateTime.TryParse(date_of_trip, out dt))
{
command.Parameters.Add("#date", SqlDbType.DateTime, 8).Value = dt;
// ...
}
else
{
// you should use a CompareValidator to check if it's a valid date
}
If you want to ensure that your given format will work even if the current-culture of the server will change you can use DateTime.TryParseExact with format "dd-MM-yyyy":
if (DateTime.TryParseExact(date_of_trip, "dd-MM-yyyy", System.Globalization.CultureInfo.InvariantCulture, System.Globalization.DateTimeStyles.None, out dt))
{
// ...
}
I guess it's because the server expects the first parameter to be a month. Try to use a real DateTime or the format yyyy-mm-dd (i.e. 2013-09-25), which should always work.
I got this error while debugging in SSIS:
Error: 0xC0049064 at Data Flow Task, Derived Column [70]: An error occurred while attempting to perform a type cast.
Error: 0xC0209029 at Data Flow Task, Derived Column [70]: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "component "Derived Column" (70)" failed because error code 0xC0049064 occurred, and the error row disposition on "output column "EVENT_DT" (118)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
Error: 0xC0047022 at Data Flow Task: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Derived Column" (70) failed with error code 0xC0209029. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED. Thread "WorkThread0" has exited with error code 0xC0209029. There may be error messages posted before this with more information on why the thread has exited.
Information: 0x40043008 at Data Flow Task, DTS.Pipeline: Post Execute phase is beginning.
Information: 0x40043009 at Data Flow Task, DTS.Pipeline: Cleanup phase is beginning.
Information: 0x4004300B at Data Flow Task, DTS.Pipeline: "component "DataReaderDest" (143)" wrote 0 rows.
Task failed: Data Flow Task
Warning: 0x80019002 at Package: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (4) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "Package.dtsx" finished: Failure.
My expression is:
(DT_DBTIMESTAMP)(SUBSTRING(EVENT_D,7,4) + "-" +
SUBSTRING(EVENT_D,4,2) + "-" +
SUBSTRING(EVENT_D,1,2) + EVENT_T)
My original data are in this sequence:
EVENT_D: DD/MM/YYYY
EVENT_T: HH:MM:SS
Any help are appreciated. I have try changing my expression numerous time but still fails.
I suspect that there are some date time values which are not in the correct format .So SSIS throws error while parsing them .
In order to find to incorrect date time value from your source table try to redirect the error rows from Derived Transformation and check the incorrect data using a data viewer
The problem with substring values are if the string data are not in the correct format then the Derived Transformation will throw error .
In your OLEDB source component try to write the sql and get only the correct datetime values
Ex:
Select col1,
case when isDate(EVENT_D) = 1 THEN EVENT_D ELSE NULL
END as [EVENT_D],
Col2,EVENT_T,other columns
from yourTable
in your derived transformations use your code to convert into DT_DBTIMESTAMP type .
Else Try to use a script component and parse the EVENT_D and EVENT_T values and convert to datetime values. No need to use Derived column with all those substring values
create a New Output column Valid_D with the datatype as DT_DBTIMESTAMP.Select the 2 input columns EVENT_D and EVENT_T in the available input Columns in Script Transformation Editor
VB.NET code
Dim result As DateTime
If DateTime.TryParseExact(Row.EVENT_D, "dd/MM/yyyy",
Nothing, Globalization.DateTimeStyles.None, result) Then
Row.ValidD = result.Add (TimeSpan .Parse (Row.EventT ) );
End If
C# code
DateTime result;
if (DateTime.TryParseExact(Row.EventD, "dd/MM/yyyy",
CultureInfo.InvariantCulture, DateTimeStyles.None,out result))
{
Row.ValidD = result.Add (TimeSpan .Parse (Row.EventT ) );
}
Now you can use Valid_D column which is of DateTime type in your subsequent transformations
Update : The problem with your syntax is you cannot add date+ time in the string format . You need to parse individually both EVENT_D and EVENT_T .
(DT_DBTIMESTAMP)(SUBSTRING(EVENT_D,7,4) + "-" +
SUBSTRING(EVENT_D,4,2) + "-" +
SUBSTRING(EVENT_D,1,2) + EVENT_T)
So your syntax is not valid.
The isDate function shows NULL for 30/04/2012 because as per MSDN
The return value of ISDATE depends on the settings set by
SET DATEFORMAT, SET LANGUAGE and Configure the default language
Server Configuration Option.
but it returns 1 for the value 01/05/2012 and 02/05/2012 because it takes as MM/dd/YYYY
so the 1st date is Jan 5th 2012 instead of May 1st 2012
So the soultion is use script component and parse these values into valid date and time and then use it in your subsequent transformations .
Please check my script transformation code above
Update 2:
I think your using SSIS 2005 .The code should be
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim result As DateTime
If DateTime.TryParseExact(Row.EVENTD, "dd/MM/yyyy", Nothing, Globalization.DateTimeStyles.None, result) Then
Row.ValidD = result.Add(TimeSpan.Parse(Row.EVENTT))
End If
End Sub
after Script transformation ,you don't need to use Derived component .The result obtained in the column Valid_D contains the valid value which is of datetime format
If you wanted to do this without a script, you could have just changed your expression from
(DT_DBTIMESTAMP)(SUBSTRING(EVENT_D,7,4) + "-" +
SUBSTRING(EVENT_D,4,2) + "-" +
SUBSTRING(EVENT_D,1,2) + EVENT_T)
to
(DT_DBTIMESTAMP)(SUBSTRING(EVENT_D,7,4) + "-" +
SUBSTRING(EVENT_D,4,2) + "-" +
SUBSTRING(EVENT_D,1,2) + " " + EVENT_T)
Not having the space between the date and the time portions is what was most likely causing your original error. You were getting
2012-04-3012:25:37
instead of
2012-04-30 12:25:37
I faced the same problem while converting a string containing date to datetimestamp field. It is all due to spaces in the field.
So we need to check for space and then convert it to datetimestamp.
Below is the expression for the same.
LEN(TRIM([Field1])) == 1 ? NULL(DT_DBTIMESTAMP) : (DT_DBTIMESTAMP)(SUBSTRING([Field1],1,4) + "-" + SUBSTRING([Field1],5,2) + "-" + SUBSTRING([Field1],7,2) + " " + SUBSTRING([Field1],10,2) + ":" + SUBSTRING([Field1],12,2))