I'm trying and playing around a CSV file to import data into the SQL Server table using SSIS.
The package is simple with File Source Task and SQL Server Destination.
The CSV file has 2 fields Transaction_Date and Account_Created. The dates in these fields are the format of 1/2/2009 6:00:00 AM. I am seeing the below error message:
"Error: An exception has occurred during data insertion, the message returned from the provider is: The given value of type String from the data source cannot be converted to type datetime of the specified target column."
Steps I tried below:
I tried using various other destination transformations.
I tried playing around the Data Types inside the Flatfile Connection Managers.
I tried using the Data Conversion Transformations between the Source Task and SQL Server Destination.
When I tried to load the data by providing connection only to Transaction_Date it works. However, when I tried to load by providing connection only to Account_Created it fails.
I am sure I'm missing something silly. Please help.
Regards,
KK
I tired a different method to build the package from start using the Wizard. I used the actual CSV file which had many other columns like Price, Product_name and so on. When I tried to execute the package I see a different error message as below:
"[Destination for AdventureWorks [58]] Error: There was an error with input column "Price" (91) on input "OLE DB Destination Input" (71). The column status returned was: "The value could not be converted because of a potential loss of data.".
"
When I tried a CSV file with only 2 date fields it worked excellent.
I am really puzzled now and thinking this is some kind of data type issues which I am not getting it correct. Can some one pls shred some light into this problem?
Regards,
KK
To load the first two fields (Transaction_Date, Account_Created), you need a DataFlow Task that contains:
Flatfile Source
Derived Column (create two columns to Replace 'Transaction_Date' and 'Account_Created' with formula below)
SQL Server Destination
Notes:
Date format like '1/2/2009 6:00:00 AM' is not parsed by SSIS, make sure the Flatfile Connection Manager treats the fields as Strings (length > 22)
In Derived Column, you can parse '01/02/2009' with this formula:
(DT_DBDATE)(SUBSTRING([Column 2],7,4) + "-" + SUBSTRING([Column 2],4,2) + "-" + SUBSTRING([Column 2],1,2))
The current date format that you have in the file '1/2/2009' makes the conversion very tricky due to the lack of advanced datetime parsing functions of SSIS. Depending on the day and month you will have to subtract from a variable length string, therefore you will have to combine SUBSTRING with FINDSTRING to determine the position of the separator '/'
Good luck
Related
We are trying to push a single order in to MS CRM (dev instance) via SSIS package.
Most of the columns coming from source (staging table) are of data type 'DT_STR' and their mapped fields in CRM are of 'DT_WSTR' data type.
I already looked for the solution on this site but in all cases the question is for converting wstr to str. In my case I need to convert str to wstr. when I run the package I get error saying,
Column xxxx cannot convert between unicode and non unicode string data type
I have already tried two solution:
1. Right click on the OLE source and convert datatype to wstr and
2. Using 'Data Conversion'
In both cases the error remains the same. Has anyone else had similar issue?
In OLE DB Source properties don't change data types. If you want you can change in
SELECT statement in OLE DB source.
you can change in 'Data Conversion'
Derived Column element
In Derived Column element code is:
(DT_WSTR, 50)([YourString])
Don't replace column, add new column in Derived column element.
You doing something wrong if you can't convert, you don't give real error message (or picture of your design), real error message is in Output window when you execute the package.
I'm pretty new to SSIS and I'm running into an issue where I'm using a sql statement in an OLE DB Source step to pull my data and this data contains 5 DATETIME fields that need to be converted and all of the data is going into a flat file.
I'm doing the conversions in sql like so:
REPLACE(CONVERT(VARCHAR(10), birthdate, 101), '/', '') AS 'Date of Birth (MMDDYYYY)
In the SSIS Flat File Connection Manager my DataType is string [DT_STR] and my OutputColumnWidth is 8. This is working in SSIS without truncation as the above conversion creates an 8 character varchar. If I try converting to a varchar(8) it truncates the data in sql server management studio.
My problem is that my next conversion attempt is failing. Here's the sql code:
REPLACE(CONVERT(VARCHAR(10), exp_date, 103), '/', '') AS 'Expiration Date
I get the following error: "The value could not be converted because of a potential loss of data" even though everything has been reproduced with the same settings. This conversion converts the date to a 'DDMMYYYY' format.
I know from researching that this is a data conversion error but I have no idea how to get around it especially with the first conversion working correctly.
The interesting thing is that when I redirect the rows on error to a different flat file the dates get pulled and converted with no problem at all.
I'm at a loss here and would appreciate any help from the group.
I figured out why it wasn't working. When I first created the package only the birthdate was converted since that's where my attention was due to the different date format. After I ran it and verified that all of the data was being pulled I realized that I needed to convert the other 4 dates as well. The problem is that even though I made the change in sql and in the Flat File Connection Manager the metadata was still set to the original settings for those 4 fields. I recreated the step and everything converted correctly. Thanks for taking the time to respond
I am developing a SSIS package that will open an Excel spreadsheet and import the data into a database table in SQL Server 2008. When I try to convert the Excel column data type: Unicode String [DT_WSTR] to a unique identifier data type: unique identifier [DT_GUID], I get the following error:
"Invalid character value for cast specification"
What do I need to do to resolve the conversion error?
I used a Derived Column Transformation Editor and to wrap the excel column value in squrly brackets {} in order for the SSIS package to properly convert the Unicode String into a GUID.
I want to mention that this will not work with SSIS 2012 in Visual Studio. If you try to do this you get an error on the derived column transformation task. I've tried both of these:
(DT_GUID)[ColumnName]
(DT_GUID)("{" + [ColumnName] + "}")
Both of these will fail.
However, if you simply set to ignore those errors instead of fail. It will work fine. Really spend way too much time trying to get this to work.
my date format is in this format YYYYMMDD and when I am converting the same with data conversion task I am getting below error:
The data value cannot be converted for reasons other than sign mismatch or Data Overflow.
In my Dataconversion I have select DT_DATE
and in database the column datatype is date.
But the strange thing is that when I am executing my package and doing casting as source SELECT CAST(myDate AS DATE) package is working fine.
It's a common issue. If you use a derived column transformation, you will need to slice it out into the component parts (years, months, days) and then concatenate it back together before casting. That's ugly and time consuming for me.
Instead, assuming this is coming from a flat file, just make it a date on import by setting the type in your connection manager to the date type that will be compatible with your destination. Then on your flat file source, under advanced settings, set FastParse to true for that column. See my answer on Import String Date in Derived column for a pictoral walkthrough of it. Also addressed it on SSIS importing datetime column into SQL Server 2008
I have a CSV file with date column in the following format mm/dd/yyyy (4/20/2012), and I need to load this column into a database that has datetime column in yyyy-mm-dd (2012-04-20).
I have used derived column transformation for this purpose and written and expression like
(DT_WSTR)(SUBSTRING(ReceivedDateTime,1,4) + "-" +
SUBSTRING(ReceivedDateTime,5,2) + "-" + SUBSTRING(ReceivedDateTime,7,2))
Upon running my package, it's throwing the error Unable to perform type cast.
If the incoming values from the CSV file are always formatted like 04/20/2012 (MM/DD/YYYY), then you don't have to perform any type casting. You just have to configure the flat file connection manager to treat the values in the file as date data type.
Let's assume that your CSV file looks something like this with a single column containing dates in .
In the SSIS package, create a flat file connection manager to read the CSV file. I stored the CSV in the path C:\temp\Source.csv
On the Advanced section, you will notice that the flat file connection manager named the first column as Column1 and the DataType property is set to string [DT_STR]. However, the values in the file are actually dates. Either you can manually configure the data types or click on the Suggest Types... button.
On the Suggest Column Types, leave the default values and click OK. This will read the first 100 rows of the file and will determine the column type according to the data available in the file.
Once you click OK on the Suggest Column Types dialog, you will notice that the data type on the flat file connection manager for the Column 0 has been changed to date [DT_Date]. Click OK to configure the flat file connection manager. You can also rename the column according to your requirements (say InvoiceDate or OrderDate etc.)
Now that you have the flat file connection manager configured, you can use it inside a Flat file source within a data flow task to read the data and populate your database. So, there is no need to manipulate values using a derived column transformation.
However, if your incoming file values are in string like 120420 (YYMMDD), these values cannot be configured as date data types. In these scenarios, you need to use Derived Column transformation as suggested in this answer.
Hope that helps.
If your incoming value is a string 20140106(YYYYMMDD) then typecasting the statement with (DDT_DBDATE) will throw an error. I had encountered this issue so I took out the typecast
and the mapping looked like this:
(LEN(TRIM(SUBSTRING([Screening Date],1,8))) > 0 ? (SUBSTRING([Screening Date],1,4) + "/" + SUBSTRING([Screening Date],5,2) + "/" + SUBSTRING([Screening Date],7,2)) : NULL(DT_WSTR,5))
Later I added a Data Conversion Transformation and changed it to database timestamp and it worked for me. You can also change it to type date(DT_DATE).
Thanks!
Try the following, ( I was successful in doing this)
(LEN(TRIM(SUBSTRING(ReceivedDateTime,1,8))) > 0 ? (DT_DBDATE)(SUBSTRING(ReceivedDateTime,1,4) + "-" + SUBSTRING(ReceivedDateTime,5,2) + "-" + SUBSTRING(ReceivedDateTime,7,2)) : (DT_DBDATE)NULL(DT_WSTR,5))
Just typecast the statement with (DT_DBDATE). And this will also serve if there is no date in that position i.e. if the position 1 to 8 of the received string is empty then it will put NULL.