I'm pretty new to SSIS and I'm running into an issue where I'm using a sql statement in an OLE DB Source step to pull my data and this data contains 5 DATETIME fields that need to be converted and all of the data is going into a flat file.
I'm doing the conversions in sql like so:
REPLACE(CONVERT(VARCHAR(10), birthdate, 101), '/', '') AS 'Date of Birth (MMDDYYYY)
In the SSIS Flat File Connection Manager my DataType is string [DT_STR] and my OutputColumnWidth is 8. This is working in SSIS without truncation as the above conversion creates an 8 character varchar. If I try converting to a varchar(8) it truncates the data in sql server management studio.
My problem is that my next conversion attempt is failing. Here's the sql code:
REPLACE(CONVERT(VARCHAR(10), exp_date, 103), '/', '') AS 'Expiration Date
I get the following error: "The value could not be converted because of a potential loss of data" even though everything has been reproduced with the same settings. This conversion converts the date to a 'DDMMYYYY' format.
I know from researching that this is a data conversion error but I have no idea how to get around it especially with the first conversion working correctly.
The interesting thing is that when I redirect the rows on error to a different flat file the dates get pulled and converted with no problem at all.
I'm at a loss here and would appreciate any help from the group.
I figured out why it wasn't working. When I first created the package only the birthdate was converted since that's where my attention was due to the different date format. After I ran it and verified that all of the data was being pulled I realized that I needed to convert the other 4 dates as well. The problem is that even though I made the change in sql and in the Flat File Connection Manager the metadata was still set to the original settings for those 4 fields. I recreated the step and everything converted correctly. Thanks for taking the time to respond
Related
I've created a table in SQL Server 2016 with this definition:
CREATE TABLE Stage.Test (f1 DATE NULL);
INSERT INTO Stage.Test (f1) VALUES ('1/1/2000');
Notice the f1 column uses the DATE data type.
Then I created a data flow task in SQL Server Data Tools (VS 2019) to load data from that table. I created an OLEDB Source Component and set the source table to Stage.Test. But when I examine the data type of the "f1" column (in both the 'External Column' and 'Output Column' columns), it says it's a Unicode string:
Why is it choosing a Unicode string instead of DT_DATE?
I haven't seen this exact example when it comes to date fields but SSIS converts data automatically when it has detected the field to be of a certain type. Perhaps it's the '/' in your date field that does it. We do not use this date format over in these parts of the world so I've never had the problem. You can especially see this when you import excel files with SSIS. I usually have this problem with strings where unicode strings can sometimes become non-unicode strings.
A way to fix could be to:
edit the sql query to explicitly cast the field as date
add a conversion step in the data flow after the source (like a derived column getting the parts of the string in the right order)
try to change the output datatype by right clicking on the source in the data flow and using the advanced editor and then edit the output column datatype:
SSIS Source Advanced Editor Output
I'm not sure if that 3 would work with this date format issue as I do not have experience with the format myself but it is working fine for my unicode/non-unicode problem.
I have an SSIS package which has been working for years, and all of a sudden it has stopped working, with the error:
An exception has occurred during data insertion, the message returned from the provider is: The given value of type String from the data source cannot be converted to type nvarchar of the specified target column.
All of my strings are already converted to DT_WSTR in derived columns. Something must have changed in my input flat file format, because it still works on files produced a week ago. I've analysed the difference between the working version and the new failed version in beyond compare, and there are no differences in format or encoding. It has 100+ columns, and with no indication of which column is causing the problem, it is very hard to diagnose.
Any ideas? Thanks
Haha, actually, not all the columns were being case to DT_WSTR. Once I made sure every single field was being cast to DT_WSTR of the appropriate length, the import succeeded.
Hi all quick question for you.
I have an SSIS2012 package that is reading a flat file (.csv) and is loading it into a SQL Server database table. However, I am getting an error for one of the columns when loading the OLEDB Destination:
[Flat File Source [32]] Error: Data conversion failed. The data conversion for column "Active_Flag" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
I am wondering if this is because in the flat file (which is comma delimited), the values are literally spelled out "TRUE" or "FALSE". The advanced page on the flat file properties has it set to "DT_BOOL" which I thought was right. It was on DT_STRING originally, and that wasn't working either.
In the SQL server table the column is set up as a bit, and allows nulls. Is this because it is literally typed out TRUE/FALSE? What's the easiest way to fix this?
Thanks for any advice!
It actually turned out there was a blank space in front of "True"/"False" in the file. Was just bad data and I missed it. Fixing that solved my issue. Thank you though, I did try that and when that didn't work that's when I knew it was something else.
I'm trying and playing around a CSV file to import data into the SQL Server table using SSIS.
The package is simple with File Source Task and SQL Server Destination.
The CSV file has 2 fields Transaction_Date and Account_Created. The dates in these fields are the format of 1/2/2009 6:00:00 AM. I am seeing the below error message:
"Error: An exception has occurred during data insertion, the message returned from the provider is: The given value of type String from the data source cannot be converted to type datetime of the specified target column."
Steps I tried below:
I tried using various other destination transformations.
I tried playing around the Data Types inside the Flatfile Connection Managers.
I tried using the Data Conversion Transformations between the Source Task and SQL Server Destination.
When I tried to load the data by providing connection only to Transaction_Date it works. However, when I tried to load by providing connection only to Account_Created it fails.
I am sure I'm missing something silly. Please help.
Regards,
KK
I tired a different method to build the package from start using the Wizard. I used the actual CSV file which had many other columns like Price, Product_name and so on. When I tried to execute the package I see a different error message as below:
"[Destination for AdventureWorks [58]] Error: There was an error with input column "Price" (91) on input "OLE DB Destination Input" (71). The column status returned was: "The value could not be converted because of a potential loss of data.".
"
When I tried a CSV file with only 2 date fields it worked excellent.
I am really puzzled now and thinking this is some kind of data type issues which I am not getting it correct. Can some one pls shred some light into this problem?
Regards,
KK
To load the first two fields (Transaction_Date, Account_Created), you need a DataFlow Task that contains:
Flatfile Source
Derived Column (create two columns to Replace 'Transaction_Date' and 'Account_Created' with formula below)
SQL Server Destination
Notes:
Date format like '1/2/2009 6:00:00 AM' is not parsed by SSIS, make sure the Flatfile Connection Manager treats the fields as Strings (length > 22)
In Derived Column, you can parse '01/02/2009' with this formula:
(DT_DBDATE)(SUBSTRING([Column 2],7,4) + "-" + SUBSTRING([Column 2],4,2) + "-" + SUBSTRING([Column 2],1,2))
The current date format that you have in the file '1/2/2009' makes the conversion very tricky due to the lack of advanced datetime parsing functions of SSIS. Depending on the day and month you will have to subtract from a variable length string, therefore you will have to combine SUBSTRING with FINDSTRING to determine the position of the separator '/'
Good luck
my date format is in this format YYYYMMDD and when I am converting the same with data conversion task I am getting below error:
The data value cannot be converted for reasons other than sign mismatch or Data Overflow.
In my Dataconversion I have select DT_DATE
and in database the column datatype is date.
But the strange thing is that when I am executing my package and doing casting as source SELECT CAST(myDate AS DATE) package is working fine.
It's a common issue. If you use a derived column transformation, you will need to slice it out into the component parts (years, months, days) and then concatenate it back together before casting. That's ugly and time consuming for me.
Instead, assuming this is coming from a flat file, just make it a date on import by setting the type in your connection manager to the date type that will be compatible with your destination. Then on your flat file source, under advanced settings, set FastParse to true for that column. See my answer on Import String Date in Derived column for a pictoral walkthrough of it. Also addressed it on SSIS importing datetime column into SQL Server 2008