I am reading data from a flat file and storing them in a OLE DB destination.
In the flat file, dates and in various formats
17/02/2014,
28-Apr-14,
30.06.14
I have used a Derived column transformation to check for empty columns and replace it with null. As far as I have seen SSIS and data base accepts 17/02/2014 format and 28-Apr-14, 30.06.14 are rejected.
I want to convert 28-Apr-14 and 30.06.14 to a valid format which DB accepts.
I have researched a bit and read that Script task can do it but I am not sure of the code how to check this.
Could you please guide which is the best way to do this.
Any suggestions/help is much appreciated.
Thanks
Rao
Check out my answer on the following question. This should work for you as well.
SSIS clean up date from csv file
Related
I have a flat file with the below value:
AI9983|01|20170802|119965.57|0.000000|0.000000|||
AI9983|01|20170803|119713.62|-0.210019|-0.210019|||
I read the file, the value 119965.57 gets stored as 119965.60
the value 119713.62 gets stored as 119713.60.
When reading the flat-file the variable is set as float[DT_R4]
It gets stored in the database as decimal(28,2).
Can someone help me to sort this out, I need to store the value as it is in the flat file into the database.
I am not using any derived column or transformation here.
Float will not return a reliable result, so it is challenging to use in database work.
In the flat file connection manager, go to the columns tab, select the column and change the data type to numeric with a DataPrecision of 28 and Scale of 6 (Based on the sample data provided in the question).
Hi all quick question for you.
I have an SSIS2012 package that is reading a flat file (.csv) and is loading it into a SQL Server database table. However, I am getting an error for one of the columns when loading the OLEDB Destination:
[Flat File Source [32]] Error: Data conversion failed. The data conversion for column "Active_Flag" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
I am wondering if this is because in the flat file (which is comma delimited), the values are literally spelled out "TRUE" or "FALSE". The advanced page on the flat file properties has it set to "DT_BOOL" which I thought was right. It was on DT_STRING originally, and that wasn't working either.
In the SQL server table the column is set up as a bit, and allows nulls. Is this because it is literally typed out TRUE/FALSE? What's the easiest way to fix this?
Thanks for any advice!
It actually turned out there was a blank space in front of "True"/"False" in the file. Was just bad data and I missed it. Fixing that solved my issue. Thank you though, I did try that and when that didn't work that's when I knew it was something else.
I am downloading CSV files which are comma-separated. The problem i'm having is that the commas are screwing-up my import into a database table (SQL Server). For example, I have a header row called hotel_name, but some of the names are like the following:
HOTEL_NAME
hilton
cambridge,the
The problem is that fields containing a comma in the hotel name will move to the adjacent column, like this I'm wondering if converting from CSV to a pipe-delimited format will work.
The problem i'm having is that i'm not sure how to get started. I've tried following the Powershell documentation but get basic errors. I think this is because i'm new to Powershell and not understanding something. Can someone please post a script of how to change the comma-separated file to a pipe-delimited file?
Sorry if this is confusing, i'm finding the formatting on StackOverflow to be a bit crazy.
Taken from Dealing with commas in a CSV file
Use " to wrap data that contains a comma.
For example
Server000,"Microsoft(R) Windows(R) Server 2003, Enterprise Edition"
I need to export varbinary data to file. But, when I do it using Column Transformations in SSIS, the exported files are corrupt. There are few junk characters at the start of the file. On removing them, the file opens fine.
A similar post for BCP, says that these characters specify the data length.
Would like to know how to address this issue in SSIS?
Thanks
Export transformation is used for converting the varbinary to files.I have tried something similar using Adventure works which has image type of var-binary data.
Following Query is used for the Source query. I have Modified the query
since it does not have the full path to write image files.
SELECT [ProductPhotoID]
,[ThumbNailPhoto]
,'D:\SSISTesting\ThumnailPhotos\'+[ThumbnailPhotoFileName]
,[LargePhoto]
,'D:\SSISTesting\LargePhotos\'+[LargePhotoFileName]
,[ModifiedDate]
FROM [Production].[ProductPhoto]
Used the Export column transformation[also available in 2005 and
2008] and configured as follows.
Mapped rest of the columns to the destination.
After running package all the image files are written into the
respective folders[D:\SSISTesting\ThumnailPhotos\ and D:\SSISTesting\LargePhotos].
Hope this helps!
How to use CAST or CONVERT in Excel Query ie in Excel Source Task. I wanted to conert a number to Text in the Excel Source Task itself using SQL Command option.
Best option is to use Data conversion between source and destination.
As in Excel source or destination connection manager we can CAST or CONVERT as in SQL.
Please add data trasformation task like below:
As like the Excel destination it can be any destination after transformation
There are many problems connected with excel files and data types. Sometimes it's very painful.
Check this out:
http://social.msdn.microsoft.com/forums/en-US/adodotnetdataproviders/thread/355afd19-8128-4673-a9d1-9a5658b72e84
or this:
http://www.connectionstrings.com/excel
When I use xls files I often convert everything to text: IMEX=1 and then I use Data Conversion Task.
So I suggest to change your connection string to:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\MyExcel.xls;Extended Properties="Excel 8.0;HDR=Yes;IMEX=1";
And then you can do everything you want with data in SSIS.