If I have a column of type DT_DBDATE or DT_DBTIMESTAMP, and I write the data to a file using a Flat File Destination component, the resulting date formatting will YYYY-MM-DD HH:MM:SS. I've experimented with changing the package locale and changing the datatype in the Flat File Connection Manager, but these don't seem to have any effect.
What determines this format and is there any way I can change the default formatting?
PS. I know I can use Data Conversion and Derived Column components to change the data types/formats. The point is we have +150 packages and get too many errors when someone forgets to do the conversion or does it incorrectly.
We're using SSIS 2012.
I had a similar issue while using SSIS 2008 with SQL Server 2008 R2. If you are trying to generate a CSV file, I would recommend first, at the database level, to store the value in the desired format. This means, for instance, if you want to display a column where the current date is in the format YYYY-MM-DD, then you should use the appropriate code to store that value. Using T-SQL will be: convert(varchar(10), cast(GETDATE() as date), 126). Double check that the value was stored in the desired format by reading the table. Now it comes the tricky part: no matter what you are using in SSIS for transforming the dates, if you open the CSV using Excel, then Excel is going to display the dates using the format was set for the operating system. Don't panic, just don't save that file when you want to close it. Instead, if you open the CSV using Notepad or Notepad++, you should be able to see the date columns right in the format you used for storing them in the database.
Related
I'm inserting a row coming from a CSV file into my SQL database. The birthdate column that I have on the CSV file has a date format like this 10/29/1992, it seems that the Google Sheets that I use to create the CSV file convert it to that format even though I input it like this 1992-10-29, since the latter format is the format in the SQL database for dates.
So I tried importing the CSV file to my table in the database, and it won't insert the birthdate column correctly, it inserts 0000-00-00 and gives me a warning Mysql date warning data truncated. So it seems it's impossible to insert in this format 10/29/1992
What could be the best way to solve this? Without handling this on the backend or server side? Can this be handled in the phpMyAdmin settings or in google sheets so that it won't convert the date I input to a different format?
I just found out that in Google Sheets where I create the CSV file, there's a setting where you can custom format your dates for a date column. I guess that's all I needed so that the date format from the CSV to the MySQL table will match. The setting can be found in Format-> Numbers-> Custom date and time
I'm trying to import data from a MySQL database into an Excel workbook via an ODBC Data Source.
I have a big problem with the time(3) fields from MySQL. Excel imports the content not as expected. As first try, Excel cut the milliseconds off. Then I edited the query at the specific fields to:
DATE_FORMAT(jobstat_0.PreProcessingQueued,'%T,%f')
This puts something like:
00:00:00,156000
That's what I expected, but I can not change the format in Excel.
Excel interprets the content as text, and if I change to the specific time format, Excel does not care...
I need to correct that to get Pivot Tables working. As a result there are Divide By Zero errors in my Pivot Table.
May be someone had something like that?
I am very new to SSIS development. Please help me
I have a ragged right flat file with no column names, which I need to import to sql server database.
The problem is, when importing these file using flat file connection manager, all the columns are stored as varchar. I have different columns like Amount (which should be a decimal data type), percentage (which should be a decimal data type), Date (which should be a date data type mm/dd/yyyy format).
For example, The amount field is loaded as 000012347834 . I need to change it to 123478.34
percentage field is loaded as 03246. I need it as 03.246 .
How can I do these conversions using SSIS.
Thank you in advance for your valuable time and help.
You can use the advanced editor on your connection manager to change the data types.
You can also use a data conversion task which is more along the lines of what SSIS is designed for... Transform is the T in ETL.
However most often if I have a lot of conversions to do on a text file import I will load into a staging table with all varchar and use a stored procedure to do the conversions.
There is a flat file with dates in YYYYMMDD format, e.g. 19990131. I'm reading it with Flat File Source, column has default setting (DT_STR, width 50)... When I try to insert data via OLE DB Destination to a table with Date column I got the error: "The value could not be converted because of a potential loss of data.".
What is the cause of it? SQLServer easily understands YYYYMMDD format...
SQL Server may understand it, but SSIS doesn't. Note this comment from the SSIS documentation:
When a string is cast to a DT_DATE, or vice versa, the locale of the
transformation is used. However, the date is in the ISO format of
YYYY-MM-DD, regardless of whether the locale preference uses the ISO
format.
SSIS is trying to interpret YYYYMMDD based on the locale of the flat file connector, which may work for some locales but apparently not for yours. As you discovered, changing it to YYYY-MM-DD works, because SSIS considers that format to be unambiguous (you could also change the locale of the flat file connector, of course, although that might affect other data).
Unfortunately, this behaviour is the exact opposite in SQL Server itself, at least for the datetime data type: YYYYMMDD is unambiguous and YYYY-MM-DD is ambiguous. On the other hand, for the newer date data type both YYYYMMDD and YYYY-MM-DD are unambiguous in SQL Server.
If I change YYYYMMDD to YYYY-MM-DD before putting data into table everything works fine. I will accept this as an answer if nobody gives another, better solution. Maybe mine is prone to localization errors?
First I'd like to say that I'm brand new to SSIS so bear with me if this is a very basic question. I've searched and cannot find an answer.
I need to read data from SQL Server that is stored in a varbinary column that contains an excel document. I then need to store this data into another table with the appropriate columns (pre-defined format).
My question is essentially... How do I read this varbinary data into something I can work with and then insert into another table?
You could use Export Column Transformation available within the Data Flow Task to read the varbinary data and then save it as a file on local disk where the SSIS package is running.
MSDN documentation about Export Column transformation.
Sample: The Export Column Transformation on BI Monkey
Using another data flow task, you can read the saved file and import the data into the table of your choice.