(SQL Server 2008)
So here's my task ..
I need to export query results to file, and then import that file using SSIS to another DB.
Specific to the task, the data contains every awkward unicode character you can think of, so delimiting with commas, pipes etc is out of the question.
Here are the options SSMS gives me for export format:
Column Aligned
Comma/Tab/Space delimited
Custom delimiter
And here are the options SSIS gives me for a flat file data source:
Delimited (custom)
Fixed Width
Ragged Right
So given that a delimiter character is out of the question ... I cannot see another method that both SSMS & SSIS agree on.
Such as fixed width ?
Seems strange that the 2 closely related MS products have such different options.
Or have I missed something here ?
Any advice appreciated !!
It seems you need to try out different combination of options while creating delimited flat file(for your exported query result).
Try setting Code page to UTF-8 with and without Unicode. Also use Text qualifier as " or any of your choice which you thought might work. Also try using different option for column delimiter.
Once you are able to create delimited file then you have to apply same setting on file while importing to another DB.
Related
I've been given a mySQL database from a custom-coded CMS and I need to get its data into a CSV file for importing into Excel for further futzing.
The problem is that the data in the database has a lot of HTML code in it (<p class="foo"> and that type of thing), so exporting as a CSV gets screwed up as some of the text has commas and other control characters in it.
Looked at all the export options via phpMyAdmin but couldn't really find anything that would work.
How can I get this into Excel?
Try tu use MySQL Workbench and there's excel xml export format, it should work also with html inside.
Actually, by searching here I found the answer tho had to take a bit of a different approach.
I used \t as the separator (tab character), Columns enclosed with ~, Columns escaped by \, put column names in the first row.
I had to use LibreOffice to import the sheet as Excel didn't have the flexibility needed. Got it working now!
I am trying to import a rather large (520k rows) .CSV file into a SQL Server 2012 table. The file uses a delimiter of ;.
Please do not edit my delimiter. It is ";" I know that may seem strange, but that is what they used. It is not just a semicolon.
I don't think the delimiter is the issue because I replaced it with a tab and it seemed to be okay. When I try importing the file, I get a text truncation error, but I set the column to 255 just to be sure it had plenty of room.
Even when I delete the row, the next row causes the error. I don't see any offending characters in the data, so I am at a loss as to what the issue is.
I ended up using the EOL Conversion and selected Windows format in Notepad++ and then created a script to import the data.
I'm creating SSIS package for to get .csv file to my local server and transfer it to FTP
When I get my csv into FTP and open into excel, My data getting shift over to other columns. Is there internally any kind set up do I need to change?
Also I tried different text qualifier still did not work.
It sounds like there may be hidden characters in your data set. If you are using comma's you may want to consider using a lesser used character for the delimiter such as a pipe "|". For instance an address may naturally have comma's. If a pipe shows up in an address field it's probably a type-o, and is far less likely. Things that shift data cells are often things like tab characters and CRLF. You can also open your data set in a text editor like notepad ++ and choose the "Show all Characters" option under "View->Show Symbols" menu option to see what the exact character is. If it's rampant in your data set you can use the replace function within the Derived Column Task to scrub the data as it comes out of the data source.
I need to export varbinary data to file. But, when I do it using Column Transformations in SSIS, the exported files are corrupt. There are few junk characters at the start of the file. On removing them, the file opens fine.
A similar post for BCP, says that these characters specify the data length.
Would like to know how to address this issue in SSIS?
Thanks
Export transformation is used for converting the varbinary to files.I have tried something similar using Adventure works which has image type of var-binary data.
Following Query is used for the Source query. I have Modified the query
since it does not have the full path to write image files.
SELECT [ProductPhotoID]
,[ThumbNailPhoto]
,'D:\SSISTesting\ThumnailPhotos\'+[ThumbnailPhotoFileName]
,[LargePhoto]
,'D:\SSISTesting\LargePhotos\'+[LargePhotoFileName]
,[ModifiedDate]
FROM [Production].[ProductPhoto]
Used the Export column transformation[also available in 2005 and
2008] and configured as follows.
Mapped rest of the columns to the destination.
After running package all the image files are written into the
respective folders[D:\SSISTesting\ThumnailPhotos\ and D:\SSISTesting\LargePhotos].
Hope this helps!
We have an SSIS package which reads from a DB, creates a flat file from that info, and drops it to a file server.
I recently made an update to the package's query which is used against the DB, adjusted the column mappings, and placed it under the SQL Job which ran the SSIS package before.
The problem is that the text qualifier in the flat file should be a quotation mark: ". But when I checked the flat file it produced, the text qualifier showing is: _x0022_
I investigated the Text Qualifier property for the DestinationConnectionFlatFile, and it is set to a quotation mark: "
How can I ensure the flat file will have a text qualifier of quotation mark?
Here is a previous answer I found when this happened to me:
SSIS exporting data to flat file renders double quotes as hexadecimal characters
Additionally ,
This issue occures because of installation issue. So if you see this sort of Issue, It mean if you are loading from file Database table and file contains 100 records , then instead of 100 records only 99 records would get loaded to database , last records would get skipped.
I had same issue, to fix that I had re-instalation of
1) MS Visual Studio
2) MS BI Studio
in the sequence mentioned above.
Given below are the two solutions :
Solution 1: Open the Package in Notepad and Edit the Value present in the "TextQualifier" of particular object to """.
object Name
"
Solution 2: open the Package and Replace the Value in the "TextQualifier" of the Flat File Connection Managers (FFD,SRC,SOURCE) to "\"".
Solution 1: Open the Package in Notepad and Edit the Value present in the "TextQualifier" of particular object to (") " follow semicolon
object Name
"
Thanks,
Prakash.A