I'm exporting data from a MS Dynamics CRM 2011 database table to a text file. Initially I was just exporting string data - i.e. columns defined as nvarchar, and it worked fine. However when I added the primary key column - uniqueidentifier/guid, SSIS just outputs unicode to the text file.
I'm explicitly casting all columns in my SQL query to VARCHAR, and have ensured all the source/destination columns are defined as non-unicode strings i.e. DT_STR, but still get unicode in the file.
Funnily enough this only happens when I run the SSIS package on our server which has SSIS 2008/Windows 2008 Server 64-bit. Our database is SQL 2008 64-bit as well, also on a seperate Windows 2008 server. The package works fine on my PC - Win7 64-bit.
Any ideas why this is happening?
You might try using the bcp utility for your export.
http://technet.microsoft.com/en-us/library/ms162802.aspx
“The bcp utility bulk copies data between an instance of Microsoft SQL Server and a data file in a user-specified format”
The –N argument is documented as “Performs the bulk-copy operation using the native (database) data types of the data for noncharacter data, and Unicode characters for character data.” Which implies that by omitting this arg you will not get Unicode in the output.
Related
We have a problem reading and loading some columns from AS/400 via OLE DB Provider for DB2 in SSIS 2015, the project is in SQL Server 2016.
We're migrating the project from an old server that is working with SQL Server 2008.
In the new one, when I preview the data in the Data Flow OLE DB Source, some columns have "System.Byte[]" as a value.
And I got the message "The metadata of the following output columns does not match thr metadata of the external columns with which the output columns are associated".
The external columns are read as DT_BYTES, while on the old server they were read as DT_STR.
The output columns are also expecting a DT_STR.
SSIS changed the metadata of the external columns, thus, making them unreadable.
I tried casting them as string, nothing works.
I tried setting the "ValidateExternalMetadata" to False, nothing.
The crazy thing is, with some other tables that have similar data types it reads them well, they have exactly the same data types as the failed ones.
There's no problem with the AS/400 itself as the old SSIS 2008 reads everything well.
So, I'm sure the problem is either with OL DB2 Provider, SSIS 2015 or SQL Server 2016.
I can't force it to read the external columns as STR with the Advanced Editor.
I also tried converting the data types in the output columns to DT_BYTES to match it, but still it didn't work.
It can't even read the columns from the beginning.
I tried a whole new package with no query, just reading the table as it is from the source, still the same problem.
I changed the Code Page to match the oiginal (1256), still nothing, then I changed it to match the new one (1252), still no success.
I'm really stuck and tried everything I could.
Any help please?
I finally solved it!
The solution is to open the connection manager, go to "All" tab, and match the Binary Codepage value (which is 0 by default) to the Host CCSID value.
I also set up the DBMS Platform to "DB2/AS400".
I am working in SQL Server 2008 and BIDS (SSIS). I have a simple data flow task in which I'm importing a flat file into an OLE DB destination. On the OLE DB destination, I'm getting an error message, stating "cannot convert between unicode and non-unicode string data types".
Now, I know one solution method is to put a data conversion in between the flat file and the OLE DB destination. However, my concern is why this is even happening in the first place. In the connection manager for the given flat file, all columns are string (DT_STR) data types, and the Unicode option is unchecked. Similarly, all columns in the destination table (upon the inspection of the metadata in SSMS) are varchar data types. So, there is no nvarchar-to-varchar mapping going on at all.
Why is SSIS thinking that I have a unicode-to-nonunicode mapping happening? And, is there an easier way to resolve this than inserting a data conversion step for the columns that have this problem?
[Since I don't seem to be allowed to comment on the question, I'm having to put my question here.]
Have you checked the table you're trying to insert the data into to see if the columns in the table are varchar or nvarchar? The SSIS metadata could be out of sync with the database table.
We have the plain Microsoft SQL Server 2008 on one of our servers
We decided to create DTSX files on the filesystem so that we can use BIDS 2008 to open the DTSX files
One SSIS Control Flow Components that takes data from around 18-19 columns from a Microsoft SQL Server 2008 SQL select query, and then converts the values in order to place them in the Microsoft Access table. I have a number of columns that I retrieve from Microsoft SQL Server 2008 table using a DataFlow Component called OLE DB Source. After retrieving the data, I have a Data Converter data Flow Component that converts the values. Finally, the converted values are placed by an OLE DB Destination data Flow Component, and placed in a Microsoft Access table.
For one of the columns from a Microsoft SQL Server 2008 , the data type is varchar(5)
The value in the above varchar(5) column needs to be extracted from Microsoft SQL Server 2008 using a SSIS DataFlow Component called OLE DB Source as a string [DT_STR] Data Type
It needs to converted using the Data Converter SSIS data Flow Component
Finally, it needs to placed in the Microsoft Access table to a column of the data type called Text.
What should the converted data type of the corresponding column within the Data Converter SSIS Data Flow Component be?
You can refer below resources.
Integration Services Data Type Mapping
Data Type Translations
Integration Services Data Types
#vikramsinh-shinde Thanks for the webpage links. The following implementation seemed to ensure that the SSIS package ran properly, however, I hope it's the proper way to handle conversion in the SSIS packages.
-varchar(5) column values in Microsoft SQL Server 2008
-extracted as string [DT_STR] that has a length of 5 and Codepage of 1252 within SSIS DataFlow Component called OLE DB Source
-converted to Unicode string [DT_WSTR] that has length of 5 ( and non-modifiable codePage of 0 ) within SSIS Data Flow Component called Data Converter
-loaded into Microsoft Access column values of data type called Text
I run SQL Server 2008 Express on my Windows 8.1. After inserting some Unicode characters in Database, characters are not displayed properly (the data has no problem just displayed improperly). Manually inserting the values using 'Management Studio' will lead to the same result.
I typed information in Persian Language and installed persial Language pack for Windows 8.1 however this does not fixed anything.
Am I missed something in SQL Server options? The same configuration on windows 7 have no problems.
Check if your field is UNICODE (nchar or nvarchar).
Check collation for your database.
Make sure your UNICODE literals have N prefix.
UPDATE: check this, possible duplicate question
Changing type of fields from VARCHAR to NVARCHAR fixes the problem but it does not recover data stored in bad format.
I want to convert a database from access to SQL Server using SSIS. I cannot convert the date/time columns of the access db. SSIS says something like:
conversion between DT_Date and DT_DBTIMESTAMP is not supported.
(Its translated from my German version, might be different in English version). In Access I have Date/Time column, in SQL Server I have datetime. In the dataflow chart of the SSIS I have a OLE DB source for the access db, an sql server target and a data conversion. In the data conversion I convert the columns to date[DT_DATE]. They are connected like this:
AccessDB -> conversion -> SQL DB
What am I doing wrong? How can I convert the Access date columns to SQL Server date columns?
Looks like you'll need to add a specific conversion to convert this column to the correct type. It's the DT_DBTIMESTAMP that you need to convert to, not DT_DATE. Using a Derived Column Transformation for instance, you could use an expression like:
(DT_DBTIMESTAMP)YourDateColumn
You use another Provider on the source side. I used the JET provider. I am doing this on a computer where no Access is installed. When I try to create a data connection using the Office 12.0 Access Database Engine OLE DB Provider I get an error message that the workgroup information file is missing. I get this message on the connection test.
The question is if the conversion depends on the db driver you use to get the data. I cannot install Access on the server I am working, so I cannot try it. Could this be the cause of the problem?
Casting D_Date column to DT_DBTIMESTAMP solve the same problem for me.