When migrating from SQL Server to MySQL, I'm having the error "Incorrect String Value" when the string has any Portuguese specific character (like in "Olá", which means hello).
The problem is that this occur in same tables, but not on others. I have a tableA which have this characters, but imports correctly, but another tableB gives me the error. Both tables have the same codepage.
I'm using the script generated my MySQL Migration Tool, which calls wbcopytables.exe.
Does anyone has any clue on what could be the problem?
Related
I have a .NET 6 application that is currently backed by MSSQL database, that is maintained by Entity Framework using a Model First approach. I am trying to migrate it to use a MySQL database backend, for a variety of reasons.
I have installed MySQL Locally (Windows) to start exploring and getting it working. I can migrate the schema easily enough (With either MySQL Workbench or using EF) but migrating the data is proving to be a little tricky.
Around half of the tables migrated fine, but the other half, relating to string data, are failing due to errors which look a little like this - the column obviously differs from table to table. The source data is nvarchar in SQL, and the destination is type `varchar'
Statement execution failed: Incorrect string value: '\xF0\x9F\x8E\xB1' for column 'AwayNote'
Does anyone know how I can get the Migration to run successfully?
The research I have read has said to ensure server and table character sets are aligned as per the below.
I have set up my Source as SQL using the ODBC FreeTDS
The data import screen is set up like this - the check box doesn;t seem to affect things especially.
I have MySQL setup with this too, which I have also read is important.
[mysql]
default-character-set = utf8mb4
Final Update
I was able to easily migrate the data with Talend. No errors, and it worked perfectly the first time with no special settings. This shows what an utter piece of garbage the MySQL Workbench Migration tool is. While the learning curve of Talend is rough (it's not intuitive at all), it appears to be one of the best data migration solutions out there. I recommend using it. Note I never figured out why the migration failed (as seen below). I'm just walking away from the utter garbage Oracle has pushed on the community. Oh, and Talend migrated the data to utf8mb4/utf8_general_ci without a hitch.
Please note there are updates at the bottom.
We have to migrate an export from TrackerRMS (which luckily doesn't have FK constraints, but the data is a total mess) to MySQL. Restoring the backup of the TrackerRMS data to SQL Server is cake; no issues. The problem is copying the data from SQL Server to MySQL.
MySQL Workbench Migration can handle all but 4 of the tables; but those 4 tables are the key problem. They have crazy content in their fields which causes the migration tool to choke. I attempted to export the data as .sql from HeidiSQL and it chokes as well.
The source table problem fields are NVARCHAR(MAX) and SQL_Latin1_General_CP1_CI_AS collation.
Note I've tried changing the collation of the source SQL Server table columns to Latin1_General_100_BIN2_UTF8 and Latin1_General_100_CI_AI_SC_UTF8 and there is no effect.
The errors are:
ERROR: `Backup_EmpowerAssociates`.`BACKUP_documents`:Inserting Data: Incorrect string value: '\xF0\x9F\x93\x8A x...' for column 'filepath' at row 13
ERROR: `Backup_EmpowerAssociates`.`BACKUP_activities`:Inserting Data: Incorrect string value: '\xF0\x9F\x91\x80' for column 'subject' at row 42
ERROR: `Backup_EmpowerAssociates`.`BACKUP_resourcehistory`:Inserting Data: Incorrect string value: '\xF0\x9D\x91\x82(\xF0...' for column 'jobdescription' at row 80
This tells me the source data has 4-byte character details (which is beyond the standard utf8). Note the destination database in MySQL is utf8mb4 and utf8mb4_unicode_ci collated, and has the default settings as such. No connection settings override this.
When migrating I use Microsoft SQL Server and ODBC (native) for localhost (SQL Server) with default options. I've also tried turning ANSI off, but it has no impact. Note the ODBC configuration for SQL Server has no charset or collation settings or options. For target, I use the localhost stored connection which I use for general access.
Note the MySQL Workbench migration tool defines the receiving table columns (for the above problem columns) as LONGTEXT CHARACTER SET 'utf8mb4'.
Could the issue be the migration proxy (ODBC?) is somehow converting it to utf8 (even though I don't have that selected)? But if that was the case, wouldn't the incoming data not be erroring out in the migration process as a UTF8MB4 solution (4-byte vs less)?
Note I tried creating and adjusting the destination MySQL table (by adjusting the SQL in the migration tool) as CHARSET latin1 and latin1_general_ci collation. Same issue.
Migration simply does not want to work (this is with SQL Server source being SQL_Latin1_General_CP1_CI_AS). And I've tried it with UTF8 both on and off for driver. No effect.
Does anyone with migration experience recognize this issue, or have recommendations on how to resolve the problem? I'm fine with scrubbing the source data in SQL Server before I migrate - I just don't know the best method to do that (or if it's necessary).
Thanks!
===
UPDATE 1
This is very strange; using the below technique to show values that won't convert, this is the result:
SELECT filepath, CONVERT(varchar,filepath) FROM BACKUP_documents WHERE filepath <> CONVERT(varchar, Filepath);
Why on earth is the data being truncated upon convert with a simple filename at the "c" in documents?
Here's a capture that might also help resolve this issue.
But the strange part is MSSQL is showing normal text (without special characters) as being non-ASCII. I'm wondering if the folks at TrackerRMS are running code written in another country/language and it's messing up the data, but it's something that's not visible?
UPDATE 2
So to make things clear, here's what one of the characters that is messing up the data looks like.
I was able to easily migrate the data with Talend. No errors, and it worked perfectly the first time with no special settings. This shows what an utter piece of garbage the MySQL Workbench Migration tool is. While the learning curve of Talend is rough (it's not intuitive at all), it appears to be one of the best data migration solutions out there. I recommend using it. Note I never figured out why the migration failed (as seen below). I'm just walking away from the utter garbage Oracle has pushed on the community. Oh, and Talend migrated the data to utf8mb4/utf8_general_ci without a hitch.
SQL Server Migration Assistant for MySQL (Chinese display error)
I am using SQL Server Migration Assistant for MySQL to migrate from Mysql to SQL Server
After the migration data is completed Chinese display ?? Error display!
迁移工具
Our data is available in four languages
zh_TW
zh_CN
pt_PT
en
I finally solved the problem completely
When I link SQL server data, I set charset = utf8, but there are still individual displays in Chinese Simplified "?"
At last, we set the field type to nvarchar (max). At first, I knew that modifying the field type could solve the problem, but we were migrating data and could not modify the field type. Using symfony schema update to generate SQL statements also did not support nvarchar (max). At last, we modified the source code of generating SQL statements. Using text type, we could compile nvarchar (max)
enter image description here
I am using MySQL 5.1.5 on a Yahoo Sight Server, with phpMyAdmin as the databse interface.
I use the following Query
UPDATE table_family SET last_name='Smith' WHERE id=1;
Then I get the following error:
Error There seems to be an error in your SQL query. The MySQL server
error output below, if there is any, may also help you in diagnosing
the problem
ERROR: Unknown Punctuation String # 34 STR: =\ SQL: UPDATE
table_family SET last_name=\'Smith\' WHERE id=1;
SQL query:
UPDATE table_family SET last_name=\'Smith\' WHERE id=1;
MySQL said: Documentation
1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use
near '\'Smith\' WHERE id=1' at line 1
Ideas? I feel like its an issue with the database, not with my code. the last_name field is a varchar(50). I actually opened a ticket with support on this issue, but it happened to me on two different domains I have with Yahoo, so that makes me think there is more than I know is going on. I have done this with integer fields that don't require the (') single quote and have had no issues. I have also run my syntax through a local access database I created just to make sure it wan't a syntax issue. Worked perfectly first time. Then I had 2 database guys I know look at it. They think its good too. So now I am lost.
Thanks for any help you can provide.
Andy
I am trying to set up a third party MySQL as a Linked Server on SQL Server 2008.
When I try to access the data via a simple OpenQuery below I get an error.
SELECT * FROM OpenQuery(SERVERNAME, 'SELECT * FROM table')
The error message:
OLE DB provider 'MSDASQL' for linked server 'SERVERNAME' returned data that does not match expected data length for column '[MSDASQL].tablename'. The (maximum) expected data length is X, while the returned data length is Y.
NOTE:(where X > Y)
I've tried with another MySQL table that I built myself and it works fine. Therefore I assume the problem might be with the MySQL source.
EDIT: After digging a little deeper into the MySQL data I found several "invalid" dates such as zero dates. These could be a reason, however I have no way to change the data source so I must find a way to convince SQL to ignore it.
I recall something form previous versions of SQL where you could switch to "Lazy Schema Validation" to force SQL not to check this. However this option appears to be gone in 2008.
Any ideas how I could make this work?