Encoding issues when migrating from MSSQL SQL Server to (Linux) MySQL - mysql

I'm using MySQL Workbench and successfully migrating a SQL Server database used with SilverStripe PHP CMS to MySQL on Linux. Problem is when the content is displayed on the Linux web server I have to change encoding to Western (Windows-1252) to get the content to display correctly. The site on Windows IIS with SQL Server displays correctly with the default UTF-8 encoding.
In the manual migration editing section on MySQL Workbench some columns say Collation Latin1_General_CI_AS migrated to utf8_general_ci so I gather this is correct.
The site is setting <meta http-equiv="Content-type" content="text/html; charset=utf-8" /> in the HTML.
Migration process:
Use MySQL Workbeanch to migrate from SQL Server to MySQL on
production server
Export MySQL database on production server to *.sql file
Import *.sql file into Linux server using PHPMyAdmin and default UTF-8 encoding
I'm not sure where in the migration process I need to fix this?

This might not be about the migration process at all.
If you are using PHP to access the new database, the connection charset might be incorrect.
After setting up the connection you should set the connection charset to UTF-8.
$db = new MySQLi(HOST, USER, PASSWORD, DATABASE);
$db->set_charset('utf8');
Or, if you are not using MySQLi:
mysql_connect(/*...*/);
mysql_set_charset('utf8');

There are 3 possibilities of what could have gone wrong:
Encoding miss-configuration at the original SQL-server setup. Look at this post for more details on how this can happen: https://stackoverflow.com/a/20824533/684229. In this case, the encoding in the SQL-database is incorrect, but due to wrong setup it displays correctly.
You made a mistake in the migration process. Then the encoding in the new MySQL database is incorrect.
The encoding in the new MySQL database is correct, but there is an encoding miss-configuration at the new Linux MySQL setup, which makes it look incorrect.
To check which case applies, you have to check the encoding in both databases by some independent tool which for sure (200% at least!!!) has the encoding configured correctly. I would use PHPMyAdmin in case of Linux, I don't know what's available on SQL server. But make sure that this tool is configured correctly, otherwise you will get fooled!!!
Post the result and I will expand my answer accordingly.
EDIT: Dave, I have numbered the steps of your migration process. Please check the encoding of your MySQL database at two points - right after step 1 (before you do the export & import) as well as after you export & import in step 3. This will have to detect the exact point where it went wrong.

Related

Microsoft SQL DB to MySQL DB

I have seen tons of post on this, but I am not very familiar with the process and nothing has worked.
My basic problem is I am trying to get data from a Microsoft SQL database (Using Microsoft Management Studio 2008) to convert and use in a MySQL database (using MySQL Workbench)
I have tried to dump the .sql file but when I try to import into MySQL Workbench I get errors about it not being in the UTF-8 format. I tried several ways to get it to be that encoding (such as notepadd++ converting I saw suggested everywhere), but nothing seemed to work.
If I run it, I get this error:
ERROR: ASCII '\0' appeared in the statement, but this is not
allowed unless option --binary-mode is enabled and mysql is
run in non-interactive mode.
Again I looked up fixed for this, but could not get it.
I have tried a few options, just no success, so looking for some ideas or guidance with this area I do not have much experience in.
UPDATE:
The problem now is that the sql file I export from Management Studio is not accepted as a valid query in Workbench.
For example the brackets are not accepted and such.
I bet your encoding is not supported in MySQL WorkBench.
You can change the encoding when saving a .sql file.
Select File|Save.sql As to invoke the save as dialog.
Notice that the Save button on the lower right hand side has a drop down icon to indicate options.
Select the drop down icon and choose the "Save with Encoding" context memu item.
Select an encoding that works in MySQL Workbench.

Exporting/Importing database from/to phpMyAdmin - Character Encoding Issues

I've got a database that I want to export and import to another database on a different server, however after I've imported it I have character encoding issues i.e in some cases where there is a space in the html, a black diamond appears with a question mark inside.
I done the export a couple of months ago so can't remember if I simply imported the database via phpMyAdmin or used mysqldump! I'm going to re-import it today and wanted to know the best way to export and import. The HTML header on the website is utf-8, each database table has collation set to latin1_swedish_ci and the type is MyISAM.
The web server I am exporting from has the following settings:
MySQL charset: UTF-8 Unicode (utf8)
Apache/2.2.12 (Ubuntu)
MySQL client version: 5.1.37
PHP extension: mysqli
PhpMyAdmin Version information: 3.2.2.1deb1
The web server I am importing to has the following settings:
MySQL charset: UTF-8 Unicode (utf8)
Apache
MySQL client version: 5.5.24
PHP extension: mysqli
phpMyAdmin Version information: 3.4.10.1deb1
I would be grateful if someone can recommend the best way to export/import and if possible explain the reason behind the character encoding issues.
Thanks

What's the set up database on Heroku?

I have deployed my Rails 3.1 app with the MySQL database to Heroku and there everything works fine. I mean, into database are saved the chars right (seems to be used UTF charset on a databases on Heroku).
But when I will run the command heroku db:pull (this command will download a whole database from Heroku into the database on localhost), so the downloaded data stored in databased have bad coding - a chars are displayed bad (it looks like my local MySQL database have a different set up of charset than the MySQL on Heorku).
Could anyone give me a tip, how I can find the set up of charset used on Heroku database and how to use it on my local MySQL database?
Many thanks!
All is not lost - you really don't have to use PostgreSQL if you don't want to.
If your database is small enough (which it will have to be since the Heroku PostGres DB is also 5Mb) and you would prefer to remain on mySQL then you could use the ClearDB mySQL addon - http://addons.heroku.com/cleardb - their entry level DB is free and is the same size as the Heroku Shared PostGres DB that you get by default but be careful that the number of connections is limited so don't be going crazy with your web dyno counts.
Once you add the addon if you look at the output of heroku config then you can use the DATABASE_URL to create a connection in your favourite mySQL administration tool locally to restore/backup etc data to ClearDB. You may even find heroku db:push would work but personally I've not tried that so would be guessing.
The problem is that Heroku does not use a MySQL database in production, but a PostgreSQL database.
Therefore you will run into all sorts of issues pulling and pushing data from a different database engine. Taps is an activerecord based process that will reduce this problem but not all the time.
Ideally you want to use PostgresSQL on your development machine (install via Homebrew for simplicity on OSX) and you'll not see any more of these problems.
Alternatively, use one of the MySQL addons as described in the comments in the question.

Encoding problem in ASP.NET MVC + MySQL

I have some trouble showing right single quote from mysql database.
Screenshot:
This problem appears only when I deploy the application on a server.
If I start the application locally, connected to the database on the server, it's ok.
Any ideas?
Perhaps the encoding is set different on the server than your system. Check what the default encoding is on your system and then the server. The MySQL will just use what is set in the DB not the system.
http://msdn.microsoft.com/en-us/library/system.text.encoding.default.aspx

How to import a SQL Server .bak file into MySQL?

The title is self explanatory. Is there a way of directly doing such kind of importing?
The .BAK files from SQL server are in Microsoft Tape Format (MTF) ref: http://www.fpns.net/willy/msbackup.htm
The bak file will probably contain the LDF and MDF files that SQL server uses to store the database.
You will need to use SQL server to extract these. SQL Server Express is free and will do the job.
So, install SQL Server Express edition, and open the SQL Server Powershell. There execute sqlcmd -S <COMPUTERNAME>\SQLExpress (whilst logged in as administrator)
then issue the following command.
restore filelistonly from disk='c:\temp\mydbName-2009-09-29-v10.bak';
GO
This will list the contents of the backup - what you need is the first fields that tell you the logical names - one will be the actual database and the other the log file.
RESTORE DATABASE mydbName FROM disk='c:\temp\mydbName-2009-09-29-v10.bak'
WITH
MOVE 'mydbName' TO 'c:\temp\mydbName_data.mdf',
MOVE 'mydbName_log' TO 'c:\temp\mydbName_data.ldf';
GO
At this point you have extracted the database - then install Microsoft's "Sql Web Data Administrator". together with this export tool and you will have an SQL script that contains the database.
MySql have an application to import db from microsoft sql.
Steps:
Open MySql Workbench
Click on "Database Migration" (if it do not appear you have to install it from MySql update)
Follow the Migration Task List using the simple Wizard.
I did not manage to find a way to do it directly.
Instead I imported the bak file into SQL Server 2008 Express, and then used MySQL Migration Toolkit.
Worked like a charm!
In this problem, the answer is not updated in a timely. So it's happy to say that in 2020 Migrating to MsSQL into MySQL is that much easy. An online converter like RebaseData will do your job with one click. You can just upload your .bak file which is from MsSQL and convert it into .sql format which is readable to MySQL.
Additional note: This can not only convert your .bak files but also this site is for all types of Database migrations that you want.
Although my MySQL background is limited, I don't think you have much luck doing that. However, you should be able to migrate over all of your data by restoring the db to a MSSQL server, then creating a SSIS or DTS package to send your tables and data to the MySQL server.
hope this helps
I highly doubt it. You might want to use DTS/SSIS to do this as Levi says. One think that you might want to do is start the process without actually importing the data. Just do enough to get the basic table structures together. Then you are going to want to change around the resulting table structure, because whatever structure tat will likely be created will be shaky at best.
You might also have to take this a step further and create a staging area that takes in all the data first n a string (varchar) form. Then you can create a script that does validation and conversion to get it into the "real" database, because the two databases don't always work well together, especially when dealing with dates.
The method I used included part of Richard Harrison's method:
So, install SQL Server 2008 Express
edition,
This requires the download of the Web Platform Installer "wpilauncher_n.exe"
Once you have this installed click on the database selection ( you are also required to download Frameworks and Runtimes)
After instalation go to the windows command prompt and:
use sqlcmd -S \SQLExpress (whilst
logged in as administrator)
then issue the following command.
restore filelistonly from
disk='c:\temp\mydbName-2009-09-29-v10.bak';
GO This will list the contents of the
backup - what you need is the first
fields that tell you the logical names
- one will be the actual database and the other the log file.
RESTORE DATABASE mydbName FROM
disk='c:\temp\mydbName-2009-09-29-v10.bak' WITH MOVE 'mydbName' TO
'c:\temp\mydbName_data.mdf', MOVE
'mydbName_log' TO
'c:\temp\mydbName_data.ldf'; GO
I fired up Web Platform Installer and from the what's new tab I installed SQL Server Management Studio and browsed the db to make sure the data was there...
At that point i tried the tool included with MSSQL "SQL Import and Export Wizard" but the result of the csv dump only included the column names...
So instead I just exported results of queries like "select * from users" from the SQL Server Management Studio
SQL Server databases are very Microsoft proprietary. Two options I can think of are:
Dump the database in CSV, XML or similar format that you'd then load into MySQL.
Setup ODBC connection to MySQL and then using DTS transport the data. As Charles Graham has suggested, you may need to build the tables before doing this. But that's as easy as a cut and paste from SQL Enterprise Manager windows to the corresponding MySQL window.
For those attempting Richard's solution above, here are some additional information that might help navigate common errors:
1) When running restore filelistonly you may get Operating system error 5(Access is denied). If that's the case, open SQL Server Configuration Manager and change the login for SQLEXPRESS to a user that has local write privileges.
2) #"This will list the contents of the backup - what you need is the first fields that tell you the logical names" - if your file lists more than two headers you will need to also account for what to do with those files in the RESTORE DATABASE command. If you don't indicate what to do with files beyond the database and the log, the system will apparently try to use the attributes listed in the .bak file. Restoring a file from someone else's environment will produce a 'The path has invalid attributes. It needs to be a directory' (as the path in question doesn't exist on your machine).
Simply providing a MOVE statement resolves this problem.
In my case there was a third FTData type file. The MOVE command I added:
MOVE 'mydbName_log' TO 'c:\temp\mydbName_data.ldf',
MOVE 'sysft_...' TO 'c:\temp\other';
in my case I actually had to make a new directory for the third file. Initially I tried to send it to the same folder as the .mdf file but that produced a 'failed to initialize correctly' error on the third FTData file when I executed the restore.
The .bak file from SQL Server is specific to that database dialect, and not compatible with MySQL.
Try using etlalchemy to migrate your SQL Server database into MySQL. It is an open-sourced tool that I created to facilitate easy migrations between different RDBMS's.
Quick installation and examples are provided here on the github page, and a more detailed explanation of the project's origins can be found here.