I want to convert our production database from latin1 to utf8.
we are using amazon rds mysql.
Please provide step by step. Will there be any downtime?
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
i use above query to convert each table.
Is it good a way? I need to do one by one or is there any other way to do in one step?
Change in PRODUCT server is always crucial. Lot of consideration need to take in concern before taking the final decision. The first question is -
is this really a show-stopper? or A dead lock situation? or Performance issue? If with all consideration you are decided that this is a must, some caution need to
perform to achieve the changes-
Step-1: Take a full database backup.
Step-2: Make sure the backup is restore-able. Make several copy of backup and preserve in different Server. This will help restore your OLD data in case of any accidental data lose.
Step-3: Make necessary changes to Development server first. Check your application performing as before without any issue specially where there are data access exist for the changed area.
Step-4: Check all database objects like (SP,FUNCTION) are using that table are still performing as expected.
Step-5: Better if you can engage some QA resource before make changes in LIVE environment.
Step-6: If all above steps goes fine, you can go for LIVE changes.
Step-7: Engage QA resource again to make sure LIVE changes are also applied without any issue.
Note: No significant downtime is required. But always it is best if you can keep stop database access from your application during the database changes. This make sure no data will hamper during insert/edit/delete data from application.
Related
I hope this is not off-topic, but I have a real problem that I oculd use some advice on.
I have an application that upgrades its own Sql Server database (from previous versions) on startup. Normally this works well, but a new version has to alter several nvarchar column widths.
On live databases with large amount of data in the table this is taking a very long time. There appear to be two problems - one is that Sql Server seems to be processing the data (possibly rewriting it), even though that isn't actually being changed, and the other is that the transaction log gobbles up a huge amount of space.
Is there any way to circumvent this issue? It's only a plain Alter Table... Alter Column command, changing nvarchar(x) to nvarchar(x+n), nothing fancy, but it is causing an 'issue' and much dissatisfaction in the field. If there was a way of changing the column width without processing the existing data, and somehow suppressing the transaction log stuff, that would be handy.
It doesn't seem to be a problem with Oracle databases.
An example command:
IF EXISTS (SELECT 1 FROM information_schema.COLUMNS WHERE table_name='ResourceBookings' AND column_name = ('ResourceBookerKey1') AND character_maximum_length <= 50)
ALTER TABLE [ResourceBookings] ALTER COLUMN [ResourceBookerKey1] NVARCHAR(80) NULL
As you can see, the table is only changed if the column width needs to be increased
TIA
Before upgrading, make sure the SQL Server database's Recovery Model is set to "Simple". Go to SSMS, right-click the database, select properties, and then click on the Options pages. Record the "Recovery Mode" value. Set the Recovery Model to "Simple", if it's not already (I assume it's set to FULL).
Then run the upgrade. After the upgrade, you can restore the value back to what it was.
Alternately you can script it with something like this:
Before upgrade:
ALTER DATABASE MyDatabase SET RECOVERY SIMPLE;
After upgrade:
ALTER DATABASE MyDatabase SET RECOVERY FULL;
Situation: there's a table that is managed by application A. Application A inserts and updates data in the table throughout the day. Once per week it DROPs the table, recreates it, and inserts all data.
Problem: application A creates the table as utf8. Application B that relies on this table require for it to be ascii_bin. I did not design either application, nor do I have access to modifying their requirements.
What's needed: a way to ensure that the table is in ascii_bin. I considered writing a script and run it via CRON, which would check the current charset and set it if needed. Is there a better way of achieving this?
Since ALTER is one of the statements that causes an implicit COMMIT, I do not believe it is possible to do it as part of a trigger after INSERT or UPDATE.
You can set ascii_bin as a default charset for your database schema. Then all the created tables will have this charset when created, unless you explicitly specify another charset.
Refer to MySQL documentation on how to set the default charset: http://dev.mysql.com/doc/refman/5.0/en/charset-database.html
See SET NAME at http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html
MySQL proxy might be a solution here.
You can rewrite the create statement when it goes through the proxy.
Alternatively, maybe you could remove privileges from Application A so it can't drop the table.
An ALTER statement that makes no changes is basically ignored. So, if the conversion to ascii_bin is run multiple times, it's not going to be much effort on the server. So, putting it in cron, or an exiting stored procedure that Applicatio B calls, or something else clever, isn't so bad.
Because mysql default settings are not quite Unicode friendly it can happen quite often to endup with a database with broken charset.
Usually you just want to reconfigure it to use utf8 character set and utf8_unicode_ci collation.
Which is the easiest command to do this for a given database?
Warning: do not post links to untested scripts, I tested at least 5 of them (written in bash/perl/php/python) and they all failed to repair a database where the collation was set correctly at database and table level but not at column level.
I managed to write a solution myself and published to:
https://gist.github.com/1068021
Notes:
mysqldump is borken, even if tell it to not include CHARSET it will include them if it i set at column level.
this solution does not assume a default charset at mysql-server level so it set it at database level and resets it to defaults for table and column level.
Feed free to post bugs or patches, I will try to solve them fast.
I've had my self hosted Wordpress blog for a long time. I just realized that my DB is not UTF8 and certain plugins won't work correctly.
My question is this. How does a very novice mysql'er go about converting my database? As you can imagine, I'm very hesitant to do this on my own as I have 5 years worth of posts I don't want to jack up.
Can anyone point me in the right direction, or even better step me through the process for converting everything to UTF8?
After backing up your database as Konerak said, run this for every table:
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8;
(you may want to check with SHOW FULL COLUMNS FROM tablename whether all (text)columns are now indeed corrrect)
And right after you connect to MySQL, run the query:
SET NAMES utf8;
Now, to tell your audience you are using utf8: you could craft a custom header in every page or in an always included file, I however find it easier to put this in an .htaccess for Apache in the root:
php_value default_charset "UTF-8"
If you have non-ASCII content in flat files instead of only in the database, you'll have to convert them too. Your favorite editor may have a batch convert tool, or you can use iconv.
Just some pointers here.
I am making fairly extensive modifications to a site, including the MySQL database.
My plan is to do everything on my development server, export the new MySQL structure for the db and import it onto the clients server.
Basically I need to know that performing a structure only import will not overwrite/delete existing data. I am not making changes to the data type or field length.
In my experience, when you export a database (through phpMyAdmin for instance), part of the SQL script that is created includes a "DROP TABLE IF EXISTS 'table_name';" before doing a "CREATE TABLE 'table_name'...;" to build the new table.
My guess is that this is not what you want to do! Certainly use the dev system to alter the structure in order to make everything correct, but then look around for a database synchronisation routine where you can provide the old structure, the new structure, and the software will create the appropriate "ALTER TABLE 'table_name'...;" scripts to make the required changes.
You should then really examine these change files before executing them on the live database, and of course BACKUP the live database, and ensure you are able to fully recover from the backup before starting any of the alterations!
I've had to do this a lot, and it always goes like this:
Make a backup of the live database, complete with data.
Make a backup of the live database schema only.
Calculate the differences between the old (live) schema and the new (devel) schema.
Create all of the 'ALTER TABLE ...' DDL statements necessary to upgrade from the old schema to the new one. Keep in mind that if you rename a field, you probably won't be able to just rename it -- you'll need to create the new field, copy the data from the old field, and then drop the old field.
If you changed relationships between tables, you'll probably need to drop indexes and foreign key relationships first, and then add them back afterwards.
You'll need to populate any new fields based upon their default values, if any.
Once you've got all the pieces working, you'll need to combine them into one large script, and then run it on a copy of the live database.
Dump the schema and compare it against the desired new schema -- if they don't match, go back to step 3 and repeat.
Dump the data and compare it against the expected changes -- again, if they don't match, go back to step 3 and repeat.
You're going to learn a lot more about SQL DDL/DML during this process than you ever thought you'd learn. (For one project, where we were switching from natural keys to UUID keys for 50+ tables, I ended up writing programs to generate all of the DDL/DML.)
Good luck, and make frequent backups.
I'd recommend to prepare a sql script for every change you do on development server, so you will be able to reproduce it on development. You shouldn't get to the point where you need to calculate differences between database structures
This is how I do it, all changes are reflected in sql scripts, and I can reconstruct the history of my database running all these files if needed.
Test the final release version on a "staging" mysql server. Make a copy of your production server on another machine and test your script to make sure everything's ok.
Of course, preliminary database backup is a must.