When I change the default charset from UTF-8 to latin1, it changes the text, varchar columns to utf8
So the new table looks something like this...
...
`include_zoneclass` varchar(255) CHARACTER SET utf8 DEFAULT NULL,
...
PRIMARY KEY (`campaignid`)
) ENGINE=InnoDB AUTO_INCREMENT=2082 DEFAULT CHARSET=latin1
How do I make the table totally latin1 and no trace of utf8 should be there.
Try using the ALTER TABLE command with CONVERT TO CHARACTER SET
See here for more:
http://dev.mysql.com/doc/refman/5.1/en/alter-table.html
Related
Hi I recently changed the hosting provider for my website. When doing this I exported the mysql database I had in my previous cpanel phpmyadmin. It had CHARACTER SET latin1 and COLLATE latin1_swedish_ci. After I importing it to my new phpmyadmin I saw there was an issue with displaying the characters written in Czech ě ř č ů which appeared as question mark or weird symbols etc. I also wasn't able to insert these letters at first but after changing the table CHARSET to utf8 I'm able to insert them. But how do I export the data from my old database and import it in the new one without messing up the data? Here's what the database looks like:
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET AUTOCOMMIT = 0;
START TRANSACTION;
SET time_zone = "+00:00";
/*!40101 SET #OLD_CHARACTER_SET_CLIENT=##CHARACTER_SET_CLIENT */;
/*!40101 SET #OLD_CHARACTER_SET_RESULTS=##CHARACTER_SET_RESULTS */;
/*!40101 SET #OLD_COLLATION_CONNECTION=##COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8mb4 */;
--
-- Database: `sambajiu_samba`
--
-- --------------------------------------------------------
CREATE TABLE `bookings` (
`id` int(11) NOT NULL,
`fname` varchar(100) NOT NULL,
`surname` varchar(100) DEFAULT NULL,
`email` varchar(255) NOT NULL,
`telephone` varchar(100) NOT NULL,
`age_group` varchar(100) DEFAULT NULL,
`hear` varchar(100) DEFAULT NULL,
`experience` text,
`subscriber` tinyint(1) DEFAULT NULL,
`booking_date` varchar(255) DEFAULT NULL,
`lesson_time` varchar(255) NOT NULL,
`booked_on` datetime DEFAULT CURRENT_TIMESTAMP
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `bookings` ADD PRIMARY KEY (`id`);
ALTER TABLE `bookings` MODIFY `id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=345;
Czech is not handled by latin1. It would be better to use utf8mb4 (which can handle virtually everything in the world). Outside of MySQL, it is called "UTF-8".
How did you do the "export" and "import"? What is in the file? Can you get the hex of a small portion of the exported file -- we need to check what encoding was used for the Czech characters.
As for "as question mark or weird symbols", see question marks and Mojibake in Trouble with UTF-8 characters; what I see is not what I stored .
Your hex probably intended to say
Rezervovat trénink zda
In the middle of the hex is
C383 C2A9
Which is UTF-8 for é. When you display the data, you might see that, or you might see the desired é. In the latter case, the browser is probably "helping" you by decoding the data twice. For further discussion on this, see "double encoding" in the link above.
"Fixing the data" is quite messy:
CONVERT(BINARY(CONVERT(CONVERT(
UNHEX('52657A6572766F766174207472C383C2A96E696E6B207A6461')
USING utf8mb4) USING latin1)) USING utf8mb4)
==> 'Rezervovat trénink zda'
But, I don't think we are finished. that acute-e is a valid character in latin1. You mentioned 4 Czech accented letters that, I think, are not in Latin1. Latin5 and dec8 may be relevant.
I have a problem when trying to execute this line in MySQL (Workbench):
INSERT INTO classification (`Type`, `Subtype`) VALUES ("тип", "подтип");
I have tried to set different charsets for table classification : cp1251, utf-8, utf8mb4, cp1251_bin.
This is a table with all charsets in my database that I have found, maybe it will help you:
UPD. I have found a solution. However, I had to change my table, so now the table risk is an edited table classification. The result of SHOW CREATE TABLE risk is:
'CREATE TABLE `risk` (
`IdRisk` int(11) NOT NULL AUTO_INCREMENT,
`IdSubtype` int(11) DEFAULT NULL,
`Content` varchar(4000) CHARACTER SET utf8 DEFAULT NULL,
PRIMARY KEY (`IdRisk`),
KEY `FK_subtype_risk_idx` (`IdSubtype`),
CONSTRAINT `FK_subtype_risk` FOREIGN KEY (`IdSubtype`) REFERENCES `subtype` (`IdSubtype`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=48 DEFAULT CHARSET=latin1'
Can't find the solution to this issue. I'm hope that someone knows a solution to it.
Thank You!
The CHARACTER SET for the table is the default for columns in the table. Please provide SHOW CREATE TABLE so we can verify what the columns are set to.
What is the encoding of the bytes in the client? cp1251 is different than utf8; utf8mb4 == utf8 for Russian.
In what way are things bad? Based on the symptom, see this for specific tips on what else might be set incorrectly.
Perhaps it was your change to NVARCHAR that forced CHARACTER SET utf8 on the columns?
I've always been surprised that MySQL and related tools tend to specify both CHARACTER SET and COLLATION in CREATE TABLE AND ALTER TABLE statements:
SHOW CREATE TABLE test;
CREATE TABLE ...
ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
When I select a collation for a column in MySQL Workbench, I get:
ALTER TABLE ...
CHARACTER SET 'latin1' COLLATE 'latin1_general_ci' ...;
I've always supposed that specifying the character set and the collation was redundant, as the collation implies the character set.
Am I wrong?
When I try to mix a collation with another charset, I get an error:
CREATE TABLE ... DEFAULT CHARACTER SET utf8 collate latin1_bin;
COLLATION 'latin1_bin' is not valid for CHARACTER SET 'utf8'
Is it safe to only ever specify the collation?
If so, why do all these tools systematically include the charset in the statement, too?
You are not required to specify character set when creating table, it's automatically set on collate as it described in the MySQL Reference Manual:
MySQL chooses the table character set and collation in the following
manner:
If both CHARACTER SET charset_name and COLLATE collation_name are specified, character set charset_name and collation collation_name are used.
If CHARACTER SET charset_name is specified without COLLATE, character set charset_name and its default collation are used. To see the default collation for each character set, use the SHOW CHARACTER SET statement or query the INFORMATION_SCHEMA CHARACTER_SETS table.
If COLLATE collation_name is specified without CHARACTER SET, the character set associated with collation_name and collation collation_name are used.
Otherwise (neither CHARACTER SET nor COLLATE is specified), the database character set and collation are used.
The table character set and collation are used as default values
for column definitions if the column character set and collation are
not specified in individual column definitions. The table character
set and collation are MySQL extensions; there are no such things
in standard SQL.
So you can create a table using this query:
CREATE TABLE `tabletest` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 COLLATE=latin1_general_ci;
This query will create a table with CHARSET=latin1 COLLATE=latin1_general_ci, so it's safe to specify COLLATE only.
As for why are there both CHARSET and COLLATE, please read the following:
What is the difference between collation and character set?
I have the following UTF-8 table:
CREATE TABLE `table` (
`id` int(11) NOT NULL DEFAULT '0',
`description` mediumtext,
`description_info` mediumtext,
`rights` mediumtext,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
inside a UTF-8 database:
CREATE DATABASE `DB` /*!40100 DEFAULT CHARACTER SET utf8 */
in this table there are rows with LATIN1 characters, like: è or ò
My question is: how can I convert those characters?
THX!
Use a CONVERT:
SELECT
description,
CONVERT(description USING utg8)
FROM
`table`
I have found a solution here: http://www.percona.com/blog/2013/10/16/utf8-data-on-latin1-tables-converting-to-utf8-without-downtime-or-double-encoding/
Those are the steps I have applied to convert latin1 text inside the 'description' column to UTF8:
ALTER TABLE table CONVERT TO CHARACTER SET latin1 -- this is because my table is in UTF8
ALTER TABLE table CHANGE description description BLOB;
ALTER TABLE table CONVERT TO CHARACTER SET utf8, CHANGE description description mediumtext;
I don't know if this is the most effective method to this... but it works!
Ok, in the existing table,
CREATE TABLE tb(id int, `text` varchar(255) character set latin1 collate latin1_bin default NULL)ENGINE=InnoDB DEFAULT CHARSET=latin1
I want to make the text column to be utf8 and it supports Case Sensitive for select query
Suppose there is "ã" & "Ã" which is utf8 in the text column
id - text
1 - ã
2 - Ã
, and when user select * from tb where text='Ã'it should show record with id=2 only
Is this query the correct one
alter table tb modify column text varchar(255) character set utf8 collate utf8_general_ci default NULL
I have no idea what utf8_general_ci means? am i doing cortrectly?
This can be done by using the Alter Command.
Like ALTER TABLE urtable MODIFY col1 VARCHAR(50) CHARACTER SET utf8;
See column charset conversion here
this is the correct one
alter table tb modify text varchar(255) character set utf8 collate utf8_bin default NULL