Unique constraint violation with utf-8 values - mysql

I've created the following table in my MySQL (5.6.20) db:
CREATE TABLE `Description` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`sign_ref` varchar(150) CHARACTER SET utf8 NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uq_sign_ref` (`sign_ref`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
When inserting the value |SU&SU| into a new sign_ref row, I get a constraint violation, because |ŠU&ŠU| already exists. This happens whether I do it via SQLAlchemy, or directly using Sequel Pro.
All charset and collation settings are utf8mb4 / utf8 / utf8_general_ci / utf8_unicode_ci, so I'm not sure how to fix this.

Related

Foreign key rejected despite columns having different data types

I'm trying to set up two tables in a database, and add a foreign key between them. They're declared as follows:
CREATE TABLE `clothing` (
`name` varchar(26) COLLATE utf8_bin NOT NULL,
`image` varchar(64) COLLATE utf8_bin NOT NULL,
`localized_name` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL CHECK (json_valid(`localized_name`)),
`main` varchar(18) COLLATE utf8_bin DEFAULT NULL,
`stars` tinyint(3) unsigned NOT NULL,
`id` tinyint(3) unsigned NOT NULL,
`splatnet` smallint(5) unsigned NOT NULL,
PRIMARY KEY (`splatnet`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `abilities` (
`name` varchar(18) COLLATE utf8_bin DEFAULT NULL,
`image` varchar(48) COLLATE utf8_bin NOT NULL,
`id` tinyint(3) unsigned NOT NULL,
`localized_name` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL CHECK (json_valid(`localized_name`))
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
I want to create a foreign key on clothing that references abilities with the following command:
ALTER TABLE `abilities` ADD FOREIGN KEY (`name`) REFERENCES `clothing` (`main`);
However, attempting to do this raises this error in return:
Source and target columns must have the same data type, there must be an index on the target columns and referenced data must exist.
Can't create table `prismarine_rusted`.`abilities` (errno: 150 "Foreign key constraint is incorrectly formed")
I'm not entirely sure what's causing this, and unless I'm overlooking something really obvious, main and name have the same type, and therefore, should be able to be tied together via a foreign key. I'm using MariaDB v10.4.12, with SQL mode set to TRADITIONAL.
Although the foreign and primary key columns involved here are the same type, you are trying to reference clothing.main, which is not a unique or primary key column. From the MariaDB documentation:
The referenced columns must be a PRIMARY KEY or a UNIQUE index.
Note that this differs from InnoDB on MySQL, where a foreign key column can in fact reference a non unique column in another table.
One way to remedy this error would be to make clothing.main a unique column:
ALTER TABLE clothing ADD UNIQUE (main);
Note that doing this might only make logical sense if the values in main are already unique. If not, then perhaps you would have to revisit your data model.
It might be because there is a value in abilities.name that has no match in the referenced table.

Altering collation on field causes UNIQUE constraint to fail

When trying to alter my table with:
ALTER TABLE segment_item
CHANGE value value VARCHAR(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL;
I'm running into a UNIQUE constraint violation:
MySQLIntegrityConstraintViolationException: Duplicate entry for key 'segment_id'
Why would this be? This alteration to the table isn't adding a new record, it's just changing the character set and collation of existing records. For full visibility, here's the structure of the table:
CREATE TABLE `segment_item` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`value` varchar(100) COLLATE latin1_bin DEFAULT NULL,
`segment_id` binary(16) DEFAULT NULL,
`item_order` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `segment_id` (`segment_id`,`value`),
KEY `value` (`value`),
CONSTRAINT `segment_item_ibfk_1` FOREIGN KEY (`segment_id`) REFERENCES `segment` (`segment_id`)
) ENGINE=InnoDB AUTO_INCREMENT=36484 DEFAULT CHARSET=latin1 COLLATE=latin1_bin
The collation is how the db engine determines when records are unique. Looks like here you're changing from a binary collation (literally unique if and only if the bytes of the string are unique) to case insensitive unicode (which is case insensitive, but also uses a list of standard transformations to normalize digraphs, code point orders, etc).
So, for example, the following two strings used to be unique before your change (é is valid Latin-1):
Beyoncé
beyonce
But after the change they would be considered identical. My guess is that you have at least two "value"... er... values that are no longer unique under the new collation.

The size of a table does not match the records in MySQL, AWS RDS

I have a problem with a MySQL table, the table has only 385 records that if I export them via Workbench the file is 894 KB in size. The problem is that the Data length is 10 GB when I try to inspect the table.
Note: In the table when making query only show 385 records.
The only solution to restore the size of the table that I have used is to drop the table and import it again.
I hope and you can help me to verify if it is a MySQL error or it is an attack or I don't know why the behavior is due.
The MySQL server is in an AWS RDS.
I have read the next article: https://docs.aws.amazon.com/es_es/AmazonRDS/latest/UserGuide/MySQL.KnownIssuesAndLimitations.html
CREATE TABLE `logs` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL DEFAULT '',
`file` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
`status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL DEFAULT 'yes',
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=130204 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;

arabic word changed to ???? after inserting into Mysql DB

when I insert data in this table, my query is
INSERT INTO urdu_word (word) VALUES ('Abdelali Abou Dher (عبد العالي ابو ذر)') ON DUPLICATE KEY UPDATE word='Abdelali Abou Dher (عبد العالي ابو ذر)' word value replace to like ???
My table structure is:
CREATE TABLE `urdu_word` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`word` varchar(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `word` (`word`),
KEY `idx_aml_word_status` (`word`)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;
I also tried to chage table structure to utf8_unicode_ci but same problem facing
CREATE TABLE `aml_word` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`word` varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `word` (`word`),
KEY `idx_aml_word_status` (`word`)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;
mysql version 5.6
when insert query run in mysql command line then it inserted well in urdu but when I inserted through code using mybatis ORM then create problem.
The default character set for MySQL at (mt) Media Temple is latin1,
with a default collation of latin1_swedish_
So you need to change your callation
Try with this query
ALTER DATABASE dbname CHARACTER SET utf8 COLLATE utf8_general_ci;
Followed by this tutorial
https://mediatemple.net/community/products/dv/204403914/default-mysql-character-set-and-collation#gs

MySQL unique 1500 varchar field error (#1071 - Specified key was too long)

I have a field (link) that is varchar (1500) and that I want to make unique. I applied changes to mysql configuaration and increased length to 3072 bytes
ROW_FORMAT=DYNAMIC, innodb_file_format = Barracuda, innodb_large_prefix = true
But when I apply unique to my field, I got next error:
"#1071 - Specified key was too long; max key length is 3072 bytes"
My field is varchar(1500) that is 3000 bytes.
What's wrong?
Update (1)
Table data:
CREATE TABLE IF NOT EXISTS `pages` (
`link` varchar(1500) NOT NULL,
`domain` varchar(255) NOT NULL,
`lastvisited` datetime DEFAULT NULL,
`id` bigint(20) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `link` (`link`(255))
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ROW_FORMAT=DYNAMIC;
Update (2)
Alter command (done via PHPMYADMIN)
ALTER TABLE `pages` ADD UNIQUE (
`link`
)
Since you will be storing URLs in the link column, you don't actually need to use UTF8 for it, because URLs can contain only ASCII characters. Specifying a plain ASCII character encoding for your link column will even allow you to raise its max length to 3072 characters.
CREATE TABLE IF NOT EXISTS `pages` (
`link` varchar(1500) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
`domain` varchar(255) NOT NULL,
`lastvisited` datetime DEFAULT NULL,
`id` bigint(20) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
UNIQUE KEY `link` (`link`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ROW_FORMAT=DYNAMIC;
(Updated as per #eggyal's suggestion for the ascii_bin collation)