MySQL utf8 multibyte (utf8mb4) insert duplicate entry problem - mysql

I have two words ('বাঁধা' and 'বাধা') to be inserted in a mysql (8.0.12 - MySQL Community Server - GPL) table. The word 'বাঁধা' is inserted correctly. But when inserting 'বাধা', mysql produces an error:
INSERT INTO lc6_words(jp_word, jp_fcharascii) VALUES('বাঁধা', 2476);
/*Query OK*/
INSERT INTO lc6_words(jp_word, jp_fcharascii) VALUES('বাধা', 2476);
/*#1062 - Duplicate entry 'বাধা' for key 'jp_word'*/
The table structure:
CREATE TABLE IF NOT EXISTS `lc6_words` (
`jp_wkey` BIGINT NOT NULL AUTO_INCREMENT,
`jp_word` varchar(255) NOT NULL,
`jp_fcharascii` int UNSIGNED NOT NULL,
`jp_word_occ` BIGINT UNSIGNED NOT NULL DEFAULT 1,
UNIQUE(`jp_word`),
PRIMARY KEY (`jp_wkey`)
) ENGINE=MyISAM DEFAULT CHARSET=UTF8MB4 COLLATE=utf8mb4_bin;
Relevant queries and their output:
SELECT jp_wkey FROM lc6_words WHERE BINARY jp_word='বাঁধা';
/* 1 */
SELECT jp_wkey FROM lc6_words WHERE BINARY jp_word='বাধা';
/* Empty */
Thanks for reading this far. And some more too if you share your thoughts :).

There seems to be problem in collation. After running the command below, all worked perfectly:
ALTER TABLE lc6_words MODIFY jp_word VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;
Note: The VARCHAR size changed from 255 to 191.

Related

How to set a default character set for a table in MySQL and MySQL Workbench?

I am forward engineering a MySQL database from an EER diagram in MySQL workbench, and am being shown the following error on the execution of the statement below:
ERROR: Error 1115: Unknown character set: 'DEFAULT'
CREATE TABLE IF NOT EXISTS `recipes_database_2`.`cleaned_string` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`string` VARCHAR(511) CHARACTER SET 'DEFAULT' NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `string_UNIQUE` (`string` ASC) VISIBLE)
ENGINE = InnoDB
AUTO_INCREMENT = 1
DEFAULT CHARACTER SET = DEFAULT
What can I change to fix this? I want the character set to be the one I set as the default for the database.
Just don't specify the CHARACTER SET at all, then the
database character set and collation are used
as one can read from the documentation.

"ERROR 1406: 1406: Data too long for column" but it shouldn't be?

I have the following table structure:
DROP TABLE IF EXISTS `tblusers`;
/*!40101 SET #saved_cs_client = ##character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `tblusers` (
`UserID` int(5) NOT NULL AUTO_INCREMENT,
`ContactPersonID` int(5) NOT NULL,
`NameOfUser` varchar(70) NOT NULL,
`LegalForm` varchar(70) DEFAULT NULL,
`Address` varchar(70) DEFAULT NULL,
`City` varchar(50) DEFAULT NULL,
`Postal` int(8) DEFAULT NULL,
`Country` varchar(50) DEFAULT NULL,
`VatNum` int(10) DEFAULT NULL,
`Username` varchar(30) NOT NULL,
`Password` varchar(20) NOT NULL,
`Email` varchar(40) NOT NULL,
`Website` varchar(40) DEFAULT NULL,
`IsSeller` bit(1) DEFAULT NULL,
`IsBuyer` bit(1) DEFAULT NULL,
`IsAdmin` bit(1) DEFAULT NULL,
`Description` text,
PRIMARY KEY (`UserID`),
KEY `ContactPersonID` (`ContactPersonID`),
CONSTRAINT `tblusers_tblpersons` FOREIGN KEY (`ContactPersonID`) REFERENCES `tblpersons` (`PersonID`)
) ENGINE=InnoDB AUTO_INCREMENT=87 DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = #saved_cs_client */;
Then once I create a user from the UI of my application, I have to manually set the very first admin, and this is the only time I am doing this directly from the DB, all the rest is envisioned to be done from the UI (granting admin privileges):
UPDATE `tblusers` SET `IsAdmin`='1' WHERE `UserID`='79';
but then I get:
Operation failed: There was an error while applying the SQL script to the database.
Executing:
UPDATE `trace`.`tblusers` SET `IsAdmin`='1' WHERE `UserID`='79';
ERROR 1406: 1406: Data too long for column 'IsAdmin' at row 1
SQL Statement:
UPDATE `trace`.`tblusers` SET `IsAdmin`='1' WHERE `UserID`='79'
Which doesn't make sense because I am doing the exact same thing on other machines and it works like a charm. The only difference is that in this scenario I have mysql 5.7 server whereas I have 5.6 versions on the machines that this does work.
I tried the following solution but it didn't work for me. Besides that, the my.ini file is unchanged in the 5.6 machine where it does work.
Downgrading to 5.6 is out of the question. I need a real solution here please.
isadmin is a column of type bit and you are storing a value of type varchar in it which is of larger size than bit. modify query as follows:-
UPDATE `tblusers` SET `IsAdmin`=b'1' WHERE `UserID`='79';
IsAdmin has the datatype of bit(1), yet you are assigning the string '1' to it. Indicate that you are assigning a bit value to it by preceeding the '1' with b or use 0b format:
UPDATE `tblusers` SET `IsAdmin`=b'1' WHERE `UserID`='79';
or
UPDATE `tblusers` SET `IsAdmin`=0b1 WHERE `UserID`='79';
The reason for this behaviour is probably that strict_all_tables or strict_trans_tables setting is enabled on the v5.7 mysql server:
Strict mode controls how MySQL handles invalid or missing values in
data-change statements such as INSERT or UPDATE. A value can be
invalid for several reasons. For example, it might have the wrong data
type for the column, or it might be out of range. A value is missing
when a new row to be inserted does not contain a value for a non-NULL
column that has no explicit DEFAULT clause in its definition. (For a
NULL column, NULL is inserted if the value is missing.) Strict mode
also affects DDL statements such as CREATE TABLE.
The BIT data type is used to store bit values. A type of BIT(M) enables storage of M-bit values. M can range from 1 to 64.
UPDATE tblusers SET IsAdmin=b'1' WHERE UserID='012';
UPDATE tblusers SET IsAdmin=b'0' WHERE UserID='012';
I had the same problem when I synchronized the Model's table from MySQL Workbench to the MySQL server which had old tables with data. the data of old column types is longer than the new column types. (for example: the old column type is char[43] but the new column type is binary[32] so the new column type can't contain all of the old data)
my solution: drop the old table and then synchronized new Model with the old database

MySQL SHA1 hash does not match

I have a weird problem with a MySQL users table. I have quickly created a simplified version as a testcase.
I have the following table
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`identity` varchar(255) NOT NULL,
`credential` varchar(255) NOT NULL,
`credentialSalt` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=ucs2 AUTO_INCREMENT=2 ;
INSERT INTO `users` (`id`, `identity`, `credential`, `credentialSalt`) VALUES
(1, 'test', '7288edd0fc3ffcbe93a0cf06e3568e28521687bc', '123');
And I run the following query
SELECT id,
IF (credential = SHA1(CONCAT('test', credentialSalt)), 1, 0) AS dynamicSaltMatches,
credentialSalt AS dynamicSalt,
SHA1(CONCAT('test', credentialSalt)) AS dynamicSaltHash,
IF (credential = SHA1(CONCAT('test', 123)), 1, 0) AS staticSaltMatches,
123 AS staticSalt,
SHA1(CONCAT('test', 123)) AS staticSaltHash
FROM users
WHERE identity = 'test'
Which gives me the following result
The dynamic salt does NOT match while the static salt DOES match.
This is blowing my mind. Can someone help me point out the cause of this?
My MySQL version is 5.5.29
It's because of the default character set of your table. You appear to be running this on a UTF8 database and something in SHA1() is having problems with the differing character sets.
If you change your table declaration to the following it will match again:
CREATE TABLE IF NOT EXISTS `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`identity` varchar(255) NOT NULL,
`credential` varchar(255) NOT NULL,
`credentialSalt` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2 ;
SQL Fiddle
As robertklep commented explicitly casting your string to a character will also work, basically ensure you're using the same characterset when doing comparisons using SHA1()
As the encryption functions documentation says:
Many encryption and compression functions return strings for which the result might contain arbitrary byte values. If you want to store these results, use a column with a VARBINARY or BLOB binary string data type. This will avoid potential problems with trailing space removal or character set conversion that would change data values, such as may occur if you use a nonbinary string data type (CHAR, VARCHAR, TEXT).
This was changed in version 5.5.3:
As of MySQL 5.5.3, the return value is a nonbinary string in the connection character set. Before 5.5.3, the return value is a binary string; see the notes at the beginning of this section about using the value as a nonbinary string.

Mysql - duplicate entry error for key with auto increment

Why do I get an error of the form:
Error in query: Duplicate entry '10' for key 1
...when doing an INSERT statement like:
INSERT INTO wp_abk_period (pricing_id, apartment_id) VALUES (13, 27)
...with 13 and 27 being valid id-s for existing pricing and apartment rows, and the table is defined as:
CREATE TABLE `wp_abk_period` (
`id` int(11) NOT NULL auto_increment,
`apartment_id` int(11) NOT NULL,
`pricing_id` int(11) NOT NULL,
`type` enum('available','booked','unavailable') collate utf8_unicode_ci default NULL,
`starts` datetime default NULL,
`ends` datetime default NULL,
`recur_type` enum('daily','weekly','monthly','yearly') collate utf8_unicode_ci default NULL,
`recur_every` char(3) collate utf8_unicode_ci default NULL,
`timedate_significance` char(4) collate utf8_unicode_ci default NULL,
`check_in_times` varchar(255) collate utf8_unicode_ci default NULL,
`check_out_times` varchar(255) collate utf8_unicode_ci default NULL,
PRIMARY KEY (`id`),
KEY `fk_period_apartment1_idx` (`apartment_id`),
KEY `fk_period_pricing1_idx` (`pricing_id`),
CONSTRAINT `fk_period_apartment1` FOREIGN KEY (`apartment_id`) REFERENCES `wp_abk_apartment` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `fk_period_pricing1` FOREIGN KEY (`pricing_id`) REFERENCES `wp_abk_pricing` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
Isn't key 1 id in this case and having it on auto_increment sufficient for being able to not specify it?
Note: If I just provide an unused value for id, like INSERT INTO wp_abk_period (id, pricing_id, apartment_id) VALUES (3333333, 13, 27) it works fine, but then again, it is set as auto_increment so I shouldn't need to do this!
Note 2: OK, this is a complete "twilight zone" moment: so after running the query above with the huge number for id, things started working normally, no more duplicate entry errors. Can someone explain me WTF was MySQL doing to produce this weird behavior?
It could be that your AUTO_INCREMENT value for the table and the actual values in id column have got out of whack.
This might help:
Step 1 - Get Max id from table
select max(id) from wp_abk_period
Step 2 - Align the AUTO_INCREMENT counter on table
ALTER TABLE wp_abk_period AUTO_INCREMENT = <value from step 1 + 100>;
Step 3 - Retry the insert
As for why the AUTO_INCREMENT has got out of whack I don't know. Added auto_increment after data was in the table? Altered the auto_increment value after data was inserted into the table?
Hope it helps.
I had the same problem and here is my solution :
My ID column had a bad parameter. It was Tinyint, and MySql want to write a 128th line.
Sometimes, your problem you think the bigger you have is only a tiny parameter...
Late to the party, but I just ran into this tonight - duplicate key '472817' and the provided answers didn't help.
On a whim I ran:
repair table wp_abk_period
which output
Number of rows changed from 472816 to 472817
Seems like mysql had the row count wrong, and the issue went away.
My environment:
mysql Ver 14.14 Distrib 5.1.73, for Win64 (unknown)
Create table syntax:
CREATE TABLE `env_events` (
`tableId` int(11) NOT NULL AUTO_INCREMENT,
`deviceId` varchar(50) DEFAULT NULL,
`timestamp` int(11) DEFAULT NULL,
`temperature` float DEFAULT NULL,
`humidity` float DEFAULT NULL,
`pressure` float DEFAULT NULL,
`motion` int(11) DEFAULT NULL,
PRIMARY KEY (`tableId`)
) ENGINE=MyISAM AUTO_INCREMENT=528521 DEFAULT CHARSET=latin1
You can check the current value of the auto_increment with the following command:
show table status
Then check the max value of the id and see if it looks right. If not change the auto_increment value of your table.
When debugging this problem check the table name case sensitivity (especially if you run MySql not on Windows).
E.g. if one script uses upper case to 'CREATE TABLE my_table' and another script tries to 'INSERT INTO MY_TABLE'. These 2 tables might have different contents and different file system locations which might lead to the described problem.

Errors while migrating from MS SQL 2000 to MySQL 5.1

I am using migration toolkit for the migration but i am getting these errors in the process of migration
Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause
Incorrect string value: '\xEF\xBF\xBDs d...' for column 'MESSAGE' at row 5
0 row(s) transferred.
For the fixing the first error i got something here http://terrencemiao.com/Webmail/msg00949.html
but i am not getting the second error what it is and why is it there how to fix it also suggest me some better ideas for fixing the first one if there any apart from what mentioned in the link
USE `MyDB`
Creating tables ...
Creating table MyTable...
DROP TABLE IF EXISTS `MyTable`
Creating table MyTable ...
SET NAMES UTF8;
CREATE TABLE `MyTable` (
`PrimaryKey` INT(10) NOT NULL AUTO_INCREMENT,
`FK_QUESTION_ID` INT(10) NOT NULL,
`ANSWER` LONGTEXT CHARACTER SET utf8 COLLATE utf8_general_ci NULL
PRIMARY KEY (`PK_ID`)
)
ENGINE = INNODB
i am getting error for answer column
*Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause*
This is right, you should not create more then one such fields.
Incorrect string value: '\xEF\xBF\xBDs d...' for column 'MESSAGE' at row 5 0 row(s) transferred.
Possible encoding error, try to run 'SET NAMES UTF8;' before inserting data
Try this statement,
CREATE TABLE `MyTable` (
PK_ID INT(11) NOT NULL AUTO_INCREMENT,
FK_QUESTION_ID INT(11) NOT NULL,
ANSWER LONGTEXT CHARACTER SET utf8 COLLATE utf8_general_ci NULL,
PRIMARY KEY (`PK_ID`)
)
ENGINE = INNODB;
You missed a comma and it was wrong field name. Be careful with migration toolkit. Check generated field types, for example if you do not need 4GB text values, you could use simple VARCHAR instead of LONGTEXT.