"Data too long for column" - why? - mysql

I've written a MySQL script to create a database for hypothetical hospital records and populate it with data. One of the tables, Department, has a column named Description, which is declared as type varchar(200). When executing the INSERT command for Description I get an error:
error 1406: Data too long for column 'Description' at row 1.
All the strings I'm inserting are less than 150 characters.
Here's the declaration:
CREATE TABLE Department(
...
Description varchar(200)
...);
And here's the insertion command:
INSERT INTO Department VALUES
(..., 'There is some text here',...), (..., 'There is some more text over here',...);
By all appearances, this should be working. Anyone have some insight?

Change column type to LONGTEXT

I had a similar problem when migrating an old database to a new version.
Switch the MySQL mode to not use STRICT.
SET ##global.sql_mode= 'NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
Error Code: 1406. Data too long for column - MySQL

There is an hard limit on how much data can be stored in a single row of a mysql table, regardless of the number of columns or the individual column length.
As stated in the OFFICIAL DOCUMENTATION
The maximum row size constrains the number (and possibly size) of columns because the total length of all columns cannot exceed this size. For example, utf8 characters require up to three bytes per character, so for a CHAR(255) CHARACTER SET utf8 column, the server must allocate 255 × 3 = 765 bytes per value. Consequently, a table cannot contain more than 65,535 / 765 = 85 such columns.
Storage for variable-length columns includes length bytes, which are assessed against the row size. For example, a VARCHAR(255) CHARACTER SET utf8 column takes two bytes to store the length of the value, so each value can take up to 767 bytes.
Here you can find INNODB TABLES LIMITATIONS

in mysql if you take VARCHAR then change it to TEXT bcoz its size is 65,535
and if you can already take TEXT the change it with LONGTEXT only if u need more then 65,535.
total size of LONGTEXT is 4,294,967,295 characters

Varchar has its own limits. Maybe try changing datatype to text.!

Turns out, as is often the case, it was a stupid error on my part. The way I was testing this, I wasn't rebuilding the Department table after changing the data type from varchar(50) to varchar(200); I was just re-running the insert command, still with the column as varchar(50).

If your source data is larger than your target field and you just want to cut off any extra characters, but you don't want to turn off strict mode or change the target field's size, then just cut the data down to the size you need with LEFT(field_name,size).
INSERT INTO Department VALUES
(..., LEFT('There is some text here',30),...), (..., LEFT('There is some more text over here',30),...);
I used "30" as an example of your target field's size.
In some of my code, it's easy to get the target field's size and do this. But if your code makes that hard, then go with one of the other answers.

For me, I defined column type as BIT (e.g. "boolean")
When I tried to set column value "1" via UI (Workbench), I was getting a "Data too long for column" error.
Turns out that there is a special syntax for setting BIT values, which is:
b'1'

With Hibernate you can create your own UserType. So thats what I did for this issue. Something as simple as this:
public class BytesType implements org.hibernate.usertype.UserType {
private final int[] SQL_TYPES = new int[] { java.sql.Types.VARBINARY };
//...
}
There of course is more to implement from extending your own UserType but I just wanted to throw that out there for anyone looking for other methods.

Very old question, but I tried everything suggested above and still could not get it resolved.
Turns out that, I had after insert/update trigger for the main table which tracked the changes by inserting the record in history table having similar structure. I increased the size in the main table column but forgot to change the size of history table column and that created the problem.
I did similar changes in the other table and error is gone.

I try to create a table with a field as 200 characters and I've added two rows with early 160 characters and it's OK. Are you sure your rows are less than 200 characters?
Show SqlFiddle

There was a similar problem when storing a hashed password into a table. Changing the maximum column length didn't help. Everything turned out to be simple. It was necessary to delete the previously created table from the database, and then test the code with new values ​​of the allowable length.

If you re using type: DataTypes.STRING, then just pass how long this string can be like DataTypes.STRING(1000)

In my case this error occurred due to entering data a wrong type for example: if it is a long type column, i tried to enter in string type. so please check your data that you are entering and type are same or not

For me, I try to update column type "boolean" value
When I tried to set column value 1 MySQL Workbench, I was getting a "Data too long for column" error.
So for that there is a special syntax for setting boolean values, which is:
UPDATE `DBNAME`.`TABLE_NAME` SET `FIELD_NAME` = false WHERE (`ID` = 'ID_VALUE'); //false for 0
UPDATE `DBNAME`.`TABLE_NAME` SET `FIELD_NAME` = true WHERE (`ID` = 'ID_VALUE'); //true for 1

I had a different problem which gave the same error so I'll make a quick recap as this seems to have quite different sources and the error does not help much to track down the root cause.
Common sources for INSERT / UPDATE
Size of value in row
This is exactly what the error is complaining about. Maybe it's just that.
You can:
increase the column size: for long strings you can try to use TEXT, MEDIUMTEXT or LONGTEXT
trim the value that is too long: you can use tools from the language you're using to build the query or directly in SQL with LEFT(value,size) or RIGHT(...) or SUBSTRING(...)
Beware that there is a maximum row size in a MySQL table as reported by this answer. Check documentation and InnoDB engine limitations.
Datatype Mismatch
One or more rows are of the wrong datatype.
common sources of error are
ENUM
BIT: don't use 1 but b'1'
Data outlier
In a long list of insert one can easily miss a row which has a field not adhering to the column typing, like an ENUM generated from a string.
Python Django
Check if you have sample_history enabled, after a change in a column size it must be updated too.

Related

How to update exact decimal value in mysql database

I am working with mysql. I created a column in database called "balance" and the datatype for this column is "DECIMAL(12,6)".
So whenever I try to update 4 digits after the decimal point then the last two digits are showing random values (e.g. balance is showing 4444.888672 for the following query).
Here is my current query
UPDATE `table` SET `balance` = '4444.8888' WHERE `token_address` = 'abc123'
Sounds like common float overflow issue, where decimal part does not fit into memory and is cut off.
In our system we use INT's instead. So you would save into database 44448888000 and in PHP you would parse it as $row['balance'] / 1000000

How to select one column value, normalize and put to another column in mysql?

In my table, I've got a column call mobile and I need this mobile field value to be normalized and save to another column call formatted_phone. For this purpose, I am using the below MySQL query and unfortunately, it is not working. I am putting my query here, please someone correct it. Thank you.
UPDATE hiring_detail
SET formatted_phone = replace(replace(
replace(replace(replace(replace(mobile,'-',''),'+',''),')',''),'(',''),' ',''),'.','')
WHERE mobile IS NOT NULL;
Error what it throws:
SQL Error (1265):Data truncated for column 'formatted_phone' at row 3
mobile column: varchar 50
formatted_phone: bigint 15
Usually, beofre blindly executing UPDATE commands, we do a simply What If analysis first, just run the query as a SELECT so you can inspect the output and importantly, you can compare it to the existing values:
http://sqlfiddle.com/#!9/47723a/2
SELECT mobile, formatted_phone
, REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(mobile,'-',''),'+',''),')',''),'(',''),' ',''),'.','') as test
FROM hiring_detail
WHERE mobile IS NOT NULL;
If that logic works for you (and it does in my tests) then there should be no issue with your UPDATE logic.
UPDATE:
If your error is
SQL Error (1265):Data truncated for column 'formatted_phone' at row 3
Then that means that your formatted phone numbers are longer than the column width for formatted_phone. If you know what the length is, you can truncate your formatted numbers, but with phone numbers, if we remove the actual number, this usually results in phone numbers that cannot be called.
I would recommend instead that you increase the width of the formatted_phone field.
This is an example of a forced truncation:
UPDATE hiring_detail
SET formatted_phone = RIGHT(replace(replace(
replace(replace(replace(replace(mobile,'-',''),'+',''),')',''),'(',''),' ',''),'.',''),10)
WHERE mobile IS NOT NULL;
Update #2
given that the column is an int, we need to convert the value into an integer.
UPDATE hiring_detail
SET formatted_phone = CAST(replace(replace(
replace(replace(replace(replace(mobile,'-',''),'+',''),')',''),'(',''),' ',''),'.','') as UNSIGNED INT)
WHERE mobile IS NOT NULL;
Warning: It is NOT advisable to store phone numbers as integers, the leading zeros can be significant in area codes in many localities, by storing as a numeric value this can have significant effects and can result in los of data. It also makes it hard to search for partial matches on the numbers. Almost all operations that you can think of (including sorting) on phone numbers will involve string manipulations, not mathematical or numerical.

Incorrect string value: '\xE2\x80\xAF(fo...' for column 'description' at row 1 Error: INSERT INTO my_table_name

Sometimes, when the text is copy pasted from a third website in my form based application (in the textarea) the data don't get inserted in database, instead throw this below error.
Incorrect string value: '\xE2\x80\xAF(fo...' for column 'my_column_name' at row 1 Error: INSERT INTO my_table_name
I tried the below query in mysql workbench to solve this issue.
ALTER TABLE my_database_name.my_table CONVERT TO CHARACTER SET utf8
But I am getting the below error from the database.
Error Code: 1118. Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs
Your column data type accepts maximum limit of 65535 bytes. you need to change the column data type to text or BLOB
One more thing while copying content from website or word document just paste in any plain text editor and check whether expected content is copied
You can use $content= preg_replace('/[\xE2\x80\xAF]/', '', $content); in programming. the above example is in PHP
Don't use whitespace in names: hex E280AF is UTF-8 FOR "NARROW NO-BREAK SPACE".
I worry that doing ALTER TABLE my_database_name.my_table CONVERT TO CHARACTER SET utf8 without first diagnosing the problem has only made things worse.
You were probably using latin1 before? Did you have any other non-English text in the database? They may (or may not) be messed up.
We may be able to fix the mess, but we need to know more details about what you originally had, and what steps lead to this.
Also, what language(s) do you expect your customers to be using?

Getting Incorrect string value: '\x92\xB7\xFF\xF3' while using AES_ENCRYPT in MySQL

I know and I've researched a lot of very similar problems in Stackoverflow. I've tried all solutions and nothing has worked to me.
I've used following query to insert user records
INSERT INTO `user`(user_name,`password`) VALUE ("D001",AES_ENCRYPT('password1234',"hello"))
When I do this, I'm getting
1 row(s) affected, 1 warning(s)
Execution Time : 00:00:00:046
Transfer Time : 00:00:00:000
Total Time : 00:00:00:046
Warning Code : 1366
Incorrect string value: '\x92\xB7\xFF\xF3\xD1\xF6...' for column 'password' at row 1
---------------------------------------------------
with the above updation, row is getting inserted with just user_user but password is empty.
What I've done by referring similar problems are as follows
1) SET NAMES utf8mb4
2) ALTER TABLE user CONVERT TO CHARACTER SET utf8mb4;
3) Inside my.ini configuration file, I've added character_set_server=utf8mb4
Still I'm getting the same problem. How to resolve this?
UPDATE:
Problem is not specific to just password. What I wanted to do to scramble all required columns so that users can't see those columns normally.
So, I've used AES_ENCRYPT.
The issue is that you are storing binary data in a char,varchar or text column.
Either
store your data as hex:
INSERT INTO user(user_name,password) VALUE ("D001",HEX(AES_ENCRYPT('password1234',"hello")))
And use the UNHEX() function when you need to decrypt it; AES_DECRYPT(UNHEX(password),"hello")
Or
convert your password column to a type that can store binary data, such as the binary or varbinary type.

Why phone numbers in MySQL database are being truncated

I have created a database table in mySQL of which two column names are "landPhone" and "mobilePhone" to store phone numbers (in the format of: 123-456-8000 for land and 098-765-6601 for mobile). These two columns' data type are set to VARCHAR(30). The data have been inserted in the table. But after SQL query, I found the phone numbers have been truncated. It shows (above two data for example) only first 3 digits (123) for landPhone and only first 2 digits after removing the leading '0' (98) for mobilePhone.
Why this is happening ?
Phone numbers are not actually numbers; they are strings that happen to contain digits (and, in your case, dashes). If you try to interpret one as a number, two things typically happen:
Leading zeros are forgotten.
Everything from the first non-digit to the end of the string is stripped off.
That sounds exactly like the result you're describing. Even if you end up stuffing the result into a string field, it's too late -- the data has already been corrupted.
Make sure you're not treating phone numbers as integers at any point in the process.
You must use
insert into sample values('123-456-8000', '098-765-6601' )
instead of
insert into sample values(123-456-8000, 098-765-6601 )
see this SQLFiddle.
Thanks all for your solution. As cHao suspected, it was me who did the mistake. When I first time created the table, I declared the datatype of the phone columns as INT, later I corrected them to VARCHAR().
When I dropped the table and inserted the same data to the new table, it is working fine.
That sounds exactly like the result you're describing. Even if you end up stuffing the result into a string field, it's too late -- the data has already been corrupted. ..cHao
Question to understand: Why mySQL doesn't override the previous datatype with the new one ?