I have two MySQL instances. The 1st one truncates strings on insert when data is too long. The 2nd one raises an error:
ERROR 1406 (22001): Data too long for column 'xxx' at row 1
I want the 2nd one to truncate the data as well.
Is there any MySQL setting to manage this behavior?
You can disable STRICT_TRANS_TABLES and STRICT_ALL_TABLES. This allows the automatic truncation of the inserted string.
Quote from MySQL Documentation.
Strict mode controls how MySQL handles invalid or missing values in
data-change statements such as INSERT or UPDATE. A value can be
invalid for several reasons. For example, it might have the wrong data
type for the column, or it might be out of range. A value is missing
when a new row to be inserted does not contain a value for a non-NULL
column that has no explicit DEFAULT clause in its definition. (For a
NULL column, NULL is inserted if the value is missing.)
Reference: MySQL Server SQL Modes
If strict SQL mode is not enabled and you assign a value to a CHAR or VARCHAR column that exceeds the column's maximum length, the value is truncated to fit and a warning is generated. For truncation of nonspace characters, you can cause an error to occur (rather than a warning) and suppress insertion of the value by using strict SQL mode. See Section 6.1.7, “Server SQL Modes”.
How you can change it:
http://dev.mysql.com/doc/refman/5.7/en/sql-mode.html
Found two ways to disable strict mode:
add below to my.cnf
sql-mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
way is using mysql console.
SET ##global.sql_mode= '';
Please test them before running on production environment.
if you use cpanel ,
replace
sql-mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
into /usr/my.cnf
to
sql-mode=""
run
/etc/init.d/mysql restart
Related
I have mysql 5.6.27 installed on my two servers.
Database has a table which has a column type bigint(20) unsigned NOT NULL.
While inserting a string type value (like 1_2_3_4) in this column on one server it storing value 1 and showing a data truncation warning.
But if i am executing the same query on another server it showing the error message for data truncation and not letting the value inserted.
Just trying to understand why mysql is casting the value on one server but not on another.
Sounds like a configuration issue, specifically strict settings, see https://www.davidpashley.com/2009/02/15/silently-truncated/
I have two MySQL instances. The 1st one truncates strings on insert when data is too long. The 2nd one raises an error:
ERROR 1406 (22001): Data too long for column 'xxx' at row 1
I want the 2nd one to truncate the data as well.
Is there any MySQL setting to manage this behavior?
You can disable STRICT_TRANS_TABLES and STRICT_ALL_TABLES. This allows the automatic truncation of the inserted string.
Quote from MySQL Documentation.
Strict mode controls how MySQL handles invalid or missing values in
data-change statements such as INSERT or UPDATE. A value can be
invalid for several reasons. For example, it might have the wrong data
type for the column, or it might be out of range. A value is missing
when a new row to be inserted does not contain a value for a non-NULL
column that has no explicit DEFAULT clause in its definition. (For a
NULL column, NULL is inserted if the value is missing.)
Reference: MySQL Server SQL Modes
If strict SQL mode is not enabled and you assign a value to a CHAR or VARCHAR column that exceeds the column's maximum length, the value is truncated to fit and a warning is generated. For truncation of nonspace characters, you can cause an error to occur (rather than a warning) and suppress insertion of the value by using strict SQL mode. See Section 6.1.7, “Server SQL Modes”.
How you can change it:
http://dev.mysql.com/doc/refman/5.7/en/sql-mode.html
Found two ways to disable strict mode:
add below to my.cnf
sql-mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
way is using mysql console.
SET ##global.sql_mode= '';
Please test them before running on production environment.
if you use cpanel ,
replace
sql-mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
into /usr/my.cnf
to
sql-mode=""
run
/etc/init.d/mysql restart
I've a table with a field DECIMAL(9,2) and I've 2 servers running both mysql 5.7
If I run this code
INSERT INTO `money_accounts` (`balance`) VALUES (9999999999.99);
On one of the servers it got inserted and value is truncated, on the other it raise a Out of range value for column balance error.
My question is, what is the configuration value that makes this happen? or why it's happening in one server and not in the other?
The definition
DECIMAL(9,2)
means 9 digits of total precision, with 2 digits after the decimal place. The value
9999999999.99
has 12 total digits of precision, with 2 after the decimal place. Hence, the value is out of range. Rightfully, MySQL should have thrown an out of range error in both cases. In the case where it "worked," my guess is that truncation occurred.
By the way, you should be using DECIMAL(12,2) or wider to store the value in your question.
Update:
One possible explanation for why one of your servers was doing the insertion while the other failed is that the first has traditional mode turned off. Run the following on both servers:
SELECT ##SESSION.sql_mode
If you see output looking like
STRICT_TRANS_TABLES, STRICT_ALL_TABLES, ...
then the server is running in traditional mode, which means it won't truncate but will reject. To turn it off, try running
SET SESSION sql_mode=''
and them the insert should succeed (with truncation). But in any case, you should not be relying on truncation in production. If you need more precision, then widen the column.
Reference: Automatically trimming length of string submitted to MySQL
If we read the documentation: http://dev.mysql.com/doc/refman/5.7/en/out-of-range-and-overflow.html
When MySQL stores a value in a numeric column that is outside the
permissible range of the column data type, the result depends on the
SQL mode in effect at the time
If strict SQL mode is enabled, MySQL rejects the out-of-range value
with an error, and the insert fails, in accordance with the SQL
standard.
If no restrictive modes are enabled, MySQL clips the value to the
appropriate endpoint of the range and stores the resulting value
instead.
So it means that You should set necessary sql mode that will not fail when out-of-range error happens.
There is no config parameter about that.
And resolution of same issue we can see here that says that disabling STRICT_TRANS_TABLES and STRICT_ALL_TABLES modes can fix Your problem.
I was quite surprised when MySQL allowed me to insert a NULL into a field that was created with NOT NULL. I did some research and discovered how to enable strict mode. However, I am not quite sure what validation MySQL does when STRICT_ALL_TABLES is enabled.
The manual says:
Strict mode controls how MySQL handles input values that are invalid or missing. A value can be invalid for several reasons. (emphasis mine) For example, it might have the wrong data type for the column, or it might be out of range.
I understand what it considers missing and how it handles that. I am unclear what it considers invalid. I've done some testing and discovered the following:
strings that are too long are invalid
numbers that are out of range are invalid
NULLs for a non-NULL column are invalid
TRUE and FALSE always seem to be valid (they become 1 and 0 respectively)
invalid dates are invalid
zero dates are valid (additional modes can be enabled to change this behaviour)
floats in an integer field are valid (they get rounded)
letters in a number field are invalid
Does MySQL do any other validation checks other than what is mentioned above?
The manual says 'wrong data type for the column', but the only situation I see where this actually comes into play is letters in a number field. Are there other examples of data type errors?
Is there a list somewhere of exactly what checks MySQL performs?
EDIT: For the record, my application already does extensive validation. I am using strict mode as a last-chance, just-in-case check. If I forget to check something, I want it to fail fast rather than 'silently mangle my data'.
A good resource is to check the MySQL source, and read mysql-test/t/strict.test to see all the cases they test for after setting STRICT mode or TRADITIONAL mode (which is a superset of STRICT).
You did excellent research and testing, but there are a few more cases, such as:
Trying to insert an undefined ENUM value.
Trying to insert a default value for a NOT NULL column with no DEFAULT defined.
Trying to use CAST() to convert strings to integers, etc.
Conversion of VARCHAR to MEDIUMTEXT or LONGTEXT if you give a length greater than 65536.
Truncation of COMMENT strings for tables and columns.
Conversion of string to YEAR type.
Defining a SET or ENUM column with duplicate entries.
Also mysql-test/include/strict_autoinc.inc, because it tests for overflow when an auto-inc value grows too large.
There are a few other test files that use STRICT mode for specific tests, but I didn't examine them.
I grepped the source code of MySQL 5.5.28 (Community Server Edition) to find instances of STRICT_ALL_TABLES being used.
From what I saw, it looks like your list is already pretty complete, but below are a few more things I learned about the use of STRICT_ALL_TABLES. The first one is another validation, while the rest all deal with errors and warnings.
Geometry columns can't have a default value. (sql/field.cc:9394)
If a table comment is longer than 2048 characters, MySQL produces an error rather than truncating the comment with a warning. (sql/unireg.cc:231)
If a field comment is longer than 1024 characters, MySQL produces an error rather than truncating the comment with a warning. (sql/unireg.cc:739)
On an insert or update, if a duplicate value exists in a SET or ENUM MySQL produces an error rather than a warning. (sql/sql_table.cc:2503)
There are a bunch of operations that will abort on warning with STRICT_ALL_TABLES rather than proceeding with just a warning. These are too numerous for me to list, and the setting of the abort_on_warning flag is too far removed from the code that creates the warnings for me to easily document (or even understand) all of them. But if anybody wants to get their hands dirty in the source code, a few places to start would be sql/sql_update.cc:630 and sql/sql_insert.cc:841
If you try to create a TEXT column on a table, and give it a default value in MySQL, you get an error (on Windows at least). I cannot see any reason why a text column should not have a default value. No explanation is given by the MySQL documentation. It seems illogical to me (and somewhat frustrating, as I want a default value!). Anybody know why this is not allowed?
Windows MySQL v5 throws an error but Linux and other versions only raise a warning. This needs to be fixed. WTF?
Also see an attempt to fix this as bug #19498 in the MySQL Bugtracker:
Bryce Nesbitt on April 4 2008 4:36pm:
On MS Windows the "no DEFAULT" rule is an error, while on other platforms it is often a warning. While not a bug, it's possible to get trapped by this if you write code on a lenient platform, and later run it on a strict platform:
Personally, I do view this as a bug. Searching for "BLOB/TEXT column can't have a default value" returns about 2,940 results on Google. Most of them are reports of incompatibilities when trying to install DB scripts that worked on one system but not others.
I am running into the same problem now on a webapp I'm modifying for one of my clients, originally deployed on Linux MySQL v5.0.83-log. I'm running Windows MySQL v5.1.41. Even trying to use the latest version of phpMyAdmin to extract the database, it doesn't report a default for the text column in question. Yet, when I try running an insert on Windows (that works fine on the Linux deployment) I receive an error of no default on ABC column. I try to recreate the table locally with the obvious default (based on a select of unique values for that column) and end up receiving the oh-so-useful BLOB/TEXT column can't have a default value.
Again, not maintaining basic compatability across platforms is unacceptable and is a bug.
How to disable strict mode in MySQL 5 (Windows):
Edit /my.ini and look for line
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
Replace it with
sql_mode='MYSQL40'
Restart the MySQL service (assuming that it is mysql5)
net stop mysql5
net start mysql5
If you have root/admin access you might be able to execute
mysql_query("SET ##global.sql_mode='MYSQL40'");
Without any deep knowledge of the mySQL engine, I'd say this sounds like a memory saving strategy. I assume the reason is behind this paragraph from the docs:
Each BLOB or TEXT value is represented internally by a separately allocated object. This is in contrast to all other data types, for which storage is allocated once per column when the table is opened.
It seems like pre-filling these column types would lead to memory usage and performance penalties.
As the main question:
Anybody know why this is not allowed?
is still not answered, I did a quick search and found a relatively new addition from a MySQL developer at MySQL Bugs:
[17 Mar 2017 15:11] Ståle Deraas
Posted by developer:
This is indeed a valid feature request, and at first glance it might seem trivial to add. But TEXT/BLOBS values are not stored directly in the record buffer used for reading/updating tables. So it is a bit more complex to assign default values for them.
This is no definite answer, but at least a starting point for the why question.
In the mean time, I'll just code around it and either make the column nullable or explicitly assign a (default '') value for each insert from the application code...
"Support for DEFAULT in TEXT/BLOB columns"
is a
feature request in the MySQL Bugtracker (Bug #21532).
I see I'm not the only one who would like to put a default value in a TEXT column.
I think this feature should be supported in a later version of MySQL.
This can't be fixed in the version 5.0 of MySQL,
because apparently it would cause incompatibility and dataloss if anyone tried to transfer a database back and forth between the (current) databases that don't support that feature and any databases that did support that feature.
You can get the same effect as a default value by using a trigger
create table my_text
(
abc text
);
delimiter //
create trigger mytext_trigger before insert on my_text
for each row
begin
if (NEW.abc is null ) then
set NEW.abc = 'default text';
end if;
end
//
delimiter ;
Support for using expression as default values was added to MySQL 8.0.13, released 2018-10-22, and works for TEXT, JSON, BLOB and GEOMETRY.
You still cannot write :
create table foo(bar text default 'baz')
But you can now write:
create table foo(bar text default ('baz'))
Which achieve the same thing.
I normally run sites on Linux, but I also develop on a local Windows machine. I've run into this problem many times and just fixed the tables when I encountered the problems. I installed an app yesterday to help someone out and of course ran into the problem again. So, I decided it was time to figure out what was going on - and found this thread. I really don't like the idea of changing the sql_mode of the server to an earlier mode (by default), so I came up with a simple (me thinks) solution.
This solution would of course require developers to wrap their table creation scripts to compensate for the MySQL issue running on Windows. You'll see similar concepts in dump files. One BIG caveat is that this could/will cause problems if partitioning is used.
// Store the current sql_mode
mysql_query("set #orig_mode = ##global.sql_mode");
// Set sql_mode to one that won't trigger errors...
mysql_query('set ##global.sql_mode = "MYSQL40"');
/**
* Do table creations here...
*/
// Change it back to original sql_mode
mysql_query('set ##global.sql_mode = #orig_mode');
That's about it.
For Ubuntu 16.04:
How to disable strict mode in MySQL 5.7:
Edit file /etc/mysql/mysql.conf.d/mysqld.cnf
If below line exists in mysql.cnf
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
Then Replace it with
sql_mode='MYSQL40'
Otherwise
Just add below line in mysqld.cnf
sql_mode='MYSQL40'
This resolved problem.
This is a very old question but still it doesn't seems to have been answered properly. And, my this answer isn't actual answer to the question - "WHY can't a text column have a default value", but as it isn't possible to write long text in comment, and as my comment could help someone to prevent the error, here it is as a separate answer:
Some are saying that the error is occurring because of OS - Windows-Linux; but this isn't directly related to OS. (However, there may be differences in default settings of MySQL within different installers for different OSes, I am not sure.)
The main reason is the flag STRICT_TRANS_TABLES for sql_mode setting. if a value is not specified in INSERT statement for TEXT datatype column and if the flag exist in the sql_mode setting then MySQL is reporting an error; and if the flag doesn't exist then MySQL is only reporting a warning and inserts the record.
So, to prevent this error, one can remove the STRICT_TRANS_TABLES from sql_mode setting of MySQL. (He my need to reset the mode to the previous value if it can affect other operations on the database.)
According to the documentation of SQL mode in MySQL ...
For STRICT_TRANS_TABLES, MySQL converts an invalid value to the closest valid value for the column and inserts the adjusted value. If a value is missing, MySQL inserts the implicit default value for the column data type. In either case, MySQL generates a warning rather than an error and continues processing the statement. Implicit defaults are described in Section 11.6, “Data Type Default Values”.
... and documentation of Data Type Default Values ...
The BLOB, TEXT, GEOMETRY, and JSON data types cannot be assigned a default value.
... TEXT column can not have a default value, but if STRICT_TRANS_TABLES is removed from sql_mode then MySQL inserts empty string '' if no value is specified for TEXT column in INSERT statement.