I have been using MySQL to create a database and sometimes warnings come up rather than errors. An example is when I entered a page of full stops as a value. I do not really understand the point of these are why they appear instead of errors. Surely every incorrect value entered into a database should come up with an error, not just some. Please could somebody explain?
If you declared a column to be an integer but are attempting to insert a string, MySQL may perform implicit type conversion and emit an warning. See this question for a detailed example.
If you want to treat all warnings as errors, you should try strict mode.
Related
My PHP/mySQL backend hosted on an external site has been running fine since 2014. Recently, it started throwing up "field has no default value" errors.
I checked the config and found STRICT_TRANS_TABLES, which gives these errors for fields with no default in some cases.
My question is whether or not it is safe for me to remove this config value. It's mySQL 5.5.5-10.3.12-MariaDB.
Alternatively I could give everything default values, but I don't know which of these solutions is more likely to cause the existing codebase to stop working properly.
I encourage use of strict mode in MySQL because if you disable strict mode, you risk causing some unwanted effects, such as:
If you insert a value to a column where it can't fit, the value is silently truncated. Like inserting a long string into a shorter varchar column, or inserting a big integer into an INT column. This leads you to potentially have bogus values in your database. I prefer these cases to be errors, to prevent such bogus values.
Non-strict mode allows nonsensical dates, like 0000-00-00. There is no such date in the calendar. I'd rather this value not be allowed. If I need to symbolize an absence of a value, I'll use NULL.
Will these cases affect your app? There's no way I or anyone else on Stack Overflow can predict that. You need to test it yourself.
My server is using a MySQL DB, connecting to it via the C++ connector. I'm nearing production and I've been spending some time trying to break things as part of hardening the server.
One action item I had was to see what would happen if I execute a statement with a string that is longer than VARCHAR. For example, if I have a column defined as VARCHAR(4) and then set it to the string "hello".
This of course throws an exception with the error code 1406 (Data too long for column).
What I was wondering was if there was a good or standard way to defend against this? Obviously one thing is to check against the string length and truncate manually. I can do this, however there are many tables and several columns with VARCHAR. So my worry is updating server code if one of the columns using VARCHAR has its length increased (i.e. code maintainability)
Note that the server does do some validation up front. I'm just trying to defend against a subtle bug or corner case that lets something slip through.
A couple of other options on the table are to disable strict so it will give a warning and truncate or to convert VARCHAR to TEXT.
I was wondering a few things.
Is there a recommended method to handle this situation?
What are the disadvantages of disabling strict?
Is it worth (and is it possible) to query the DB at runtime the VARCHAR lengths? Note that I'm using the C++ connector. I suppose I could also write a tool that is run before compiling which would extract out VARCHAR lengths from the SQL code used to generate tables. But that then makes me wonder is I'm over engineering this.
I'm just sorting through the possible approaches now and thought I'd seek advice from those with more experience with MySQL.
As an experience database engineer I would recommend a combination of the follow two strategies:
1) If you that know that a there is a chance, however small, that data for your varchar(4) could go higher than 4 then make the varchar field larger than 4. For example, if you expect that the field can go as high as 8 then set the field to varchar(10). The beauty of using a varchar field instead of a char is that a varchar will only use whatever storage it needs.
2) If there is a real issue with data constantly being larger than the varchar field length then you should right your own exception handler to trap for the 1406 error. For the exception to work properly you will need to come up with some type of strategy on exactly how you want to handle the exception. For example, you could send an error to the user and ask them to fix the problem, you could accept the data but truncated it so it fits into the field, or you could send the error to a log file to get fixed at a later time.
If I have a table with some varchar columns, whose lengths will obviously be limited, then I would have to show on the front-end whenever insertion of too large values fails. For example, if the limit on the name column is 20, but someone enters a name that is 30 characters long, I should notify them of the error.
This gets to be a lot of work when the application becomes big.
What I would like, to make life a bit easier, and skip taking care of individual limits for every step of the users' journey, is to just carry on with the normal functioning of the application, but show them a warning that their data was not saved in entirety because it was too long. So if MySQL would provide some method that would allow me to ask if all data was saved in its entirety, or some strings were truncated due to their respective varchar fields being shorter (or maybe a property on the MySQLi object that I can check), then my main method for saving data in the database could always check that after any inserts or updates have been executed and just issue a warning on the next page load.
Does MySQL provide such functionality?
Sure you can. MySQL throws a warning, when data is truncated.
You can check is any warning occured by checking ##warning_count
SELECT ##warning_count;
Or
SHOW COUNT(*) WARNINGS;
To check what warning has occured:
SHOW WARNINGS [LIMIT [offset,] row_count]
More info:
http://dev.mysql.com/doc/refman/5.0/en/show-warnings.html
I was quite surprised when MySQL allowed me to insert a NULL into a field that was created with NOT NULL. I did some research and discovered how to enable strict mode. However, I am not quite sure what validation MySQL does when STRICT_ALL_TABLES is enabled.
The manual says:
Strict mode controls how MySQL handles input values that are invalid or missing. A value can be invalid for several reasons. (emphasis mine) For example, it might have the wrong data type for the column, or it might be out of range.
I understand what it considers missing and how it handles that. I am unclear what it considers invalid. I've done some testing and discovered the following:
strings that are too long are invalid
numbers that are out of range are invalid
NULLs for a non-NULL column are invalid
TRUE and FALSE always seem to be valid (they become 1 and 0 respectively)
invalid dates are invalid
zero dates are valid (additional modes can be enabled to change this behaviour)
floats in an integer field are valid (they get rounded)
letters in a number field are invalid
Does MySQL do any other validation checks other than what is mentioned above?
The manual says 'wrong data type for the column', but the only situation I see where this actually comes into play is letters in a number field. Are there other examples of data type errors?
Is there a list somewhere of exactly what checks MySQL performs?
EDIT: For the record, my application already does extensive validation. I am using strict mode as a last-chance, just-in-case check. If I forget to check something, I want it to fail fast rather than 'silently mangle my data'.
A good resource is to check the MySQL source, and read mysql-test/t/strict.test to see all the cases they test for after setting STRICT mode or TRADITIONAL mode (which is a superset of STRICT).
You did excellent research and testing, but there are a few more cases, such as:
Trying to insert an undefined ENUM value.
Trying to insert a default value for a NOT NULL column with no DEFAULT defined.
Trying to use CAST() to convert strings to integers, etc.
Conversion of VARCHAR to MEDIUMTEXT or LONGTEXT if you give a length greater than 65536.
Truncation of COMMENT strings for tables and columns.
Conversion of string to YEAR type.
Defining a SET or ENUM column with duplicate entries.
Also mysql-test/include/strict_autoinc.inc, because it tests for overflow when an auto-inc value grows too large.
There are a few other test files that use STRICT mode for specific tests, but I didn't examine them.
I grepped the source code of MySQL 5.5.28 (Community Server Edition) to find instances of STRICT_ALL_TABLES being used.
From what I saw, it looks like your list is already pretty complete, but below are a few more things I learned about the use of STRICT_ALL_TABLES. The first one is another validation, while the rest all deal with errors and warnings.
Geometry columns can't have a default value. (sql/field.cc:9394)
If a table comment is longer than 2048 characters, MySQL produces an error rather than truncating the comment with a warning. (sql/unireg.cc:231)
If a field comment is longer than 1024 characters, MySQL produces an error rather than truncating the comment with a warning. (sql/unireg.cc:739)
On an insert or update, if a duplicate value exists in a SET or ENUM MySQL produces an error rather than a warning. (sql/sql_table.cc:2503)
There are a bunch of operations that will abort on warning with STRICT_ALL_TABLES rather than proceeding with just a warning. These are too numerous for me to list, and the setting of the abort_on_warning flag is too far removed from the code that creates the warnings for me to easily document (or even understand) all of them. But if anybody wants to get their hands dirty in the source code, a few places to start would be sql/sql_update.cc:630 and sql/sql_insert.cc:841
My specific problem is solved before completing the question. But I still have a general question. It might be a duplicate but I am sorry I have been unable to find it on SO or elsewhere.
I have a table named reports. There is one row containing data. When I was trying to execute an update query I get an error:
error 1064: There is an error in your sql syntax near 'table='defect',Filter='' ,dtFilter='',query='Select allfields from defect' ,`da' at line 1
My question: is there a way to get my specific error? As it was just telling that there is error in SQL syntax but nothing about what type of error is.
Do I always have to google for such errors or there exists some technique to get their exact specific description locally?
Background (optional reading): after normal googling I got hint while posting question for better googling and get solved my specific problem as I came to know that it was problem of reserve words table and datetime. The problem solved when I put quotes around those words. This usually happens with me. But my general problem is still there...
It my be of no use now but I am sharing my schema here on sqlfiddle it might be useful
MySQL can't tell you that you are using a reserved word in wrong place (or your place is correct but used word is a reserve word, it cant differentiate) , it only only sees what you enter, it can't read the mind, so it can't know that you accidentally used (say) order as a field name, it'll just see a valid part of a query in the wrong place.e.g... a syntax error. It also won't report on ALL syntax errors in a query, because one syntax error renders the whole thing wrong. it can't skip over a word and keep parsing.
Nope - it just tells you the position, which hopefully is a decent clue.