I just wanted to know if somebody could explain this.
I was just testing my code and didn't check for empty input fields (I know I have to, but just testing).. In my database table, all the fields are NOT NULL, and I was expecting a exception because I wasn't inserting anything.. But it turns out that MySQL inserts all with blank values, also, from MySQL workbench is the same thing..
Is there a way to prevent this? (From a MySQL perspective)
This behavior, although atypical, is quite well documented:
Inserting NULL into a column that has been declared NOT NULL. For
multiple-row INSERT statements or INSERT INTO ... SELECT statements,
the column is set to the implicit default value for the column data
type. This is 0 for numeric types, the empty string ('') for string
types, and the “zero” value for date and time types. INSERT INTO ...
SELECT statements are handled the same way as multiple-row inserts
because the server does not examine the result set from the SELECT to
see whether it returns a single row. (For a single-row INSERT, no
warning occurs when NULL is inserted into a NOT NULL column. Instead,
the statement fails with an error.)
So, if you want to get an error, use VALUES() with a single row. Alternatively, define a trigger that does the check.
Why does MySQL work this way? I don't know, to differentiate itself from other databases and prevent ANSI-compatibility? More seriously, I assume that this a question of efficiency, and related to the fact that MySQL does not implement check constraints. The NOT NULL declaration is just an example of a check constraint, and these are not supported.
Related
(NOTE: I know this is an error that's commonly asked about, but most of the time, the issue is in an INSERT statement. I couldn't find a question on this website where this error happened during an UPDATE.)
I have a table in MySQL (InnoDB / v. 5.7.19) called RESULTS which has, among others, two columns called TYPE and STATUS. Both are of type ENUM, with PASS, FAIL and IGNORE being the supported values in both. I'm trying to run this UPDATE statement on that table, using Workbench (also tried the same directly on the DB machine, using the mysql command):
update `RESULTS` set `TYPE`='IGNORE' where `STATUS`='IGNORE';
I'm getting this error:
Error Code: 1136. Column count doesn't match value count at row 1
Changing the single quotes to double quotes didn't help. I'm able to run this query successfully:
select count(`TYPE`) from `RESULTS` where `STATUS`='IGNORE';
I'm probably making a silly mistake here, but can anyone point out what's wrong with the UPDATE statement?
As requested I am posting it as an answer.
The error basically is self-explanatory like performing an operation on set of attributes but the values provided in the query are not enough. But in your case, you are performing an update operation with all attributes and their values and still, this error appears it may be a case that there is some trigger is registered for this table probably on before/after the event, If that is the case you need to update or remove that trigger if no needed.
I just noticed that phpMyAdmin for some reason always seems to use numeric string values instead of integer values (i.e. '5' instead of 5) in its generated SQL queries for writing data to integer fields in MySQL databases, which I did not know was even valid SQL.
This made me curious, so I did some follow-up experiments myself, and could confirm that this is indeed allowed, even in DDL queries like:
ALTER TABLE MyTable ALTER COLUMN MyIntField SET DEFAULT '5'
(where, possibly needless to say, the "MyIntField" is of the "int" type)
and also in normal insert and update queries like:
INSERT INTO MyTable (MyIntField) VALUES ('5');
UPDATE MyTable SET MyIntField = '5' WHERE id = 1;
How/why is this allowed by MySQL? Databases normally care about types, and strings and integers are definitely different types?
Also, with this "feature", the same SQL injection sanitation procedure could be used for both string values and integer values (instead of integer validation testing input values for integer fields and quote-escape-testing input values for string fields separately), so my follow-up question is: Can I depend on the existence and support of this feature under all conditions in MySQL, and also in future versions of MySQL, i.e. is it documented somewhere officially?
MySQL is pretty lax about most things, by default fixing these errors by doing everything it can to not error out.
MySQL does have a strict mode though which would not allow that behaviour
http://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sql-mode-strict
For more info.
This 'feature' can also cause performance issues, comparing a sting to an index'ed int makes the index useless (in some cases, sometimes the optimiser can spot this and cast only the input)
Strict SQL Mode
Strict mode controls how MySQL handles invalid or missing values in
data-change statements such as INSERT or UPDATE. A value can be
invalid for several reasons. For example, it might have the wrong data
type for the column, or it might be out of range. A value is missing
when a new row to be inserted does not contain a value for a non-NULL
column that has no explicit DEFAULT clause in its definition. (For a
NULL column, NULL is inserted if the value is missing.) Strict mode
also affects DDL statements such as CREATE TABLE.
I would add, numeric strings is always safer than the opposite.
It will never lead to any critical problem if you use a numeric string to query a numeric field, i.e. update table where number='1' will work like a charm.
Whereas the opposite, if happens could be a disaster. If you compare a number with a string the latter is converted to a number. I.e. update table where string=0 will likely update all rows in the database while a numeric string won't cause any harm: update table where string='0' will just leave all strings alone.
I'm looking for a standard SQL "UPSERT" statement. A one call for insert and update if exists.
I'm looking for a working, efficient and cross platform call.
I've seen MERGE, UPSERT, REPLACE, INSERT .. ON DUPLICATE UPDATE but no statement meets the needs.
BTW I use MYSQL and HSQLDB for unitests. I understand that HSQLDB is limited and may not cover what I need, but I couldn't find a standard way even without it.
A statement that only MYSQL and HSQLDB will also be enough for now.
I've been looking around for a while and couldn't get an answer.
My table:
CREATE TABLE MY_TABLE (
MY_KEY varchar(50) NOT NULL ,
MY_VALUE varchar(50) DEFAULT NULL,
TIME_STAMP bigint NOT NULL,
PRIMARY KEY (MY_KEY)
);
Any idea?
The only solution that is supported by both MySQL and HSQLDB is to query the rows you intend to replace, and conditionally either INSERT or UPDATE. This means you have to write more application code to compensate for the differences between RDBMS implementations.
START TRANSACTION.
SELECT ... FOR UPDATE.
If the SELECT finds rows, then UPDATE.
Else, INSERT.
COMMIT.
MySQL doesn't support the ANSI SQL MERGE statement. It supports REPLACE and INSERT...ON DUPLICATE KEY UPDATE. See my answer to "INSERT IGNORE" vs "INSERT ... ON DUPLICATE KEY UPDATE" for more on that.
Re comments: Yes, another approach is to just try the INSERT and see if it succeeds. Otherwise, do an UPDATE. If you attempt the INSERT and it hits a duplicate key, it'll generate an error, which turns into an exception in some client interfaces. The disadvantage of doing this in MySQL is that it generates a new auto-increment ID even if the INSERT fails. So you end up with gaps. I know gaps in auto-increment sequence are not ordinarily something to worry about, but I helped a customer last year who had gaps of 1000-1500 in between successful inserts because of this effect, and the result was that they exhausted the range of an INT in their primary key.
As #baraky says, one could instead attempt the UPDATE first, and if that affects zero rows, then do the INSERT instead. My comment on this strategy is that UPDATEing zero rows is not an exception -- you'll have to check for "number of rows affected" after the UPDATE to know whether it "succeeded" or not.
But querying the number of rows affected returns you to the original problem: you have to use different queries in MySQL versus HSQLDB.
HSQLDB:
CALL DIAGNOSTICS(ROW_COUNT);
MySQL:
SELECT ROW_COUNT();
The syntax for doing an upsert in a single command varies by RDBMS.
MySQLINSERT…ON DUPLICATE KEY UPDATE
HSQLDBMERGE
PostgresINSERT…ON CONFLICT…
See Wikipedia for more.
If you want a cross platform solution, then you'll need to use multiple commands. First check for the existing row, then conditionally insert or update as appropriate.
I have a MySQL DB which is using strict mode so I need to fill all NOT NULL values when I insert a row. The API Im creating is using just DUPLICATE KEY UPDATE functionality to do both inserts/updates.
The client application complains if any NOT NULL attributes are inserted which is expected.
Basic example (id is primary key and theare are two fields that are NOT NULL aaa and xxx)
INSERT INTO tablename (aaa, xxx, id ) VALUES ( "value", "value", 1)
ON DUPLICATE KEY UPDATE aaa=VALUES(aaa), xxx=VALUES(xxx)
All good so far. Once it is inserted, the system would allow doing updates. Nevertheless, I get the following error when updating only one of the fields.
INSERT INTO tablename (aaa, id ) VALUES ( "newValue", 1)
ON DUPLICATE KEY UPDATE aaa=VALUES(aaa)
java.sql.SQLException: Field 'xxx' doesn't have a default value
This Exception is a lie as the row is already inserted and xxx attribute has "value" as value. I would expect the following sentence to be equivalent to:
UPDATE tablename SET aaa="newValue" WHERE id=1
I would be glad if someone can shed some light about this issue.
Edit:
I can use the SQL query in PhpMyAdmin successfully to update just one field so I am afraid that this is not a SQL problem but a driver problem with JDBC. That may not have solution then.
#Marc B: Your insight is probably true and would indicate what I just described. That would mean that there is a bug in JDBC as it should not do that check when the insert is of ON DUPLICATE type as there may be a default value for the row after all. Can't provide real table data but I believe that all explained above is quite clear.
#ruakh: It does not fail to insert, neither I am expecting delayed validation. One requirement I have is to have both insert/updates done using the same query as the servlet does not know if the row exists or not. The JAVA API service only fails to update a row that has NOT NULL fields which were already filled when the insert was done. The exception is a lie because the field DOES have a default value as it was inserted before the update.
This is a typical case of DRY / SRP fail; in an attempt to not duplicate code you've created a function that violates the single responsibility principle.
The semantics of an INSERT statement is that you expect no conflicting rows; the ON DUPLICATE KEY UPDATE option is merely there to avoid handling the conflict inside your code, requiring another separate query. This is quite different from an UPDATE statement, where you would expect at least one matching row to be present.
Imagine that MySQL would only check the columns when an INSERT doesn't conflict and for some reason a row was just removed from the database and your code that expects to perform an update has to deal with an exception it doesn't expect. Given the difference in statement behaviour it's good practice to separate your insert and update logic.
Theory aside, MySQL puts together an execution plan when a query is run; in the case of an INSERT statement it has to assume that it might succeed when attempted, because that's the most optimal strategy. It prevents having to check indices etc. only to find out later that a column is missing.
This is per design and not a bug in JDBC.
Can't find a straight answer to the following problem / question.
I created a database, and one of the fields is of type ENUM and restriction NOT NULL.
It works fine, but when I try to insert a record with a value that is not allowed for this field, the records is not rejected but inserted anyway with the field empty. That is, I think that it is an empty string (""), since I added the restriction NOT NULL.
I want the record of course rejected as a whole. How can I achieve that.
I assume sample code is not necessary since it is common knowledge.
If you insert an incorrect value into an ENUM column or if you force a value into an ENUM column with IGNORE, it is set to the reserved enumeration value of 0, which is displayed as an empty string in string context.
http://dev.mysql.com/doc/refman/5.1/en/constraint-enum.html
Try this:
SET sql_mode = 'STRICT_ALL_TABLES'; or STRICT_TRANS_TABLES
Depending on if your table is transactional or nontransactional. This should take care of it through mySQL and cause it to abort and rollback with an integrity constraint before the method even tries to operate.
I found that you should enable strict mode in mysql. To do this you should:
edit /etc/mysql/my.cnf
add/edit sql-mode = STRICT_ALL_TABLES
restart MySQL
This is solution on MySQL level. If you try inserting an empty value in an ENUM field, it should now be rejected.