The following sql query (generated by Hibernate):
SELECT DISTINCT foreign_id FROM customer WHERE foreign_id IS NOT NULL ORDER BY name
Returns this error message:
Error Code: 3065. Expression #1 of ORDER BY clause is not in SELECT list, references column 'name' which is not in SELECT list; this is incompatible with DISTINCT
This is because the mysql mode ONLY_FULL_GROUP_BY is set on the mysql server. We're not allowed to tinker with the sql modes.
Is there a way to tell Hibernate to include every necessary column in the select list? Is there another way to avoid this error message?
I had the same problem and before finding the answer I first landed here. Just to help others I will post the link to the other forum where I found the answer.
How to avoid this error message ? Disable ONLY_FULL_GROUP_BY in hibernate
Thanks
Related
In a ColdFusion application I have a query with several joins and I need one MAX number from one of the tables. I am getting all tangled up in 'only_full_group_by'. I cannot disable it in etc or in the Administrator, because I am on a shared system and have no access to those. I'd like to disable it at run time. Here's a simplified version of the query:
<cfquery name = 'test' datasource = 'whatever'>
select PersActExt,PersActPaid,PersActMdate,PersActbl,
PersTrId,PersTrMas,PersTrSub,PersTrCode,
MAX(PersTrPaid), MAX(PersTrDate)
from PersActiv
left join PersTrans on
PersTrId = PersActId and
PersTrMas = PersActMas and
PersTrSub = PersActSub and
PersTrCode = PersActCode
where PersActMas = 'bill'
group by PersTrId,PersTrMas,PersTrSub,PersTrCode
</cfquery>
The fact is the condition PersActMas = 'bill' reduces this to one record per Person in the PersAct table. I suppose I could put a MAX on all those Persact type fields, but there are other tables that also have to be joined in; and frankly I don't want a MAX on everything just to pick up one Max number in PersTr.
Is there a way to disable the ONLY_FULL_GROUP_BY mode within the query? Or better yet within the application? Or can anyone suggest another way around this problem. I know I can split this up into several queries and then stitch everything back together in ColdFusion, but that's a lot of code to get around this bad MySql mode.
Thinking laterally about the problem, could you restructure your query to join the PersAct table to a pre-aggregated subquery? This removes the need to perform a grouping on the outer query and running up against the limitation of your setup.
SELECT
PersActExt,
PersActPaid,
PersActMdate,
PersActbl,
PersTrId,
PersTrMas,
PersTrSub,
PersTrCode,
MaxPersTrPaid,
MaxPersTrDate
FROM
PersActiv
LEFT JOIN
(
SELECT
PersTrId,
PersTrMas,
PersTrSub,
PersTrCode,
MAX(PersTrPaid) AS MaxPersTrPaid,
MAX(PersTrDate) AS MaxPersTrDate
FROM
PersTrans
GROUP BY
PersTrId,
PersTrMas,
PersTrSub,
PersTrCode
) AS PersTr ON
PersTrId = PersActId and
PersTrMas = PersActMas and
PersTrSub = PersActSub and
PersTrCode = PersActCode
WHERE PersActMas = 'bill'
Edit:
For those wanting to know how to remove the 'ONLY_FULL_GROUP_BY' on an ad hoc basis, you need to add the below snippet to your query.
SET SESSION sql_mode = sys.list_drop(##session.sql_mode, 'ONLY_FULL_GROUP_BY');
This will only work if the DB user used in ColdFusion has the correct permissions and the option to allow multiple queries in a cfquery call is set.
Tested the query on MySQL 5.7 here: https://dbfiddle.uk/E1em1_k2
I tested with your query, with the last line as you wrote it:
...
group by PersTrId,PersTrMas,PersTrSub,PersTrCode;
This resulted in the error due to the ONLY_FULL_GROUP_BY:
Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'test.PersActiv.PersActExt' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
Then I tested with an alternative last line:
...
group by PersActId,PersActMas,PersActSub,PersActCode;
This works with the ONLY_FULL_GROUP_BY at the default value. See the demo for proof. Note that I had to do my alternative query first, because the dbfiddle aborts when it gets an error.
The reason is that MySQL 5.7 introduced the capability to infer functional dependencies when using GROUP BY on the primary key.
https://dev.mysql.com/doc/refman/5.7/en/group-by-handling.html
SQL:1999 and later permits such nonaggregates per optional feature T301 if they are functionally dependent on GROUP BY columns: If such a relationship exists between name and custid, the query is legal. This would be the case, for example, were custid a primary key of customers.
MySQL 5.7.5 and later implements detection of functional dependence.
Since the four columns in the GROUP BY are in fact the primary key of the PersActive table, MySQL can tell that other columns of that table are guaranteed to have one value per grouping.
There are multiple rows in PersTrans, but the functional dependencies are still analyzed, so the columns that are known to be equal to primary key columns of PersActiv are still okay to use in the query without aggregation. The other columns of PersTrans that are not functionally dependent are already in MAX() expressions in your query, which also satisfies the requirement.
I conclude that functional dependency analysis works in MySQL 5.7, and this would be an effective workaround in the case you show.
You wrote:
I'm used to a system where if there is an aggregated field and some fields are not aggregated, the first or last value of that the non-aggregated field is automatically used.
In my experience, this includes only MySQL prior to 5.7, or SQLite. It's illegitimate in the ANSI/ISO SQL standard, and other SQL implementations conform to the standard.
MySQL 5.7 does not have a "bad sql mode" — it has been corrected to enforce the rule per the SQL standard, like most other brands of SQL also do.
Back to your original question:
Is there a way to disable the ONLY_FULL_GROUP_BY mode within the query? Or better yet within the application?
You can change the sql mode globally, either in the my.cnf options file, or using SET GLOBAL sql_mode='...';. I would do both, because if you change options dynamically with SET GLOBAL, they are reset to whatever is in the options file the next time you restart the MySQL Server.
You can also change the sql mode in a session with SET SESSION sql_mode='...'; and it takes effect for the remainder of the current session. But SET SESSION is a separate statement that you would have to execute in the session prior to your query. It can't be combined with a query.
There is an option in MySQL 8.0 to use the "hint" syntax to set certain options for the scope of a single SQL query. But in your case, you're using MySQL 5.7, so you'd have to upgrade to get this feature.
A similar feature exists in Percona Server 5.6 and later to set a variable for one statement. This does not apply if you use stock MySQL. You'd have to use the Percona branch of MySQL.
Regardless, I recommend you do not change the sql mode. It's proper that it behaves the way it does, because it prevents queries that may inadvertently produce wrong results.
Hello im going through a tutorial on SQL joins and this statement below is throwing "sql_mode=only_full_group_by" error. I copied this directly from the tutorial and its working for him. Do I just need to disable this mode in the terminal?
SELECT *
FROM customers
JOIN orders
ON customers.id = orders.customer_id
GROUP BY orders.customer_id;
If the ONLY_FULL_GROUP_BY SQL mode is enabled (which it is by default), MySQL rejects queries for which the select list, HAVING condition, or ORDER BY list refer to nonaggregated columns that are neither named in the GROUP BY clause nor are functionally dependent on them. [1]
You can disable this, but it really makes sense to leave it enabled. Think about the following case:
id | type
---|-----
1 | foo
2 | foo
3 | bar
If you used a similar query to yours, select * from table group by type, what would you expect the output to be? Perhaps you expect two rows, foo and bar, but what do you expect to be in the id column, namely for the foo record? Should we just choose the earliest record? The latest record? A random record? This is what the error is telling you - instead of having the database engine choose for you, you must be explicit.
So, in your example, you need to think about what the resulting data should look like. Maybe instead of select * you want select customers.name, count(*) ..., etc.
The reason this works for the tutorial you're following is either they don't have only_full_group_by enabled or they're using a MySql version older than 5.7.5 where this was introduced. In this case, the server is free to choose any value from each group, which is probably not what you want.
Finally, if you want your console to behave in the same way your tutorial is behaving, you can disable ONLY_FULL_GROUP_BY in the console by:
mysql > SET GLOBAL sql_mode=(SELECT REPLACE(##sql_mode,'ONLY_FULL_GROUP_BY',''));
[1] https://dev.mysql.com/doc/refman/5.7/en/group-by-handling.html
I need MySQL/MariaDB to throw error messages instead of warning messages.
Reason: In my development environment, warnings are ignored in SQL/PHP (and both SQL and PHP continue processing), therefore I can't detect any issues when testing. In production, however, all warnings crash the query, thus terminating the API, and returning a "failure" message to users.
This is a very bad result and caused many headaches in the past.
Here is a very specific scenario that caused many problems:
I have a table "testtable" with two columns "pk" and "bRequiredBoolean", where pk is the primary key (thus auto increments) and bRequiredBoolean HAS NO DEFAULT VALUE:
CREATE TABLE `test`.`testtable` ( `pk` INT NOT NULL AUTO_INCREMENT , `bRequiredBoolean` BOOLEAN NOT NULL , PRIMARY KEY (`pk`)) ENGINE = InnoDB;
In my development server, I can run the following query:
INSERT INTO `testtable`() VALUES ()
and receive the following warning:
Warning: #1364 Field 'bRequiredBoolean' doesn't have a default value
In my development server, the entry IS INSERTED and the API continues executing.
In my production server, the entry ISN'T INSERTED and the API crashes.
I know that the development/testing servers should be identical to prevent issues like this, but I currently don't have the funds for extra licenses, so I would like an alternative solution to throw error messages instead of warnings to catch scenarios like the one mentioned above.
Important note: I don't want to insert any extra code within each query, I'd prefer to edit some configuration file. For instance, I don't want to add the following code TO EVERY SINGLE QUERY:
SHOW COUNT(*) WARNINGS
I'd rather edit some configuration file so it will always throw an error, even if I do ad-hoc queries directly through the GUI.
I found other similar questions, but nobody has responded to those threads:
MariaDB shows warnings instead of errors
Throw error on mysql warning pdo
Any help is much appreciated.
Thanks to Barmar's comment, I found the solution to my problem.
You can check your SQL_Mode via:
SELECT ##SQL_MODE;
Apparently, there is a SQL_Mode setting that specifically throws errors for all warnings called "Traditional"
You can view the official mySQL website for more info regarding SQL_Mode values: https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sql-mode-full
So I ran the following query:
SET GLOBAL sql_mode = 'TRADITIONAL';
After running the query, I executed the same query as before:
INSERT INTO `testtable`() VALUES ()
and received the following error:
Error
SQL query: Copy
INSERT INTO `testtable`() VALUES ();
MySQL said: Documentation
#1364 - Field 'bRequiredBoolean' doesn't have a default value
This is exactly what I needed. Thank you so much #Barmar!
(NOTE: I know this is an error that's commonly asked about, but most of the time, the issue is in an INSERT statement. I couldn't find a question on this website where this error happened during an UPDATE.)
I have a table in MySQL (InnoDB / v. 5.7.19) called RESULTS which has, among others, two columns called TYPE and STATUS. Both are of type ENUM, with PASS, FAIL and IGNORE being the supported values in both. I'm trying to run this UPDATE statement on that table, using Workbench (also tried the same directly on the DB machine, using the mysql command):
update `RESULTS` set `TYPE`='IGNORE' where `STATUS`='IGNORE';
I'm getting this error:
Error Code: 1136. Column count doesn't match value count at row 1
Changing the single quotes to double quotes didn't help. I'm able to run this query successfully:
select count(`TYPE`) from `RESULTS` where `STATUS`='IGNORE';
I'm probably making a silly mistake here, but can anyone point out what's wrong with the UPDATE statement?
As requested I am posting it as an answer.
The error basically is self-explanatory like performing an operation on set of attributes but the values provided in the query are not enough. But in your case, you are performing an update operation with all attributes and their values and still, this error appears it may be a case that there is some trigger is registered for this table probably on before/after the event, If that is the case you need to update or remove that trigger if no needed.
Following is a simple SQL query:
SELECT * FROM *table_name*
GROUP BY *column_name*
In my system I have MySQL 5.5. It is working absolutely fine.
Whereas in my friend's system he have MySQL 5.7, and he is getting the following error:
ERROR 1055 (42000): Expression #1 of SELECT list is not in GROUP BY
clause and contains nonaggregated column 'testdb.assetentry.entryId'
which is not functionally dependent on columns in GROUP BY clause;
this is incompatible with sql_mode=only_full_group_by
It is clearly visible this is happening because the versions are different.
But what I want to know is the reason behind this.
Can any one please explain.
First of all, please read Group by clause in mySQL and postgreSQL, why the error in postgreSQL?
It is not SQL Standard behaviour.
12.16.3 MySQL Handling of GROUP BY
To disable the MySQL GROUP BY extension and enable standard SQL behavior, enable the ONLY_FULL_GROUP_BY SQL mode. In this case, columns not named in the GROUP BY clause cannot be used in the select list or HAVING clause unless enclosed in an aggregate function.
It looks like on second server you have acitivated ONLY_FULL_GROUP_BY mode.
SELECT ##sql_mode;
You could simulate this behaviour on your MySQL 5.5:
SET SESSION sql_mode = 'ONLY_FULL_GROUP_BY';
SELECT *
FROM tab
GROUP BY col;
-- tab.col2' isn't in GROUP BY
SqlFiddleDemo
From MySQL 5.7:
Implementation of the ONLY_FULL_GROUP_BY SQL mode has been made more
sophisticated, to no longer reject deterministic queries that
previously were rejected. In consequence, ONLY_FULL_GROUP_BY is now
enabled by default, to prohibit nondeterministic queries containing
expressions not guaranteed to be uniquely determined within a group.