I have a table in a database mysql (5.7.21) like this:
+----------+--------------+-----------+-----------+
| id_price | id_reference | price_usd | unix_time |
+----------+--------------+-----------+-----------+
And I need to extract the average price (price_usd) grouped by week of year, or month (unix_time).
I prepare this query:
SELECT CONCAT(WEEKOFYEAR(FROM_UNIXTIME(unix_time)),
'-',
YEAR(FROM_UNIXTIME(unix_time))) as date,
AVG(price_usd) AS "model"
FROM price_avg
INNER JOIN reference ON reference.id_reference=price_avg.id_reference
WHERE price_avg.id_reference=1
GROUP BY WEEKOFYEAR(FROM_UNIXTIME(unix_time)),
YEAR(FROM_UNIXTIME(unix_time)),
price_avg.id_reference
ORDER BY unix_time ASC
The inner join is useful to get the name of the product having the the id.
I get this error:
#1055 - Expression #1 of SELECT list is not in GROUP BY clause and
contains nonaggregated column 'name_of_db.price_avg.unix_time' which is not
functionally dependent on columns in GROUP BY clause; this is
incompatible with sql_mode=only_full_group_by
I cannot change the settings of MySQL (I can't disable ONLY_FULL_GROUP_BY mode or anything else).
How do I have to change the query to extract the data in MySQL 5.7.21?
Thanks in advance.
You can use sub query so that it will be a full group by.
Select `date`,
AVG(price_usd) AS "model"
From (
SELECT CONCAT(WEEKOFYEAR(FROM_UNIXTIME(unix_time)),
'-',
YEAR(FROM_UNIXTIME(unix_time))) as `date`,
price_usd
FROM price_avg
INNER JOIN reference ON reference.id_reference=price_avg.id_reference
WHERE price_avg.id_reference=1
) t
GROUP BY `date`
ORDER BY substring(`date`, -4), substring(`date`, 1, 2) ASC;
Result:
date model
48-1998 11.99
36-2001 19.99
I solved with this query:
SELECT DATE_FORMAT(FROM_UNIXTIME(unix_time),'%Y-%m') as date, AVG(price_usd) AS model FROM price_avg INNER JOIN reference ON reference.id_reference=price_avg.id_reference WHERE price_avg.id_reference=1
GROUP BY date, price_avg.id_reference ORDER BY date ASC
I have week and year inverted but I can resolve in client enviroment!
Thank you all.
Hit this same problem, reported it as a possible bug on MySQL:
https://bugs.mysql.com/bug.php?id=90792&thanks=4
From the initial response, it sounded like it might be treated as a bug and fixed, but I think the follow-up suggests that GROUP BY expressions (instead of columns) aren't part of the SQL standard, and that determining if a complex expression is completely derived from GROUPed expressions is difficult and is, for now at least, something they decided not to pursue:
https://mysqlserverteam.com/when-only_full_group_by-wont-see-the-query-is-deterministic/
There are some workarounds in the meantime.
Related
I'm attempting to sort an aggregate column, which contains some zero values. I need the zero values to be last.
For non-aggregate columns I can do something like this (simplified example query):
SELECT age FROM books
ORDER BY
age = 0,
age ASC
However, for aggregate columns I'm getting an error as the column doesn't exist:
SELECT avg(age) as avg_age FROM books
GROUP BY book.type
ORDER BY
avg_age = 0,
avg_age ASC
The error is:
SQLSTATE[42S22]: Column not found: 1247 Reference 'avg_age' not supported (reference to group function)
I totally appreciate why this is happening, but I wasn't able to find a workaround, any tips?
There seams to be a (old) related bug report
[21 Mar 2016 9:22] Jiří Kavalík
Description: When using alias to aggregated column in ORDER BY only
plain alias is allowed, using it in any expression returns error.
http://sqlfiddle.com/#!9/e87bb/7
Workarounds:
- select the expression and use its alias
- use a derived table and order the outer one
How to repeat: create table t(a int);
-- these work select sum(a) x from t group by a order by x; select sum(a) x from t group by a order by sum(a); select sum(a) x from t
group by a order by -sum(a);
-- this one wrongly gives "Reference 'x' not supported (reference to group function)" select sum(a) x from t group by a order by -x;
source
You would have to write, this is better as the query is then also ANSI/ISO SQL standard valid meaning the query is most likely better portable between most databases vendor software.
SELECT
avg(books.age) as avg_age
FROM books
GROUP BY books.type
ORDER BY
avg(books.age) = 0
, avg(books.age) ASC
see demo this bug is fixed in MySQL 8.0 see demo
Try repeating the code
SELECT avg(age) as avg_age
FROM books
GROUP BY book.type
ORDER BY avg(age) = 0, avg(age) ASC
After Updating to MySQL 5.7.11 we are getting a number of errors related to the sql_mode including only_full_group_by.
Research shows we can turn this off but it would probably be better to know how to achieve the intended goal with properly structured SQL statements.
Error:
#1140 - In aggregated query without GROUP BY, expression #1 of SELECT list contains nonaggregated column 'tblslideshow.slideImage'; this is incompatible with sql_mode=only_full_group_by
Query:
SELECT
tblslideshow.slideImage,
COUNT(tblslideshow.slideID) AS countVal
FROM
tblslideshow
WHERE
tblslideshow.parentID = 3424
The goal is to return a list of values but also a Count for all returned records.
What's the best way to achieve this goal without having to change the MySQL my.ini ? And why is it a Group By error is we're not even grouping? Or is Grouping part of the proper solution?
Expected Result should be:
slideImage | countVal
Image1.jpg | 3
Image2.jpg | 3
Image3.jpg | 3
Edit: Since we are using PHP and MySQLi it may be wiser to exclude the Count() and just use "mysqli_num_rows"
When you use Aggregate function, you need to add non-aggregate columns on Group by clause.
SELECT
tblslideshow.slideImage,
COUNT(tblslideshow.slideID) AS countVal
FROM
tblslideshow
WHERE
tblslideshow.parentID = 3424
Group by tblslideshow.slideImage
EDIT
You can try to use subquery on select
SELECT t.slideImage,
(
SELECT
COUNT(tblslideshow.slideID) AS countVal
FROM
tblslideshow
WHERE
tblslideshow.parentID = 3424
) countVal
FROM tblslideshow t
WHERE t.parentID = 3424
What you are looking for is COUNT OVER, available as of MySQL 8.0:
SELECT
tblslideshow.slideImage,
COUNT(*) OVER () AS countVal
FROM
tblslideshow
WHERE
tblslideshow.parentID = 3424;
This keeps the single rows and adds the aggregate value to them.
I have a question similar to the one found here: How to find rows in SQL that start with the same string (similar rows)?, and this solution works in MySQL 5.6 but not 5.7.
I have a database (t) with multiple columns, the important ones being id and filepath, and what I am trying to accomplish is retrieving all the file paths which have the same last 5 characters. The following works in MySQL5.6, and the second SELECT works fine in 5.7:
SELECT id, filepath FROM t
WHERE SUBSTRING(filepath, -5) IN
(
SELECT SUBSTRING(filepath, -5)
FROM t
GROUP BY SUBSTRING(filepath, -5)
HAVING COUNT(*) > 1
)
But when I try to run it on 5.7 I get the error
Expression #1 of HAVING clause is not in GROUP BY clause and contains
nonaggregated column 't.filepath' which is not functionally dependent on
columns in GROUP BY clause; this is incompatible with
sql_mode=only_full_group_by
Sample data:
id filepath
1 /Desktop/file1.txt
2 /Desktop/file2.txt
3 /Desktop/file1.txt
and I would want to return the rows with id 1 and 3. How can I fix this for MySQL5.7?
EDIT: Also can anybody point me in the right direction for the SQL to remove the duplicates? So I would want to remove the entry for id 3 but keep the entry for id 1 and 2.
Please read the mysql documentation on the subject GROUP BY and sql_mode only_full_group_by (like your error message says):
https://dev.mysql.com/doc/refman/5.7/en/group-by-handling.html
I think changing the inner query to this might fix the problem:
SELECT SUBSTRING(filepath, -5) AS fpath
FROM t
GROUP BY fpath
HAVING COUNT(fpath) > 1
Edit:
As to your question of why adding the "AS fpath" works:
Adding the alias "fpath" is just a clean way to do this. The point of ONLY_FULL_GROUP_BY is that each field you use in the SELECT, HAVING, or ORDER BY must also be in the GROUP BY.
So I added the fpath-alias for multiple reasons:
For performance: The query you wrote had SUBSTRING(filepath, -5) twice, which
is bad for performance. Mysql has to execute that SUBSTRING call twice,
while in my case it has to do it only once (per row).
To fix the group-by issue: You had COUNT() in the having, but "" was not in your GROUP BY statement (I'm not even sure whether that would be possible). You had to count "something", so since "fpath" was in your SELECT and in your GROUP BY, using that as your COUNT() would fix the problem.
I prefer not to put subqueries in an IN() predicate because MySQL tends to run the subquery many times.
You can write the query differently to put the subquery in the FROM clause as a derived table. That will make MySQL run the subquery just once.
SELECT id, filepath
FROM (
SELECT SUBSTRING(filepath, -5) AS suffix, COUNT(*) AS count
FROM t
GROUP BY suffix
HAVING count > 1
) AS t1
JOIN t AS t2 ON SUBSTRING(t2.filepath, -5) = t1.suffix
This is bound to do a table-scan though, so it's going to be a costly query. It can't use an index when doing a substring comparison like that.
To optimize this, you might create a virtual column with an index.
ALTER TABLE t
ADD COLUMN filepath_last VARCHAR(10) AS (SUBSTRING_INDEX(filepath, '/', -1)),
ADD KEY (filepath_last);
Then you can query it like this, and at least the subquery uses an index:
SELECT id, filepath
FROM (
SELECT filepath_last, COUNT(*) AS count
FROM t
GROUP BY filepath_last
HAVING count > 1
) AS t1
STRAIGHT_JOIN t AS t2 ON t2.filepath_last = t1.filepath_last
The solution that ended up working for me was found here: Disable ONLY_FULL_GROUP_BY
I ran SELECT ##sql_mode then SET ##sql_mode = followed by a string containing all the values returned by the first query except for only_full_group_by, but I'm still interested in how this is to be accomplished without changing the SQL settings.
I have this query
SELECT substring(TGLLAHIR, 1, 7) as TGLLAHIR_,
substring(TGLLAHIR, 5, 2) as BULAN, `TGLLAHIR` as `TGL`
FROM `m_pasien`
WHERE substring(TGLLAHIR,1,4) = '2013'
GROUP BY substring(TGLLAHIR, 1, 7)
ORDER BY `TGLLAHIR` ASC
but message error showing
Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'rsukemba_kojarsuk.m_pasien.TGLLAHIR' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
How do I solve this?
You don't need group by, use distinct instead
SELECT distinct
substring(TGLLAHIR, 1, 7) as TGLLAHIR_
,substring(TGLLAHIR, 5, 2) as BULAN
,`TGLLAHIR` as `TGL`
FROM `m_pasien`
WHERE substring(TGLLAHIR,1,4) = '2013'
ORDER BY `TGLLAHIR` ASC
It seems that you are misusing the group by clause in this query. Group by is useful when aggregating data. For example, if you had a table of accounts, dates, and deposit amounts; and you wanted to know the total deposits into each account over all dates, you could write something like
select account, sum(deposits)
from some_table
group by account
It's unclear from you question alone what exactly you're trying to do, and even harder because your strangely-named variables are listed without any context. BUT! it's probably safe to say that you don't need to use a group by here, so just remove it.
P.S. I would be wary of doing a select distinct unless you know what you're throwing away and are certain that you don't need it.
You should use ANY_VALUE() aggregate function to suppress this warning message and make the query runnable. MySQL doesn't know which value of the non-aggregated field it should include in the result set. Every field that is not part of the GROUP BY clause should be wrapped in the aggregate function Here is, how it should look like:
SELECT substring(TGLLAHIR, 1, 7) as TGLLAHIR_,
substring(TGLLAHIR, 5, 2) as BULAN,
ANY_VALUE(`TGLLAHIR`) as `TGL`
FROM `m_pasien`
WHERE substring(TGLLAHIR,1,4) = '2013'
GROUP BY substring(TGLLAHIR, 1, 7)
ORDER BY `TGLLAHIR` ASC
And here is the explanation of this point in MySQL refference: https://dev.mysql.com/doc/refman/5.7/en/miscellaneous-functions.html#function_any-value
I think this problem due to strict mode enabled in your MySQL version. Kindly disable strict mode and try again.
To check whether strict mode is enabled or not run the below sql:
SHOW VARIABLES LIKE 'sql_mode';
If one of the value is STRICT_TRANS_TABLES, then strict mode is enabled.
To disable strict mode run the below sql:
set global sql_mode='';
Try again..
I have the following table
Date, TelephoneNumber, Type
02/02/12, 123456, b
04/02/12, 123456, b
07/02/12, 123456, a
03/02/12, 789999, a
15/02/12, 789999, b
When running the following SQL
select TelephoneNumber, max(Date) as datetime, Type
from Table1
where Date > '2012-03-25 00:00'
group by TelephoneNumber
order by date desc;
I noted that the Type does not match its related Date. For example I am getting
07/02/12, 123456, b
15/02/12, 789999, a
It seems that it is taking the first record in Type .... even when I sorted in the other way round. Can someone please help me how I can solve this problem? I am using MySQL
Thanks in advance.
sandro
This question seems to pop up quite often.
Here's my solution:
SELECT TelephoneNumber, Date AS datetime, Type
from ( SELECT *
FROM Table1
WHERE Date > '2012-03-25 00:00'
ORDER BY Date DESC) AS h
GROUP BY TelephoneNumber
ORDER BY date DESC;
Check explanation here
This is because, as §11.16.3 "GROUP BY and HAVING with Hidden Columns" in the MySQL 5.6 Reference Manual puts it:
MySQL extends the use of GROUP BY so that the
select list can refer to nonaggregated columns not named in the
GROUP BY clause. This means that the
preceding query is legal in MySQL. You can use this feature to
get better performance by avoiding unnecessary column sorting
and grouping. However, this is useful primarily when all values
in each nonaggregated column not named in the GROUP
BY are the same for each group. The server is free to
choose any value from each group, so unless they are the same,
the values chosen are indeterminate. Furthermore, the selection
of values from each group cannot be influenced by adding an
ORDER BY clause. Sorting of the result set
occurs after values have been chosen, and ORDER
BY does not affect which values the server chooses.
[emphasis mine]
Instead, you need to write something like this:
select t1a.TelephoneNumber, t1a.Date, t1a.Type
from Table1 as t1a
left
join Table1 as t1b
on t1b.TelephoneNumber = t1a.TelephoneNumber
and t1b.Date > t1a.Date
where t1a.Date > '2012-03-25 00:00'
and t1b.TelephoneNumber IS NULL -- i.e., the join failed
;
to find the record with the greatest Date for each value of TelephoneNumber.
Order is applied after grouping. The value of max(Date) has already been computed, you're not sorting it in any way. Also, what is msisdn? It's impossible to tell what's going on with your query because what you're actually grouping on is opaque.