I have this query.
SELECT * FROM (SELECT * FROM private_messages ORDER BY id DESC) a
The "DESC" works on my local server but doesn't work on another server.
However if i just write it like this:
SELECT * FROM private_messages ORDER BY id DESC
it works on both servers. What would cause this?
This is going to be too long for a comment..
It's not a bug. It's documented here:
As of MySQL 5.7.6, the optimizer handles propagation of an ORDER BY
clause in a derived table or view reference to the outer query block
by propagating the ORDER BY clause if the following conditions apply:
The outer query is not grouped or aggregated; does not specify
DISTINCT, HAVING, or ORDER BY; and has this derived table or view
reference as the only source in the FROM clause. Otherwise, the
optimizer ignores the ORDER BY clause. Before MySQL 5.7.6, the
optimizer always propagated ORDER BY, even if it was irrelevant or
resulted in an invalid query.
However it's not the case for your query. So i guess your second server is running MariaDB which seams to ingnore any ORDER BY in a subquery without LIMIT
A "table" (and subquery in the FROM clause too) is - according to the
SQL standard - an unordered set of rows. Rows in a table (or in a
subquery in the FROM clause) do not come in any specific order. That's
why the optimizer can ignore the ORDER BY clause that you have
specified. In fact, SQL standard does not even allow the ORDER BY
clause to appear in this subquery (we allow it, because ORDER BY ...
LIMIT ... changes the result, the set of rows, not only their order).
You need to treat the subquery in the FROM clause, as a set of rows in
some unspecified and undefined order, and put the ORDER BY on the
top-level SELECT.
Why is ORDER BY in a FROM Subquery Ignored?
So best you can do is just to move the ORDER BY clause to the outer query. Or don't use a subquery at all.
Related
My requirements are: I now have a table, I need to group according to one of the fields, and get the latest record in the group, and then I search the scheme on the Internet,
SELECT
* FROM(
SELECT
*
FROM
record r
WHERE
r.id in (xx,xx,xx) HAVING 1
ORDER BY
r.time DESC
) a
GROUP BY
a.id
, the result is correct, but I can't understand the meaning of "having 1" after the where statement. I hope a friend can give me an answer. Thank you very much.
It does nothing, just like having true would. Presumably it is a placeholder where sometimes additional conditions are applied? But since there is no group by or use of aggregate functions in the subquery, any having conditions are going to be treated no differently than where conditions.
Normally you select rows and apply where conditions, then any grouping (explicit, or implicit as in select count(*)) occurs, and the having clause can specify further constraints after the grouping.
Note that your query is not guaranteed to give the results you want; the order by in the subquery in theory has no effect on the outer query and the optimizer may skip it. It is possible the presence of having makes a difference to the optimizer, but that is not something you should rely on, certainly from one version of mysql to another.
I used the following query with MySQL 5.5 (or previous versions) for years without any problems:
SELECT t2.Code from (select Country.Code from Country order by Country.Code desc ) AS t2;
The order of the result was always descending as I needed.
Last week, I just migrated to a new MySQL Version (In fact, I migrated to MariaDB 10.0.14) and now the same query with the same database is not sorted descending anymore. It is sorted ascending (or sorted using the natural order, not sure in fact).
So, can somebody could tell me if this is a bug or if this is a change of the behaviour in recent versions of MySQL/MariaDB?
After a bit of digging, I can confirm both your scenarios:
MySQL 5.1 does apply the ORDER BY inside the subquery.
MariaDB 5.5.39 on Linux does not apply the ORDER BY inside the subquery when no LIMIT is supplied. It does however correctly apply the order when a corresponding LIMIT is given:
SELECT t2.Code
FROM (
SELECT Country.Code FROM Country ORDER BY Country.Code DESC LIMIT 2
) AS t2;
Without that LIMIT, there isn't a good reason to apply the sort inside the subquery. It can be equivalently applied to the outer query.
Documented behavior:
As it turns out, MariaDB has documented this behavior and it is not regarded as a bug:
A "table" (and subquery in the FROM clause too) is - according to the SQL standard - an unordered set of rows. Rows in a table (or in a subquery in the FROM clause) do not come in any specific order. That's why the optimizer can ignore the ORDER BY clause that you have specified. In fact, SQL standard does not even allow the ORDER BY clause to appear in this subquery (we allow it, because ORDER BY ... LIMIT ... changes the result, the set of rows, not only their order).
You need to treat the subquery in the FROM clause, as a set of rows in some unspecified and undefined order, and put the ORDER BY on the top-level SELECT.
So MariaDB also recommends applying the ORDER BY in the outermost query, or a LIMIT if necessary.
Note: I don't currently have access to a proper MySQL 5.5 or 5.6 to confirm if the behavior is the same there (and SQLFiddle.com is malfunctioning). Comments on the original bug report (closed as not-a-bug) suggest that MySQL 5.6 probably behaves the same way as MariaDB.
In newer versions of MySQL and MariaDB you can force the ORDER BY in a sub query by applying a LIMIT. If you don't want to limit the rows, use the biggest number of BIGINT as a LIMIT.
This may come in handy at times, when the sub query needs to be generated in a desired order, for applying line numbers, for example.
This is more of an academic question, because in my particular case I can create an easy workaround, but I would like to understand the reason behind this anyway.
Using an InnoDB table (MariaDB 10.0.31) with (among others) columns customer and uri, I wanted to select the distinct uris for a specific customer. Now, the table is quite large (around 50M entries), so there is a composite index on customer and uri.
Basically what I don't understand is why the order of the columns in the group by clause matters.
explain select customer, uri from `tableName` group by customer,uri;
tells me it will use the existing index for group by, but
explain select customer, uri from `tableName` group by uri,customer;
won't do so.
Could someone explain why this is the case? I always thought of the group by clause as declarative.
Maybe it's because it's Friday, but I can't think of a case, where the order of the group by columns would affect the result.
Your observation is correct. Results would be different as the "prefix" order of columns mentioned in the composite index declaration is used for decision making by the Cost based optimizer. This behavior is due to the usage of B-TREE index
GROUP BY clause is used for ordering the result and hence if
the correct order of index is used or
only leftmost columns are used in group by
leftmost column is used in WHERE clause and rest in correct order in GROUP BY clause index would be used.
More on this and topic of Loose/Tight Index Scan can be found here
https://dev.mysql.com/doc/refman/5.7/en/group-by-optimization.html
In index is basically an ordered table. In you case it is ordered according like ORDER BY customer, uri (because this is how your index is defined).
MySQL executes group by by first ordering the result according to the group by clause and then collapsing the rows with the same values (that happen to follow each other after sorting).
Apparently, MySQL is not smart enough to recognize that the different group by clause could also be executed when the result is ordered the other way.
More about this:
http://use-the-index-luke.com/sql/sorting-grouping
In particular: http://use-the-index-luke.com/sql/sorting-grouping/indexed-group-by
Write a feature request at bugs.mysql.com .
On the one hand, GROUP BY is (or was) defined to imply ORDER BY with the same columns in the same order.
On the other hand, if you ignore that non-standard feature, even by saying ORDER BY NULL, MySQL fails to shuffle the columns in order to use the index.
5.7 (and before) says
GROUP BY implicitly sorts by default (that is, in the absence of ASC
or DESC designators), but relying on implicit GROUP BY sorting is
deprecated. To produce a given sort order, use explicit ASC or DESC
designators for GROUP BY columns or provide an ORDER BY clause. GROUP
BY sorting is a MySQL extension that may change in a future release;
for example, to make it possible for the optimizer to order groupings
in whatever manner it deems most efficient and to avoid the sorting
overhead.
and
If a query includes GROUP BY but you want to avoid the overhead of
sorting the result, you can suppress sorting by specifying ORDER BY
NULL.
But, watch out; 8.0 says
Previously, relying on implicit GROUP BY sorting was deprecated but
GROUP BY did sort by default (that is, in the absence of ASC or DESC
designators). In MySQL 8.0, GROUP BY no longer sorts by default, so
query results may differ from previous MySQL versions. To produce a
given sort order, use explicit ASC or DESC designators for GROUP BY
columns or provide an ORDER BY clause.
I need to sort selected_booking by cost first and then assign the index i to every row. My variant doesn't work properly (outer SELECT breaks the order):
SELECT (#i:=#i + 1) AS i, selected_booking.*
FROM (SELECT * FROM booking ORDER BY cost DESC) AS selected_booking;
Is there any way to save the order of inner selection when doing outer one?
Q: Is there any way to save the order of inner selection when doing outer selection?
A: Absent an ORDER BY clause on the outer query, MySQL is free to return the rows in any order it chooses.
If you want rows from the inline view (derived table) returned in a specific order, you need to specify that in the outer query... you'd need to add an ORDER BY clause on the outer query.
NOTE: The behavior of user-defined variables as in your query is not guaranteed, the MySQL Reference Manual warns of this. But in spite of that warning, we do observe repeatable behavior in MySQL 5.1 and 5.5.
It's not at all clear why you need an inline view (aka a derived table, in the MySQL venacular) in the example you give.
It seems like this query would return the result you seem to want:
SET #i = 0 ;
SELECT #i:=#i+1 AS i
, b.*
FROM booking b
ORDER BY b.cost DESC ;
Alternatively, you could do this in a single statement, and initialize #i within the query, rather than a separate SET statement.
SELECT #i:=#i+1 AS i
, b.*
FROM booking b
JOIN (SELECT #i:=0) i
ORDER BY b.cost DESC
(This initialization works, again, because of the way the MySQL processes inline views, the inline view query is run BEFORE the outer query. This isn't guaranteed behavior, and may change in a future release (it may have already changed in 5.6)
NOTE: For improved performance of this query, if a suitable index is available with cost as the leading column, e.g.
... ON booking (cost)
that may allow MySQL to use that index to return rows in order and avoid a "Using filesort" operation.
I used the following query with MySQL 5.5 (or previous versions) for years without any problems:
SELECT t2.Code from (select Country.Code from Country order by Country.Code desc ) AS t2;
The order of the result was always descending as I needed.
Last week, I just migrated to a new MySQL Version (In fact, I migrated to MariaDB 10.0.14) and now the same query with the same database is not sorted descending anymore. It is sorted ascending (or sorted using the natural order, not sure in fact).
So, can somebody could tell me if this is a bug or if this is a change of the behaviour in recent versions of MySQL/MariaDB?
After a bit of digging, I can confirm both your scenarios:
MySQL 5.1 does apply the ORDER BY inside the subquery.
MariaDB 5.5.39 on Linux does not apply the ORDER BY inside the subquery when no LIMIT is supplied. It does however correctly apply the order when a corresponding LIMIT is given:
SELECT t2.Code
FROM (
SELECT Country.Code FROM Country ORDER BY Country.Code DESC LIMIT 2
) AS t2;
Without that LIMIT, there isn't a good reason to apply the sort inside the subquery. It can be equivalently applied to the outer query.
Documented behavior:
As it turns out, MariaDB has documented this behavior and it is not regarded as a bug:
A "table" (and subquery in the FROM clause too) is - according to the SQL standard - an unordered set of rows. Rows in a table (or in a subquery in the FROM clause) do not come in any specific order. That's why the optimizer can ignore the ORDER BY clause that you have specified. In fact, SQL standard does not even allow the ORDER BY clause to appear in this subquery (we allow it, because ORDER BY ... LIMIT ... changes the result, the set of rows, not only their order).
You need to treat the subquery in the FROM clause, as a set of rows in some unspecified and undefined order, and put the ORDER BY on the top-level SELECT.
So MariaDB also recommends applying the ORDER BY in the outermost query, or a LIMIT if necessary.
Note: I don't currently have access to a proper MySQL 5.5 or 5.6 to confirm if the behavior is the same there (and SQLFiddle.com is malfunctioning). Comments on the original bug report (closed as not-a-bug) suggest that MySQL 5.6 probably behaves the same way as MariaDB.
In newer versions of MySQL and MariaDB you can force the ORDER BY in a sub query by applying a LIMIT. If you don't want to limit the rows, use the biggest number of BIGINT as a LIMIT.
This may come in handy at times, when the sub query needs to be generated in a desired order, for applying line numbers, for example.