UPDATE:
In the following question, I thought when you select rows, MySQL create a kind of row index, and then the LIMIT and OFFSET clause cuts off this list by its numbers. The problem is I'm using wrong values or field combination to sort the rows as the have almost the same value.
Here is my mistake, assuming to much "intelligence" form MySQL ;-)
The solution is always to have a more reliable sorting field as a fallback, like the ID, like so ORDER BY priority DESC, id ASC
This is a strict MySQL question. Why does it seem the LIMIT OFFSET clause is applied before the ORDER BY? or what am I missing?
Here is the example, first we select a list of rows ordered by a field called priority:
SELECT d0_.name, d0_.id AS id_0, d0_.priority AS priority_1 FROM destination d0_ WHERE d0_.active = 1 ORDER BY d0_.priority DESC;
The results looks like this:
Then I want to select the first 10 rows from this list using the following query:
SELECT d0_.name, d0_.id AS id_0, d0_.priority AS priority_1 FROM destination d0_ WHERE d0_.active = 1 ORDER BY d0_.priority DESC LIMIT 10 OFFSET 10;
And I've got this result:
Why does not the list goes from "Grandvalira" to "Sierra nevada"?
The problem of this, is not the actual order but some rows are never reached! like "Vallnord Ordino-Arcalís". As I change the OFFSET value, it does not go through all the rows, and it event repeats some rows.
This is the basic question. But this is giving me problems at the end when using the "KnpPaginatorBundle (2.5.3)" of Symfony. I thought was a problem of the php code, but mysql queries are giving this unexpected results for me.
Any help or clue of whats going on?
Thanks!
You are not getting the results you are expecting because your data has many rows with the same value for priority.
When you use 'order by' on priority, all the rows with priority can come in any order. There is no guarantee about the ordering with the same value of priority. To resolve the tie, you can add additional fields to your order by clause. Depending on your choice you might choose to include name or id field in the 'order by' clause.
Related
I ran into a very interesting MySQL query issue that only seems to apply when no sorting order is specified and limits & offsets are used in the query. One of the items from my table is repeated as a result when no sorting order is specified. This doesn't seem to make sense based on how the ordering should work.
I've created a simple database and was able to reproduce the original problem I ran into:
Structure:
Data:
When using SQL queries such as:
SELECT SQL_CALC_FOUND_ROWS * FROM test WHERE test_customer = '1' LIMIT 4 OFFSET 4
As I adjust the offset, the value "yes" doesn't appear in the last offset query. Element 13 is repeated. That doesn't seem to make sense.
Anyone know why this happening?
For a more detailed explanation of what's going on, please see this full example that shows the default results versus ascending order results:
https://eamster.tk/mysql_debug_test/mysql_debug.php
Any help is appreciated. I'm just trying to understand what's going on... for now, I'll just work around the issue by specifying the ORDER BY clause.
SQL tables and result sets (with no ORDER BY) represent unordered sets. Ordering is arbitrary and capricious. It can change from query execution to query execution.
This query:
SELECT SQL_CALC_FOUND_ROWS *
FROM test
WHERE test_customer = '1'
LIMIT 4 OFFSET 4;
Returns an arbitrary 4 rows for the customer. The OFFSET means nothing.
If you want data in a particular order, then use ORDER BY. Otherwise, the data is in an arbitrary order that can change from query execution to query execution. There is no "default ordering".
So:
SELECT SQL_CALC_FOUND_ROWS *
FROM test
WHERE test_customer = 1
ORDER BY testid
LIMIT 4 OFFSET 4;
Note that I removed the single quotes from the =. The value is an integer, so use a numeric constant.
I have a simple query
SELECT * FROM table_name order by non_unique_column LIMIT 0,50
This gives me list of records in some order.
But when I am removing * and putting some column names from the table, then it is changing the list order.
For some combination of columns in select clause it is giving different order.
I am not able to find, what pattern of column names in select clause gives different ordering.
So I am asking does any one had this kind of problem? or can any one suggest how the order by clause works in this scenario?
Note: If I am adding all the columns in select clause it gives me same result as *.
Please feel free to suggest any possible suggestion.
Thanks.
Update: Also when I am changing limit clause like LIMIT 0,50 to LIMIT 0,15, I am getting different sets of records.
Update2:
I have created a sqlfiddle :
http://sqlfiddle.com/#!2/8f2340/3
Please see the result from below queries
SELECT `M_ID`, `CON_FRST_NM`, `CON_PREFIX`, `CON_LST_NM`, `CON_MOB`,`CON_FAX` FROM `sample_table` order by `CON_PREFIX` LIMIT 0,60
SELECT `M_ID`, `CON_FRST_NM`, `CON_PREFIX`, `CON_LST_NM`, `CON_MOB` FROM `sample_table` order by `CON_PREFIX` LIMIT 0,60
The only difference in the query is CON_FAX is not present in select clause of second query.
Please help me out why this difference is coming.
And also please see the result of below queries with different limit clause.
Note: It is second query from above queries.
SELECT `M_ID`, `CON_FRST_NM`, `CON_PREFIX`, `CON_LST_NM`, `CON_MOB` FROM `sample_table` order by `CON_PREFIX` LIMIT 0,60
SELECT `M_ID`, `CON_FRST_NM`, `CON_PREFIX`, `CON_LST_NM`, `CON_MOB` FROM `sample_table` order by `CON_PREFIX` LIMIT 0,15
Thanks.
PERSONAL OPINION: I see this as a bug/defect, because even if I haven't defined any unique order by clause and mysql is applying some random ordering based on its calculations, I feel it should not depend on what is present in select clause. My problem is for a given set of records it should return same ordered records regardless of whatever column I am selecting.
The issue with this is because there is a duplicated value that you specify for your order by aka its either Mr. or Mrs. there is no guaranteed select order with this because they are all Mr. or Mrs.
if you want to ensure that it is always going to be in a specific order then you should also include the primary key as a second ordering to keep it all the same.. aka
ORDER BY CON_PREFIX ASC, M_ID ASC
as it stands right now ORDER BY CON_PREFIX is giving you exactly what it should be giving you, the prefixes in ascending order, there is nothing related to the limit for causing this, you simply just havent told MySQL how else you want the data to be returned to you.
to respond to your PERSONAL OPINION edit in the question....
what you are describing is unreasonable, think of it this way any default order that would be used to pull out data is now gone because you are specifying an order by. if there wasn't any order by on the clause then mysql has a generic select pattern but again that is gone once you put an order by on the query
The order by clause works from left to right
Let me give an example
If you use
SELECT serialNo,Name FROM table_name order by serialNo,Name;
what this does it it will order serialNo in the first instance and then it will order Name in the second instance . When it orders name in the second instance the oredering of the serialNo will change according to the ordering of the name .
Hope I was clear in my explanation .
I have the following SQL query , it seems to run ok , but i am concerned as my site grows it may not perform as expected ,I would like some feeback as to how effective and efficient this query really is:
select * from articles where category_id=XX AND city_id=XXX GROUP BY user_id ORDER BY created_date DESC LIMIT 10;
Basically what i am trying to achieve - is to get the newest articles by created_date limited to 10 , articles must only be selected if the following criteria are met :
City ID must equal the given value
Category ID must equal the given value
Only one article per user must be returned
Articles must be sorted by date and only the top 10 latest articles must be returned
You've got a GROUP BY clause which only contains one column, but you are pulling all the columns there are without aggregating them. Do you realise that the values returned for the columns not specified in GROUP BY and not aggregated are not guaranteed?
You are also referencing such a column in the ORDER BY clause. Since the values of that column aren't guaranteed, you have no guarantee what rows are going to be returned with subsequent invocations of this script even in the absence of changes to the underlying table.
So, I would at least change the ORDER BY clause to something like this:
ORDER BY MAX(created_date)
or this:
ORDER BY MIN(created_date)
some potential improvements (for best query performance):
make sure you have an index on all columns you querynote: check if you really need an index on all columns because this has a negative performance when the BD has to build the index. -> for more details take a look here: http://dev.mysql.com/doc/refman/5.1/en/optimization-indexes.html
SELECT * would select all columns of the table. SELECT only the ones you really require...
Someone sent me a SQL query where the GROUP BY clause consisted of the statement: GROUP BY 1.
This must be a typo right? No column is given the alias 1. What could this mean? Am I right to assume that this must be a typo?
It means to group by the first column of your result set regardless of what it's called. You can do the same with ORDER BY.
SELECT account_id, open_emp_id
^^^^ ^^^^
1 2
FROM account
GROUP BY 1;
In above query GROUP BY 1 refers to the first column in select statement which is
account_id.
You also can specify in ORDER BY.
Note : The number in ORDER BY and GROUP BY always start with 1 not with 0.
In addition to grouping by the field name, you may also group by ordinal, or position of the field within the table. 1 corresponds to the first field (regardless of name), 2 is the second, and so on.
This is generally ill-advised if you're grouping on something specific, since the table/view structure may change. Additionally, it may be difficult to quickly comprehend what your SQL query is doing if you haven’t memorized the table fields.
If you are returning a unique set, or quickly performing a temporary lookup, this is nice shorthand syntax to reduce typing. If you plan to run the query again at some point, I’d recommend replacing those to avoid future confusion and unexpected complications (due to scheme changes).
It will group by first field in the select clause
That means *"group by the 1st column in your select clause". Always use GROUP BY 1 together with ORDER BY 1.
You can also use GROUP BY 1,2,3... It is convenient, but you need to pay attention to that condition; the result may not be what you want if someone has modified your select columns and it's not visualized.
It will group by the column position you put after the group by clause.
for example if you run 'SELECT SALESMAN_NAME, SUM(SALES) FROM SALES GROUP BY 1'
it will group by SALESMAN_NAME.
One risk on doing that is if you run 'Select *' and for some reason you recreate the table with columns on a different order, it will give you a different result than you would expect.
When querying the db for a set of ids, mysql doesnot provide the results in the order by which the ids were specified. The query i am using is the following:
SELECT id ,title, date FROM Table WHERE id in (7,1,5,9,3)
in return the result provided is in the order 1,3,5,7,9.
How can i avoid this auto sorting
If you want to order your result by id in the order specified in the in clause you can make use of FIND_IN_SET as:
SELECT id ,title, date
FROM Table
WHERE id in (7,1,5,9,3)
ORDER BY FIND_IN_SET(id,'7,1,5,9,3')
There is no auto-sorting or default sorting going on. The sorting you're seeing is most likely the natural sorting of rows within the table, ie. the order they were inserted. If you want the results sorted in some other way, specify it using an ORDER BY clause. There is no way in SQL to specify that a sort order should follow the ordering of items in an IN clause.
The WHERE clause in SQL does not affect the sort order; the ORDER BY clause does that.
If you don't specify a sort order using ORDER BY, SQL will pick its own order, which will typically be the order of the primary key, but could be anything.
If you want the records in a particular order, you need to specify an ORDER BY clause that tells SQL the order you want.
If the order you want is based solely on that odd sequence of IDs, then you'd need to specify that in the ORDER BY clause. It will be tricky to specify exactly that. It is possible, but will need some awkward SQL code, and will slow down the query significantly (due to it no longer using a key to find the records).
If your desired ID sequence is because of some other factor that is more predictable (say for example, you actually want the records in alphabetical name order), you can just do ORDER BY name (or whatever the field is).
If you really want to sort by the ID in an arbitrary sequence, you may need to generate a temporary field which you can use to sort by:
SELECT *,
CASE id
WHEN 7 THEN 1
WHEN 1 THEN 2
WHEN 5 THEN 3
WHEN 3 THEN 4
WHEN 9 THEN 5
END AS mysortorder
FROM mytable
WHERE id in (7,1,5,9,3)
ORDER BY mysortorder;
The behaviour you are seeing is a result of query optimisation, I expect that you have an index on id so that the IN statement will use the index to return records in the most efficient way. As an ORDER BY statement has not been specified the database will assume that the order of the return records is not important and will optimise for speed. (Checkout "EXPLAIN SELECT")
CodeAddicts or Spudley's answer will give the result you want. An alternative is assigning a priority to the id's in "mytable" (or another table) and using this to order the records as desired.