Does anyone answer me why MySQL table raw id not serially?
MySQL will try to give you the results as quick as it can, based on your query.
If you didn't tell MySQL to sort on any field, MySQL will probably pick the order in which it's sorted on the disk.
Records in mysql aren't always stored in order on disk. An example where they might go out of order is if you delete a record in the middle of the table. The next record might (for space saving reasons) be inserted in the position where you deleted a record earlier.
Don't worry about this. This is normal. If you create an application that uses MySQL, make sure you include ORDER BY id at the end of your query to get your predictable order.
If you don't use an ORDER BY clause to make your desired order explicit, the query results are up to the implementation. That means MySQL gets to choose the order if you don't.
In the case of MyISAM tables, the default order is the order rows are stored in the table, which can get mixed up over time as rows are added and deleted. New rows may fit into gaps left by deleted rows, even if that makes them stored "out of order".
In the case of InnoDB tables, the default order is by the index used to read the rows. This is often the primary key, but it might not be. It depends on your table definition and the SQL query you use to read the rows.
Just use ORDER BY if you want the rows ordered by a specific column.
Related
I'm working with MySQL 5.6. In all of my use-cases I build a read-only table to then query.
The common use-case of my customers is to sort results by insertion order. We even added a column with index that defines insertion order. The dilemma is if we should inject an "ORDER BY" to queries (unless another sort was requested by users).
One of the guys said it might hurt performance, which sounds reasonable. He also said that there is no configuration that ensures table's order since table might be updates - although in our case the table is always read-only.
Is there a way to define a read-only table that does ensure a specific order to avoid injecting "ORDER BY" in every query? Meaning it will somehow save the table in a defined order to avoid performance penalty?
Will be happy for any suggestions.
Thanks in advance.
No. SQL tables represent unordered sets. There is no ordering in the table unless you specify an order by.
What you can do is have an index on the column you want to order by. Then, MySQL does not actually need to sort the data -- it just needs to read the index. Or, if you can, you can make the column you want to order by a primary key. This makes sense if your column is really an auto_increment column.
I have a table, where one column is sorted. Does MySQL somehow check if it's sorted or not? If I make such a query SELECT from tab WHERE col1>890 AND col1<74893798 where the col1 is sorted, is it the same as e.g. SELECT from tab WHERE id>16 AND id<798176
I was unable to test this properly because of lack of admin rights to flush the query cache.
To make it clear, the col1 is sorted, because it was inserted there from another table which had been sorted by the column.
No, it is not sorted by the column you use on your where clause. The id records look sorted because that's probably your primary key and already dictates the position of the data in your table. You need to add ORDER BY column if you want your data sorted
MySQL does not sort results by default (What drive the natural result order for an unordered MySQL request). Generally, Relational DataBase Management Systems (RDBMS) do not guarantee any specific row order unless you specify an order with an ORDER BY clause.
The advantage of this is that the database may internally sort data in whatever format it deems to be most optimal.
I have the following query
SELECT *
FROM table_1
INNER JOIN table_2 ON table_1.orders = table_2.orders
ORDER BY table_2.purchasetime;
The above query result is indeterminate i.e it can change with different queries when the purchase time is of same value as per the MySQL manual itself.To overcome this we give sort ordering on a unique column and combine it with the regular sort ordering.
The customer does not want to see different results with different page refreshes so we have put in the above fix specifically for MySQL which is unnecessary and needs extra compound indexes for both asc and desc.
I am not sure whether the same is applicable for postgres.So far I have not been able to reproduce the scenario.I would appreciate if someone could answer this for postgres or point me in the right direction.
Edit 1 : The sort column is indexed.So assuming the disk data has no ordering, but in the case of index (btree data structure) a constant ordering might be possible with postgres ?
No, it will not be different in PostgreSQL (or, in fact, in any other relational database that I know of).
See http://www.postgresql.org/docs/9.4/static/queries-order.html :
After a query has produced an output table (after the select list has been processed) it can optionally be sorted. If sorting is not chosen, the rows will be returned in an unspecified order. The actual order in that case will depend on the scan and join plan types and the order on disk, but it must not be relied on. A particular output ordering can only be guaranteed if the sort step is explicitly chosen.
Even if by accident you manage to find a PostgreSQL version and index that will guarantee the order in all the test you run, please don't rely on it. Any database upgrade, data change or a change in the Maya calendar or the phase of the moon can suddenly upset your sorting order. And debugging it then is a true and terrible pain in the neck.
Your concern seems to be that order by table_2.purchasetime is indeterminate when there are multiple rows with the same value.
To fix this -- in any database or really any computer language -- you need a stable sort. You can turn any sort into a stable sort by adding a unique key. So, adding a unique column (typically an id of some sort) fixes this in both MySQL and Postgres (and any other database).
I should note that instability in sorts can be a very subtle problem, one that only shows up under certain circumstances. So, you could run the same query many times and it is fine. Then you insert or delete a record (perhaps even one not chosen by the query) and the order changes.
In our application, we have a page that displays user a set of data, a part of it actually. It also allows user to order it by a custom field. So in the end it all comes down to query like this:
SELECT name, info, description FROM mytable
WHERE active = 1 -- Some filtering by indexed column
ORDER BY name LIMIT 0,50; -- Just a part of it
And this worked just fine, as long as the size of table is relatively small (used only locally in our department). But now we have to scale this application. And let's assume, the table has about a million of records (we expect that to happen soon). What will happen with ordering? Do I understand correctly, that in order to do this query, MySQL will have to sort a million records each time and give a part of it? This seems like a very resource-heavy operation.
My idea is simply to turn off that feature and don't let users select their custom ordering (maybe just filtering), so that the order would be a natural one (by id in descending order, I believe the indexing can handle that).
Or is there a way to make this query work much faster with ordering?
UPDATE:
Here is what I read from the official MySQL developer page.
In some cases, MySQL cannot use indexes to resolve the ORDER BY,
although it still uses indexes to find the rows that match the WHERE
clause. These cases include the following:
....
The key used to
fetch the rows is not the same as the one used in the ORDER BY:
SELECT * FROM t1 WHERE key2=constant ORDER BY key1;
So yes, it does seem like mysql will have a problem with such a query? So, what do I do - don't use an order part at all?
The 'problem' here seems to be that you have 2 requirements (in the example)
active = 1
order by name LIMIT 0, 50
The former you can easily solve by adding an index on the active field
The latter you can improve by adding an index on name
Since you do both in the same query, you'll need to combine this into an index that lets you resolve the active value quickly and then from there on fetches the first 50 names.
As such, I'd guess that something like this will help you out:
CREATE INDEX idx_test ON myTable (active, name)
(in theory, as always, try before you buy!)
Keep in mind though that there is no such a thing as a free lunch; you'll need to consider that adding an index also comes with downsides:
the index will make your INSERT/UPDATE/DELETE statements (slightly) slower, usually the effect is negligible but only testing will show
the index will require extra space in de database, think of it as an additional (hidden) special table sitting next to your actual data. The index will only hold the fields required + the PK of the originating table, which usually is a lot less data then the entire table, but for 'millions of rows' it can add up.
if your query selects one or more fields that are not part of the index, then the system will have to fetch the matching PK fields from the index first and then go look for the other fields in the actual table by means of the PK. This probably is still (a lot) faster than when not having the index, but keep this in mind when doing something like SELECT * FROM ... : do you really need all the fields?
In the example you use active and name but from the text I get that these might be 'dynamic' in which case you'd have to foresee all kinds of combinations. From a practical point this might not be feasible as each index will come with the downsides of above and each time you add an index you'll add supra to that list again (cumulative).
PS: I use PK for simplicity but in MSSQL it's actually the fields of the clustered index, which USUALLY is the same thing. I'm guessing MySQL works similarly.
Explain your query, and check, whether it goes for filesort,
If Order By doesnt get any index or if MYSQL optimizer prefers to avoid the existing index(es) for sorting, it goes with filesort.
Now, If you're getting filesort, then you should preferably either avoid ORDER BY or you should create appropriate index(es).
if the data is small enough, it does operations in Memory else it goes on the disk.
so you may try and change the variable < sort_buffer_size > as well.
there are always tradeoffs, one way to improve the preformance of order query is to set the buffersize and then the run the order by query which improvises the performance of the query
set sort_buffer_size=100000;
<>
If this size is further increased then the performance will start decreasing
Can I prevent using filesort in mysql when field on which condition in one table and field on which order in another. Can use index in this situation? Both tables are large - more than 1 million records
You are dealing with 1 million records, definitely you need to add indexing to gain some speed otherwise it will be overkill anyone visiting you site.
You need to closely examine which fields you will be add indexing to. Thanks
You need to carefully look at the indexes using explain.
If there are where clauses, either add the column you are ordering by to the index being used for that table as well and see if it gets rid of the filesort, or if its not currently using one for that table (which I doubt is the case with that many records) then just create a new one.
Also worth flagging, which someone else may be able to offer more info on, is that mysql often (or always?) can't use an index to sort DESC - I've been in a situation before where its been performant to have a computed field that will index in reverse order, and order by that ASC.
EG: If you have an integer field that you want to order by DESC, add a field where you store the integer value subtracted from 1000000000 (or some other large number), index it, and order by that field ascending.
As I say, I can't remember specifics, it may affect older mysql only or something, but have a feeling its a current limitation.