I have a very large sql table that I want to display. However, ist most cases I just want to have it sorted by my datetime-column.
Unfortunately, the regular select-query only displays the table ordered by the primary key (id) - therefore, I have to always manually add 'order by' to my select-query, which is a problem, because sorting the table takes several seconds, even with index.
So my question, is there a way to insert new entries sorted into the table? (by 'sorted', I mean, sorted by the datetime-column, instead of the id column). The incoming entry (id=50) should be sorted before id=49, because it has an earlier datetime-value
Then, using a SELECT * FROM myTable should already show everything sorted by datetime
I realize that it's a bad practice and that I should use order-by, but its not really feasible in my case, because I have millions of entries, which have to be sorted again and again for every select
When you have a MyISAM table, you can do
ALTER TABLE your_table ORDER BY whatever_column;
But the table does not stay this way, when you insert or update entries.
Read more about it here.
This does not make sense for InnoDB tables, however. They are sorted by the clustered index. In this case an option can be to extend your primary key to (datetime_column, column_thats_primary_key_so_far). This is untested, personally I would add an ORDER BY to the SELECT, just in case.
It won't keep the data sorted for future inserts, but you can change the table ordering like this:
ALTER TABLE myTable ORDER BY timestamp
Related
My MySQL table is set up with an auto-incrementing primary key. Whenever I
SELECT * FROM [MYTABLE]
The record with the highest primary key value is not displayed last. Is there a reason for this? I can
SELECT * FROM [MYTABLE] ORDER BY [MYTABLE].ID ASC
and the highest is displayed last. Sorry, I am not at liberty to share anything from the database. Perhaps there is some sort of underlying data field (like a record number) not contained in the table that is being used for the sort. This is MySQL 5.7.19 on a Windows server. It seems to me that sorting by a primary key makes more sense than what it's doing.
MySQL version 5.7.19.
What you want is not applicable.
Well, of course, MySquile inherently fetches data no matter how it is stored.
Sort the index table each time you change it, list the main key in the table, or sort the call each time.
select with ordering
SELECT * FROM [MYTABLE] ORDER BY [MYTABLE].ID ASC
Keep in mind that you can never guarantee the order of an auto-incremented column once you start inserting and removing data, so you shouldn't be relying on any order except that which you specify using ORDER BY in your queries. This is an expensive operation that you are doing, as it requires indexes to be completely re-created, so I wouldn't suggest doing it often.
Step 1:
I am creating a simple table.
CREATE TABLE `indexs`.`table_one` (
`id` INT NOT NULL AUTO_INCREMENT,
`name` VARCHAR(45) NULL,
PRIMARY KEY (`id`));
Step 2:
I make two inserts into this table.
insert into table_one (name) values ("B");
insert into table_one (name) values ("A");
Step 3:
I make a select, I get a table, the records in which are ordered by id.
SELECT * FROM table_one;
This is the expected result, because in mysql the primary key is a clustered index, therefore the data will be physically ordered by it.
Now the part I don't understand.
Step 4:
I am creating an index on the name column.
CREATE INDEX index_name ON table_one(name)
I repeat step 3 again, but I get a different result. The lines are now ordered according to the name column.
Why is this happening? why the order of the rows in the table changes in accordance with the new index on the name column, because as far as I understand, in mysql, the primary key is the only clustered index, and all indexes created additionally are secondary.
I make a select, I get a table, the records in which are ordered by id. [...] This is the expected result, because in mysql the primary key is a clustered index, therefore the data will be physically ordered by it.
There is some misunderstanding of a concept here.
Table rows have no inherent ordering: they represent unordered set of rows. While the clustered index enforces a physical ordering of data in storage, it does not guarantee the order in which rows are returned by a select query.
If you want the results of the query to be ordered, then use an order by clause. Without such clause, the ordering or the rows is undefined: the database is free to return results in whichever order it likes, and results are not guaranteed to be consistent over consecutive executions of the same query.
select * from table_one order by id;
select * from table_one order by name;
(GMB explains most)
Why is this happening? why the order of the rows in the table changes in accordance with the new index on the name column
Use EXPLAIN SELECT ... -- it might give a clue of what I am about to suggest.
You added INDEX(name). In InnoDB, the PRIMARY KEY column(s) are tacked onto the end of each secondary index. So it is effectively a BTree ordered by (name,id) and containing only those columns.
Now, the Optimizer is free to fetch the data from the index, since it has everything you asked for (id and name). (This index is called "covering".)
Since you did not specify an ORDER BY, the result set ordering is valid (see GMB's discussion).
Moral of the story: If you want an ordering, specify ORDER BY. (The Optimizer is smart enough to "do no extra work" if it can see how to provide the data without doing a sort.
Further experiment: Add another column to the table but don't change the indexes. Now you will find SELECT * FROM t is ordered differently than SELECT id, name FROM t. I think I have given you enough clues to predict this difference, if not, ask.
I am creating a table which will store around 100million rows in MySQL 5.6 using InnoDB storage engine. This table will have a foreign key that will link to another table with around 5 million rows.
Current Table Structure:
`pid`: [Foreign key from another table]
`price`: [decimal(9,2)]
`date`: [date field]
and every pid should have only one record for a date
What is the best way to create indexes on this table?
Option #1: Create Primary index on two fields pid and date
Option #2: Add another column id with AUTO_INCREMENT and primary index and create a unique index on column pid and date
Or any other option?
Only select query i will be using on this table is:
SELECT pid,price,date FROM table WHERE pid = 123
Based on what you said (100M; the only query is...; InnoDB; etc):
PRIMARY KEY(pid, date);
and no other indexes
Some notes:
Since it is InnoDB, all the rest of the fields are "clustered" with the PK, so a lookup by pid is acts as if price were part of the PK. Also WHERE pid=123 ORDER BY date would be very efficient.
No need for INDEX(pid, date, price)
Adding an AUTO_INCREMENT gains nothing (except a hint of ordering). If you needed ordering, then an index starting with date might be best.
Extra indexes slow down inserts. Especially UNIQUE ones.
Either method is fine. I prefer having synthetic primary keys (that is, the auto-incremented version with the additional unique index). I find that this is useful for several reasons:
You can have a foreign key relationship to the table.
You have an indicator of the order of insertion.
You can change requirements, so if some pids allows two values per day or only one per week, then the table can support them.
That said, there is additional overhead for such a column. This overhead adds space and a small amount of time when you are accessing the data. You have a pretty large table, so you might want to avoid this additional effort.
I would try with an index that attempts to cover the query, in the hope that MySQL has to access to the index only in order to get the result set.
ALTER TABLE `table` ADD INDEX `pid_date_price` (`pid` , `date`, `price`);
or
ALTER TABLE `table` ADD INDEX `pid_price_date` (`pid` , `price`, `date`);
Choose the first one if you think you may need to select applying conditions over pid and date in the future, or the second one if you think the conditions will be most probable over pid and price.
This way, the index has all the data the query needs (pid, price and date) and its indexing on the right column (pid)
By the way, always use EXPLAIN to see if the query planner will really use the whole index (take a look at the key and keylen outputs)
I have a table with a large number of records ( > 300,000). The most relevant fields in the table are:
CREATE_DATE
MOD_DATE
Those are updated every time a record is added or updated.
I now need to query this table to find the date of the record that was modified last. I'm currently using
SELECT mod_date FROM table ORDER BY mod_date DESC LIMIT 1;
But I'm wondering if this is the most efficient way to get the answer.
I've tried adding a where clause to limit the date to the last month, but it looks like that's actually slower (and I need the most recent date, which could be older than the last month).
I've also tried the suggestion I read elsewhere to use:
SELECT UPDATE_TIME
FROM information_schema.tables
WHERE TABLE_SCHEMA = 'db'
AND TABLE_NAME = 'table';
But since I might be working on a dump of the original that query might result into NULL. And it looks like this is actually slower than the original query.
I can't resort to last_insert_id() because I'm not updating or inserting.
I just want to make sure I have the most efficient query possible.
The most efficient way for this query would be to use an index for the column MOD_DATE.
From How MySQL Uses Indexes
8.3.1 How MySQL Uses Indexes
Indexes are used to find rows with specific column values quickly.
Without an index, MySQL must begin with the first row and then read
through the entire table to find the relevant rows. The larger the
table, the more this costs. If the table has an index for the columns
in question, MySQL can quickly determine the position to seek to in
the middle of the data file without having to look at all the data. If
a table has 1,000 rows, this is at least 100 times faster than reading
sequentially.
You can use
SHOW CREATE TABLE UPDATE_TIME;
to get the CREATE statement and see, if an index on MOD_DATE is defined.
To add an Index you can use
CREATE INDEX
CREATE [UNIQUE|FULLTEXT|SPATIAL] INDEX index_name
[index_type]
ON tbl_name (index_col_name,...)
[index_option]
[algorithm_option | lock_option] ...
see http://dev.mysql.com/doc/refman/5.6/en/create-index.html
Make sure that both of those fields are indexed.
Then I would just run -
select max(mod_date) from table
or create_date, whichever one.
Make sure to create 2 indexes, one on each date field, not a compound index on both.
As for a discussion of the difference between this and using limit, see MIN/MAX vs ORDER BY and LIMIT
Use EXPLAIN:
http://dev.mysql.com/doc/refman/5.0/en/explain.html
This tells You how mysql executes statement, thanks to that You can figure out most efficient way, cause it depends on Your db structure and there is no one universal solution.
I like being able to go to phpmyadmin and seeing my rows ordered.
Is there an easy way to insert a row into the correct order by it's primary key?
There is no order you can rely on.
Unless you specify ORDER BY, there are no guarantees about what rows
will be returned first.
But in practice, the order will typically match the clustered index
order, and will not vary between calls. Don't rely on this behaviour.
Does MySQL's LIMIT keyword guarantee order of returned data?
There is no default sort order. Even if the table has a clustered
index, you are not guaranteed to get the results in that order. You
must use an order by clause if you want a specific order.
SQL best practice to deal with default sort order
Just insert the data; tables are not fundamentally ordered. The only ordering of table results occurs when you define it yourself in your select statement.
Is it possible to insert data into the mid section of a table using the INSERT command?
You can't. There is no such thing.
To be precise: in MyISAM engine, current order is whatever the order
was when the last ALTER TABLE ORDER BY was performed, modified by all
the deletions and insertions since.
InnoDB just sorts the tables as it wishes.
view current table order
However be aware of the advice from another poster above: the order in
which rows are returned without an ORDER BY clause is arbitrary and
may change without notification. It would be best to amend your table
if at all possible.
Reverse the “natural order” of a MySQL table without ORDER BY?
.... and maaany more.