I am trying to fetch the last created item in a large table like this:
SELECT `raw_detection`.* FROM `raw_detection`
WHERE `raw_detection`.`duplicated` = 0
AND `raw_detection`.`audio_source_id` = 100
ORDER BY created_at desc LIMIT 1
But this query takes a long time to run (more than 2 seconds).
I have this index:
KEY `index_raw_detections_audio_source`(`audio_source_id`,`duplicated`,`created_at`)
Is there any better way to fetch the last created item for a specific audio source?
Your key references three columns. It cannot be used to speed up queries using only the created_at portion of the key. Try creating an additional key for just created_at.
For reference, from the MySQL doc:
If the table has a multiple-column index, any leftmost prefix of the index can be used by the optimizer to look up rows. For example, if you have a three-column index on (col1, col2, col3), you have indexed search capabilities on
(col1), (col1, col2), and (col1, col2, col3).
MySQL cannot use the index to perform lookups if the columns do not form a leftmost prefix of the index.
https://dev.mysql.com/doc/refman/5.0/en/multiple-column-indexes.html
In your select, specify the columns needed rather than doing select *
Related
I have a very simple query
SELECT col1, col2, col3, col4 FROM table FORCE INDEX (col2)
WHERE col2 IN ('there', 'are, 'around', 'six', 'values', 'here')
with index col2 for col2. My table has around 10 millions row. I used FORCE INDEX here because there are other indices in my table and MySQL uses one of other indices instead of index col2. The other index is very slow for this query.
List of all indices in my table:
INDEX col2 (col2)
UNIQUE INDEX ind1 (col1, col2)
INDEX ind2 (col1, col2)
INDEX ind3 (col2, col1)
This query (with FORCE INDEX) is not slow (takes 6 seconds on AWS RDS free tier) but there is a need to make it as fast as possible. Is there any thing else I could do to speed up this query?
First, you should try not forcing the index on col2, and instead just look at the explain plan. It is likely that a single column index on col2 would be used here. However, you can try adding the following composite covering index on your table:
CREATE INDEX idx ON yourTable (col2, col1, col3, col4);
This index would cover the WHERE clause, and also includes the other columns which appear in the SELECT clause. If it chooses, MySQL could use this index to completely cover the entire query without needing to seek back to the clustered index (i.e. the original table).
INDEX col2 (col2)
UNIQUE INDEX ind1 (col1, col2)
INDEX ind2 (col1, col2)
INDEX ind3 (col2, col1)
Some of these indexes are redundant. MySQL can use (col2, col1) for searches on col2 as well as searches on both col2 and col1. And ind2 is fully redundant with ind1.
The redundancy might be confusing the optimizer.
To cover all combinations of col1 and col2, as well as enforce uniqueness, you only need...
INDEX col2 (col2)
UNIQUE INDEX ind1 (col1, col2)
Removing the redundant indexes will speed up inserts and save space.
See 8.3.6 Multiple-Column Indexes.
The query planner makes its guesses based on table statistics. Sometimes those statistics are out of date. Try running analyze table to update them.
I've been dropped in a system that was poorly designed. Now I'm doing DBA on their DB, and I have a lot of situation like the following (Pseudo-code):
Table t1{
c1;
c2;
c3;
c4;
key(c1);
key(c2);
key(c1,c2);
key(c1,c2,c3);}
Are the single column indexes really necessary, since I already have a multicolumn one containing those columns?
Or on the other hand - is the multiline column needed since I already have the single column ones?
A short excerpt from the documentation page about how MySQL uses indexes:
If the table has a multiple-column index, any leftmost prefix of the index can be used by the optimizer to look up rows. For example, if you have a three-column index on (col1, col2, col3), you have indexed search capabilities on (col1), (col1, col2), and (col1, col2, col3). For more information, see Section 8.3.5, “Multiple-Column Indexes”.
You better remove the indexes on (c1) and (c1,c2). They are not used but they use storage space and consume processor power to be kept up-to-date when the table data changes.
The single column index on c1 is redundant. The two column index is redundant with the three column index as well.
The only two indexes you need are on (c2) and (c1, c2, c3).
MySQL has pretty good documentation on composite indexes.
I have multi-column index for 2 columns. Can I make first column unique without making separate index for that?
If I understand correctly mysql can use only first column in this index for lookups, so can it use it to detect uniqueness?
The short answer is "No". Because it doesn't make much sense.
Indeed, MySQL is able to use a multiple-column index for operations that use only the leftmost "n" columns from the index definition.
Let's say you have an index on columns (col1, col2). MySQL can use it to find records matching conditions on both col1 and col2, GROUP BY col1, col2 or ORDER BY col1, col2. It is important to notice that col1 and col2 needs to used in this order in the GROUP BY or ORDER BY clause. Their order doesn't matter on WHERE or ON clauses as long as both are used.
MySQL can also use the same index for WHERE or ON conditions and GROUP BY or ORDER BY clauses that contain only col1. It cannot, however, use the index if col2 appears without col1.
What happens when you have an index on columns (col1, col2) and all the rows have distinct values in column col1?
Let's assume we have a table that have distinct values in column col1 and it has an index on columns (col1, col2). When MySQL needs to find the rows that match WHERE col1 = val1 AND col2 = val2, by consulting the index it can find the row that have col1 = val1. It doesn't need to use the index to refine the list of candidate rows because there is no list: there is at most one row having col1 = val1.
Sure, most of the times MySQL will use the index to check if col2 = val2 but having col2 in this index doesn't bring more useful information to the index. The storage space it takes and the processing power it uses on table data updates are too big for the tiny contribution it adds to rows searching.
The whole purpose of having indexes on multiple columns is to help searching by shrinking the list of matching rows for a given set of values when the columns included in a multiple-column index cannot be used individually because they don't contain enough distinct values.
Technically speaking, there is no way to tell MySQL you want to have a multiple-column index on (col1, col2) that must have unique values on col1. Create an UNIQUE INDEX on col1 instead. Then think about the data you have in the table and the queries you run against it and decide if another index on col2 only isn't better than the multiple-column index on (col1, col2).
In order to decide you can create the new indexes (UNIQUE on col1, INDEX on col2), put EXPLAIN in front of the most frequent queries you run on the table and check what index will pick MySQL up for use.
You need to have enough data (thousands of rows, at least, more is better) in the table to get accurate results.
You asked.
I have multi-column index for 2 columns. Can I make first column unique without making separate index for that?
The answer is no. You need a separate unique index on the first column to enforce a uniqueness constraint.
I have a database table (potentially huge, with hundreds of millions of records in the future) on which I would execute the following query very often:
select *
from table1
where col1 = [some number]
order by col2
Obviously having an index on "col1" would make it run fast. col1 is not unique, so many rows (2000+ I expect) would be returned.
Does it make sense to create an index on (col1, col2)? Would MySQL use it for this query?
Also, if I just query without "order by" part, would this index be used as well for the "where" part?
Yes, it will help, mysql will use composite index with first part on WHERE and second part on ORDER BY. You can read about ORDER BY optimization here: http://dev.mysql.com/doc/refman/5.5/en/order-by-optimization.html
This is a question that I've had forever.
As far as I know the order of indices matter. So an index like [first_name, last_name] is not the same as [last_name, first_name], right?
If I only define the first index, does it mean that it will only used for
SELECT * FROM table WHERE first_name="john" AND last_name="doe";
and not for
SELECT * FROM table WHERE last_name="doe" AND first_name="john";
Since I am using a ORM, I have no idea in which order these columns are going to be called. Does that mean that I have to add indices on all permutations? That is doable if I have a 2 column index, but what happens if my index is on 3 or 4 columns?
Index order matters when your query conditions only apply to PART of the index. Consider:
SELECT * FROM table WHERE first_name="john" AND last_name="doe"
SELECT * FROM table WHERE first_name="john"
SELECT * FROM table WHERE last_name="doe"
If your index is (first_name, last_name) queries 1 and 2 will use it, query #3 won't.
If your index is (last_name, first_name) queries 1 and 3 will use it, query #2 won't. Changing the condition order within WHERE clause has no effect in either case.
Details are here
Update:
In case the above is not clear - MySQL can only use an index if the columns in query conditions form a leftmost prefix of the index. Query #2 above can not use (last_name, first_name) index because it's only based on first_name and first_name is NOT the leftmost prefix of the (last_name, first_name) index.
The order of conditions WITHIN the query does not matter; query #1 above will be able to use (last_name, first_name) index just fine because its conditions are first_name and last_name and, taken together, they DO form a leftmost prefix of (last_name, first_name) index.
ChssPly76 is correct that the order of boolean expressions does not have to match the order of columns in the index. Boolean operators are commutative, and the MySQL optimizer is smart enough to know how to match the expression to the index in most cases.
I also want to add that you should learn how to use the EXPLAIN feature of MySQL so you can see for yourself which indexes the optimizer will choose for a given query.
Why not to extend the answer a little bit to make completely everything crystal clear at once.
If the table has a multiple-column index, any leftmost prefix of the index can be used by the optimizer to find rows. For example, if you have a three-column index on (col1, col2, col3), you have indexed search capabilities on (col1), (col1, col2), and (col1, col2, col3).
MySQL cannot use an index if the columns do not form a leftmost prefix of the index. Suppose that you have the SELECT statements shown here:
SELECT * FROM tbl_name WHERE col1=val1;
SELECT * FROM tbl_name WHERE col1=val1 AND col2=val2;
SELECT * FROM tbl_name WHERE col2=val2;
SELECT * FROM tbl_name WHERE col2=val2 AND col3=val3;
If an index exists on (col1, col2, col3), only the first two queries use the index. The third and fourth queries do involve indexed columns, but (col2) and (col2, col3) are not leftmost prefixes of (col1, col2, col3). - MySQL dev