Currently I am creating indexes as I need them for a particular sql query.
But they are starting to overlap each other.
Is there any rule to define them effectively?
For example:
If I have two indexes for column1 and column2, does the composite index by column1, column2 improve select by both columns?
What is there any difference between an index by column1 and column2 over index by column2 and column1?
Q: If I have two indexes for column1 and column2, does the composite index by column1, column2 improve select by both columns?
Yes then composite Index is better.
From Mysql
"mysql> SELECT * FROM tbl_name WHERE col1=val1 AND col2=val2;
If a multiple-column index exists on col1 and col2, the appropriate rows can be fetched directly. If separate single-column indexes exist on col1 and col2, the optimizer attempts to use the Index Merge optimization (see Section 8.3.1.4, “Index Merge Optimization”), or attempts to find the most restrictive index by deciding which index excludes more rows and using that index to fetch the rows."
Q: What is there any difference between an index by column1 and column2 over index by column2 and column1?
Yes it will make difference. It depends on how you form your query.
From the Mysql docs:
Example If you have index like below on table :
INDEX name (last_name,first_name)
"The name index is an index over the last_name and first_name columns. The index can be used for lookups in queries that specify values in a known range for combinations of last_name and first_name values. It can also be used for queries that specify just a last_name value because that column is a leftmost prefix of the index."
You will get advantage of index for below query:
SELECT * FROM test WHERE last_name='Widenius';
But index is not used for lookups in the following queries:
SELECT * FROM test WHERE first_name='Michael';
Hope this will help !!
An index over two columns A and B is working as an index for column A also, but not as an index for column B.
I don't know if there is an simple rule for indexes, maybe just look for the columns involved in where clauses and order by statements in your queries and evaluate which to use.
Keep in mind that an index makes sense for a large number of rows where you search for a small subset. Indexes also slow down insertion and updates, so use them wisely. It is often enough to simply index all rows that are used to JOIN and add further ones if you run into performance issues.
If I have two indexes for column1 and column2, does the composite index by column1, column2 improve select by both columns?
Yes, it does. Separate indexes have to get merged first or just one index is used.
What is there any difference between an index by column1 and column2 over index by column2 and column1?
In a compound index the order of the columns matters, yes. If your query has only column2 in the WHERE clause, a compound index over (column1, column2) will not be used.
If you have a compound index over (column1, column2) it can also be used if your query only has column1 in WHERE clause.
For additional information see other answers I gave to questions about indexing:
MYSQL Long super-keys
MySQL query optimization of LIKE term% ORDER BY int
Q)If I have two indexes for column1 (A) and column2 (B), does the composite index by column1, column2 improve select by both columns?
yes if you have a just two indices A and B, SELECT something from table where A = 1 and B =2 will be using just one of two indices, but if you have compound index A,B it will use it, which should be faster
Q) What is there any difference between an index by column1 and column2 over index by column2 and column1?
sequence do matters in compound indices. Suppose you have the same query as above, and your table is 1 mln entries and only 50 of them match A=1 and 10000 match B=1, than compound index A,B will perform much better than B,A. So you need to choose first element of the index with the smallest cardinality.
This might be usefull
http://dev.mysql.com/doc/refman/5.0/en/multiple-column-indexes.html
http://stackoverflow.com/questions/1823685/when-should-i-use-a-composite-index
Related
I have multi-column index for 2 columns. Can I make first column unique without making separate index for that?
If I understand correctly mysql can use only first column in this index for lookups, so can it use it to detect uniqueness?
The short answer is "No". Because it doesn't make much sense.
Indeed, MySQL is able to use a multiple-column index for operations that use only the leftmost "n" columns from the index definition.
Let's say you have an index on columns (col1, col2). MySQL can use it to find records matching conditions on both col1 and col2, GROUP BY col1, col2 or ORDER BY col1, col2. It is important to notice that col1 and col2 needs to used in this order in the GROUP BY or ORDER BY clause. Their order doesn't matter on WHERE or ON clauses as long as both are used.
MySQL can also use the same index for WHERE or ON conditions and GROUP BY or ORDER BY clauses that contain only col1. It cannot, however, use the index if col2 appears without col1.
What happens when you have an index on columns (col1, col2) and all the rows have distinct values in column col1?
Let's assume we have a table that have distinct values in column col1 and it has an index on columns (col1, col2). When MySQL needs to find the rows that match WHERE col1 = val1 AND col2 = val2, by consulting the index it can find the row that have col1 = val1. It doesn't need to use the index to refine the list of candidate rows because there is no list: there is at most one row having col1 = val1.
Sure, most of the times MySQL will use the index to check if col2 = val2 but having col2 in this index doesn't bring more useful information to the index. The storage space it takes and the processing power it uses on table data updates are too big for the tiny contribution it adds to rows searching.
The whole purpose of having indexes on multiple columns is to help searching by shrinking the list of matching rows for a given set of values when the columns included in a multiple-column index cannot be used individually because they don't contain enough distinct values.
Technically speaking, there is no way to tell MySQL you want to have a multiple-column index on (col1, col2) that must have unique values on col1. Create an UNIQUE INDEX on col1 instead. Then think about the data you have in the table and the queries you run against it and decide if another index on col2 only isn't better than the multiple-column index on (col1, col2).
In order to decide you can create the new indexes (UNIQUE on col1, INDEX on col2), put EXPLAIN in front of the most frequent queries you run on the table and check what index will pick MySQL up for use.
You need to have enough data (thousands of rows, at least, more is better) in the table to get accurate results.
You asked.
I have multi-column index for 2 columns. Can I make first column unique without making separate index for that?
The answer is no. You need a separate unique index on the first column to enforce a uniqueness constraint.
I have a query like
SELECT * FROM Table_name WHERE column1 = '1' AND column2 IN ('1','2','3');
And index exists on (column1, column2, column3). Is my above query used index I have created or not? Basically I am confused with the IN keyword, without this it is using, but with IN I am not sure. Please explain me.
MySQL can use indexes with IN conditions. If you only have an index on column2, it will most likely be used. If you have indexes on each of column1 and column2, only one of them can be used, and the query planner will have to decide which one seems better for a particular query. If you have a composite index on (column1, column2) then it should be able to use that index to match both columns in the WHERE clause.
MySQL is capable of using indices with IN .. now, is MySQL using indices in the particular query? Well, ask the query planner!
In this case an index over (column1, column2, ..) could be used - because all the leftward components have been satisfied. That is, an index seek could be done on column1 (for =), and then column2 (for IN). But again, ask the query planner as unexpected plans are not unheard of; just because the query planner could choose an index doesn't mean that it will.
See EXPLAIN, for how to ask:
When EXPLAIN is used with an explainable statement, MySQL displays information from the optimizer about the statement execution plan. That is, MySQL explains how it would process the statement, including information [like index usage] about how tables are joined and in which order ..
.. With the help of EXPLAIN, you can see where you should add indexes to tables so that the statement executes faster by using indexes to find rows.
We have an update query that needs to use the "order by" clause to ensure we update the correct row as follows:
UPDATE `plugin_name_codes`
USE|FORCE INDEX abc
SET `plugin_name_entry_id` = 2
WHERE `plugin_name_codes`.`plugin_name_form_id` = 3
AND (plugin_name_entry_id IS NULL)
ORDER BY /*plugin_name_form_id asc, plugin_name_entry_id asc,*/ id ASC
LIMIT 1
We want to use an index to do the sorting for the ID column, but:
Index Hints are not used in UPDATE queries (according to the MySQL documentation).
The ORDER BY clause appears to ignore the index no matter how I create the indexes (i.e.
using all three queried columns (plugin_name_entry_id, plugin_name_form_id, and id), or
just the first 2 columns.
and the no matter which columns I include in the ORDER BY clause, the query always
ignores the index.
This ORDER clause adds a full 1 second to each query which is very unacceptable. Anyone know how I can use the index to sort by ID, or some other workaround?
Just briefly on composite indexes; the first thing to consider is the way you will be querying your data. If you have a composite key such as (col1, col2) and you attempt to query SELECT * FROM tbl WHERE col2 = 5, the index will not be used because you need to SELECT all the keys from the left first.
With UPDATE, the only time an index will be used is with a WHERE clause (and possibly ORDER BY). Taking into consideration the information above, you will need to make sure that the left most column of your index is being used (plugin_name_form_id). If you had a composite key of (col1, plugin_name_form_id), that update statement would not make any use of the index. You could also reverse the order of the index (plugin_name_form_id, col1) for it to be useful.
Hope this helps.
I have a table with 3 columns. This table contains many raws (millions). When I select rows from the table I frequently use the following where clauses:
where column2=value1 and column3=value2
where column1=value
To speed up the select query I want to declare column1 and column2 as indexes. My questions is if declaring the second column as an index will not reduce the positive effect of declaring the first column as index.
I also would like to ask if declaring the second column as index will speed up the queries of this type: where column2=value1 and column3=value2.
ADDED
The column1, column2, and column3 are entity, attribute, value. It's very general. As entities I use person, movies, cities, countries and so on. Attributes are things like: "located in", "date of birth", "produced by".
You should create indexes that support your queries. In this case you want to create an index on column2,column3 together (not two separate indexes, but one index for the combination of columns) to support the first query, and another on column1 to support the second query. More generally, if a query uses a set of columns, adding an index for all those columns will speed it up (although there are many exceptions, of course).
An index on column2 would speed up the query column2=value1 and column1=value2, and so would an index on column2,column3 (the important thing is that column2 is the first column in the index).
When working with indexes the EXPLAIN keyword is very useful. Prefix your queries with EXPLAIN (e.g. EXPLAIN SELECT * FROM table) to get a description of how the database is going to perform your query. It will tell you if it's going to use an index, and in that case which.
Seems like neither of your plans are going to work. Based on both of the where clauses I would suggest having the primary key on column1 and a second index column2,column3. This would speed up both of your queries.
This is a question that I've had forever.
As far as I know the order of indices matter. So an index like [first_name, last_name] is not the same as [last_name, first_name], right?
If I only define the first index, does it mean that it will only used for
SELECT * FROM table WHERE first_name="john" AND last_name="doe";
and not for
SELECT * FROM table WHERE last_name="doe" AND first_name="john";
Since I am using a ORM, I have no idea in which order these columns are going to be called. Does that mean that I have to add indices on all permutations? That is doable if I have a 2 column index, but what happens if my index is on 3 or 4 columns?
Index order matters when your query conditions only apply to PART of the index. Consider:
SELECT * FROM table WHERE first_name="john" AND last_name="doe"
SELECT * FROM table WHERE first_name="john"
SELECT * FROM table WHERE last_name="doe"
If your index is (first_name, last_name) queries 1 and 2 will use it, query #3 won't.
If your index is (last_name, first_name) queries 1 and 3 will use it, query #2 won't. Changing the condition order within WHERE clause has no effect in either case.
Details are here
Update:
In case the above is not clear - MySQL can only use an index if the columns in query conditions form a leftmost prefix of the index. Query #2 above can not use (last_name, first_name) index because it's only based on first_name and first_name is NOT the leftmost prefix of the (last_name, first_name) index.
The order of conditions WITHIN the query does not matter; query #1 above will be able to use (last_name, first_name) index just fine because its conditions are first_name and last_name and, taken together, they DO form a leftmost prefix of (last_name, first_name) index.
ChssPly76 is correct that the order of boolean expressions does not have to match the order of columns in the index. Boolean operators are commutative, and the MySQL optimizer is smart enough to know how to match the expression to the index in most cases.
I also want to add that you should learn how to use the EXPLAIN feature of MySQL so you can see for yourself which indexes the optimizer will choose for a given query.
Why not to extend the answer a little bit to make completely everything crystal clear at once.
If the table has a multiple-column index, any leftmost prefix of the index can be used by the optimizer to find rows. For example, if you have a three-column index on (col1, col2, col3), you have indexed search capabilities on (col1), (col1, col2), and (col1, col2, col3).
MySQL cannot use an index if the columns do not form a leftmost prefix of the index. Suppose that you have the SELECT statements shown here:
SELECT * FROM tbl_name WHERE col1=val1;
SELECT * FROM tbl_name WHERE col1=val1 AND col2=val2;
SELECT * FROM tbl_name WHERE col2=val2;
SELECT * FROM tbl_name WHERE col2=val2 AND col3=val3;
If an index exists on (col1, col2, col3), only the first two queries use the index. The third and fourth queries do involve indexed columns, but (col2) and (col2, col3) are not leftmost prefixes of (col1, col2, col3). - MySQL dev