Does a sort use a MySQL index if there is an index on the sorting column? Also what other things is the index used for?
What difference does it make in a combined and separate indexes of the columns?
Yes, MySQL uses your index to sort the information when the order is by the sorted column.
Also, if you have indexes in all columns that you have added to the SELECT clause, MySQL will not load the data from the table itself, but from the index (which is faster).
The difference between combined and separate indexes is that MySQL can not use more than one index per query, so, if your query filters by many columns and you would like to have it correctly indexed you will need to create a combined index of all columns.
But before adding lots of indexes to your tables, remember that each index makes insert/update/delete operations go slower.
I would also highly recommend the High Performance MySQL book by O'Reilly that will cover in depth all of these issues and a lot of other hints you need to know to really be able to use MySQL to the limit.
Related
Consider we have A,B,C,D,E,F,G,H columns in my table and if I make composite indexing on column ABCDE because these column are coming in where clause and then I want composite indexing on ABCEF then I create new composite indexing on ABCEF in same table but in a different query, we want indexing on column ABCGH for that i make another composite indexing in same table,
So my question is can we make too many composite indexes as per our requirements because sometimes our column in where clause gets change and we have to increase its performance so tell me is there any other way to optimise the query or we can make multiple composite indexes as per the requirements.
I tried multiple composite indexing and it did not give me any problem till now but still i want to know from all of you that will it give me any problem in future or is it ok to use only composite indexes and not single index.
Your answers will help me alot waiting for your replies.
Thanks in advance.
You can have as many as you want. However, each additional index has a cost when updating, inserting or deleting. The trick is to
find common segments and make indexes for those.
Or create them as required when queries are too slow.
As an example, if you are "needing" indexes for ABCDE, ABDEF, and ABGIH then create an index on just AB
InnoDB supports up to 64 indexes per table (cf. https://dev.mysql.com/doc/refman/8.0/en/innodb-limits.html).
If you try to create a composite index for every permutation of N columns, you would need N-factorial indexes. So for 8 columns, you would need 40,320 indexes. Clearly this is more than InnoDB supports.
You probably don't need that many indexes. In practice, I've rarely seen more than 6 indexes in a given table. All queries that are needed are optimized by one of those.
I understand you said sometimes you change the terms in your query's WHERE clause, so it might need a composite index with different columns.
You can rely on indexes that have a subset of all the columns that would be optimal. That won't be 100% optimized, but it will be better than no index.
You can't predict the optimal set of indexes for a given query until you write that query.
There is a limit of 64 secondary indexes (at least in InnoDB).
Order the indexes so that columns being tested with = come first. (The order of those columns in the INDEX does not matter.)
The leftmost columns in an index are the most important.
There is little or now use in including more than one column that will be searched by a range.
Study your likely queries, and find the most common combinations of 2 or 3 columns; build indexes starting with those.
In your two examples (ABCDEFGH and ABCEF), ABC would work for both (assuming at least A and B are tested with =). If you do throw on more columns, that one INDEX can still be used for both cases.
Maybe you would what to declare both ABCDEFGH and BCEFA; This handles your ABCDEF case, plus cases that have B, but not A. (Remember 'leftmost'.)
Use the SlowLog to find the slowest queries and make better indexes for them.
More on indexing: Index Cookbook
Each index requires space on the disk to be stored, and time to be updated every time you update(/insert/delete) an indexed column value.
So as long as you don't run out of storage or write operations are too slow, because you have to update too many indexes, you are not limited to create as many specific indexes as you want.
This depends on your use case and should be measured with production like data.
A common solution would be to create one index specific for your most important query e.g. in your case ABCDE.
Other queries can still use the as many columns from left to right until there is a first difference. e.g. a query searching for ABCEF could still use ABC on the previous mentioned index.
To also utilise column E you could add a where condition to D to your query in a way you know it matches all values e.g. D < 100 if you know there are only values 1-99.
Consider fetching data with
SELECT * FROM table WHERE column1='XX' && column2='XX'
Mysql will filter the results matching with the first part of WHERE clause, then with the second part. Am I right?
Imagine the first part match 10 records, and adding the second part filters 5 records. Is it needed to INDEX the second column too?
You are talking about short circuit evaluation. A DBMS has cost-based optimizer. There is no guarantee wich of both conditions will get evaluated first.
To apply that to your question: Yes, it might be benificial to index your second column.
Is it used regulary in searches?
What does the execution plan tell you?
Is the access pattern going to change in the near future?
How many records does the table contain?
Would a Covering Index would be a better choice?
...
Indexes are optional in MySQL, but they can increase performance.
Currently, MySQL can only use one index per table select, so with the given query, if you have an index on both column1 and column2, MySQL will try to determine the index that will be the most beneficial, and use only one.
The general solution, if the speed of the select is of utmost importance, is to create a multi-column index that includes both columns.
This way, even though MySQL could only use one index for the table, it would use the multi-column index that has both columns indexed, allowing MySQL to quickly filter on both criteria in the WHERE clause.
In the multi-column index, you would put the column with the highest cardinality (the highest number of distinct values) first.
For even further optimization, "covering" indexes can be applied in some cases.
Note that indexes can increase performance, but with some cost. Indexes increase memory and storage requirements. Also, when updating or inserting records into a table, the corresponding indexes require maintenance. All of these factors must be considered when implementing indexes.
Update: MySQL 5.0 can now use an index on more than one column by merging the results from each index, with a few caveats.
The following query is a good candidate for Index Merge Optimization:
SELECT * FROM t1 WHERE key1=1 AND key2=1
When processing such query RDBMS will use only one index. Having separate indices on both colums will allow it to choose one that will be faster.
Whether it's needed depends on your specific situation.
Is the query slow as it is now?
Would it be faster with index on another column?
Would it be faster with one index containing both
columns?
You may need to try and measure several approaches.
You don't have to INDEX the second column but it may speed up your SELECT
Im trying to understand indexes better for when I use Mysql. One issue is Im still having a hard time to determine what type of index I should use such as individual indexes, multi column indexes, covering indexes etc.
One question I have is, is there a general rule to decide what type of indexes to use? When I design my database layout I dont know exactly what all queries will be used until the application is done being built. For one table I could query on one or multiple fields as well as query it for reporting. So if I query a table like so:
SELECT * FROM table1 WHERE field1 = this AND field2 = that GROUP BY field3 ORDER BY field4
Would I create a multiple column index on field1,field3,field3 and field4?
Also what if I have a different query on the same table like:
SELECT * FROM table1 WHERE field1 = this and field3 = that
If I had the multiple column index from the first query will that same index work for the second query since field1 is on the farthest left of the index?
And another question I had was is there a specific order mysql looks for indexes? So for multiple column or a covering index do I add indexes in order of the where clause? Then anything in group clause then anything in order clause? Or does mysql automatically do this?
Sorry for all the questions, just looking for help on this.
Engine
First you have to decide which Engine you want to use for a given table
InnoDB is preferable (transactions...) but does not offer fulltext index
If you need fulltext index, you have to chose MyISAM
(Full text index keeps an index based on words in a column)
Tables
You have to know that MySQL uses only one index per table maximum in a join. So, don't expect MySQL to combine two indexes of a given table.
Multi-columns
Chose the order of the column based on the queries, provided that MySQL can use the top of the index if necessary
For instance
CREATE INDEX myindex ON mytable (col1,col2,col3)
MySQL can use (col1), (col1,col2) and (col1,col2,col3) as index. So to answer your question, your index should be created on
(field1,field3,field2,field4).
since your two queries needs (field1,field3) and (field1,field2,field3,field4).
When I design my database layout I dont know exactly what all queries will be used until the application is done being built
Correct. Don't build indexes until you know all the queries. It's okay to add, change, alter and remove indexes. Indeed, good designers change the indexes as the use of the software changes.
Would I create a multiple column index on field1,field3,field3 and field4?
Rarely.
If I had the multiple column index from the first query will that same index work for the second query since field1 is on the farthest left of the index?
No.
And another question I had was is there a specific order mysql looks for indexes?
No.
So for multiple column or a covering index do I add indexes in order of the where clause?
No
Then anything in group clause then anything in order clause?
No.
Or does mysql automatically do this?
More-or-less.
Here's the rule.
Design the database.
Write the queries.
Find the most common queries. 20% of your queries do 80% of the work. Focus on the few, slow queries that need indexes.
Explain the query execution plans for only the most common queries. There's an EXPLAIN statement for this.
Measure the performance of those queries with realistic loads of data. You have to build fake data for this. Some queries will be slow. Indexes may help. Some queries will not be slow.
Now comes the hard part. Try different indexes until (a) the explain plan looks optimal and (b) the measured query performance meets your expectations.
You cannot get all queries to be fast.
You do not build indexes for all queries.
Focus on the 20% of the queries that cost 80% of the time.
Sorry all,
I do have this mini table with 3 columns but, I will list the values either by one or other column.
1) Is it ok to have two of those columns with indexes?
2) Should we have only one index per table?
3) At the limit, if we have a table with 100 columns for example, and we have 50 of them with indexes, is this ok?
Thanks,
MEM
It's fine to have many indexes, even multiple indexes on one table, as long as you are using them.
Run EXPLAIN on your queries and check which indexes are being used. If there are indexes that are not being used by any queries then they aren't giving you any benefit and are just slowing down modifications to the table. These unused indexes should be removed.
You may also want to consider multiple-column indexes if you have not already done so.
To quickly answer your questions, I think:
Yes, it's OK to have two or even more columns with indexes
Not necessarily. You can have more than one index per table.
It might be OK, it might be not OK. It depends. The thing with indexes is that they take space (in DISK) and they make operations that modify your data (INSERT/UPDATE/DELETE) slower since for each and everyone of them, there will be a TABLE and INDEX update involved.
Your index creation should be driven by your queries. See what queries are you going to have on your system and create indexes accordingly.
There is no problem with having more than one index per table, and no, one index per table isn't really a guideline.
Having too many indexes is inefficient because
It requires additional storage
MySQL needs to determine which index to use for a particular query
Edit : As per Pablo and Mark, you need to understand how your data is being accessed in order for you to build effective indices. You can then refine and reduce the indexes.
How do you create MySQL index? Multi-column or one column per index?
What's your thinking?
This is something that depends heavily on the data stored in your tables. Is this a db with a single table with queries that are only run against the unique id/PK of the record? If so, then just index a single column, the id column. If not, it may call for indexing across more than one column.
It's a case by case issue that can't be generalized. Indexing can be very tricky and over indexing can actually hurt performance (especially on inserts and updates). Make extensive use of the "EXPLAIN" statement in mysql to see how your index changes affect the way mysql actually queries the data.