Is it possible to find out how often an index of a MySQL table was used?
I have several indices for a table and I'd like to find out, if there are indice which are not used by MySQL.
Yes, it is. You should query Performance Schema:
select * from performance_schema.table_io_waits_summary_by_index_usage
where object_schema = 'your_schema'
The count_star column show how many times each index was used since MySQL was started. If you add the following, you got the never used indexes:
and count_star = 0
Addition to #user1970667's answer, you may also use:
select * from sys.schema_unused_indexes;
to get a list of unused indexes.
NOTE: This answer is no longer valid as of 5.5.3!
See https://stackoverflow.com/a/43205887/1251127
Original answer below.
Currently, MySQL does not offer statistics on index usage.
One way to generate those stats on your own would be to log all queries (Be careful of the extra i/o here) and parse it out. Percona has a nice tool for this, pt-index-usage.
Related
I am trying to speed up a simple SELECT query on a table that has around 2 million entries, in a MariaDB MySQL database. It took over 1.5s until I created an index for the columns that I need, and running it through PhpMyAdmin showed a significant boost in speed (now takes around 0.09s).
The problem is, when I run it through my PHP server (mysqli), the execution time does not change at all. I'm logging my execution time by running microtime() before and after the query, and it takes ~1.5s to run it, regardless of having the index or not (tried removing/readding it to see the difference).
Query example:
SELECT `pair`, `price`, `time` FROM `live_prices` FORCE INDEX
(pairPriceTime) WHERE `time` = '2022-08-07 03:01:59';
Index created:
ALTER TABLE `live_prices` ADD INDEX pairPriceTime (pair, price, time);
Any thoughts on this? Does PHP PDO ignore indexes? Do I need to restart the server in order for it to "acknowledge" that there is a new index? (Which is a problem since I'm using a shared hosting service...)
If that is really the query, then it needs an INDEX starting with the value tested in the WHERE:
INDEX(time)
Or, to make a "covering index":
INDEX(time, pair, price)
However, I suspect that most of your accesses involve pair? If so, then other queries may need
INDEX(pair, time)
especially if you as for a range of times.
To discuss various options further, please provide EXPLAIN SELECT ...
PDO, mysqli, phpmyadmin -- These all work the same way. (A possible exception deals with an implicit LIMIT on phpmyadmin.)
Try hard to avoid the use of FORCE INDEX -- what helps on today's query and dataset may hurt on tomorrow's.
When you see puzzling anomalies in timings, run the query twice. Caching may be the explanation.
The mysql documenation says
The FORCE INDEX hint acts like USE INDEX (index_list), with the addition that a table scan is assumed to be very expensive. In other words, a table scan is used only if there is no way to use one of the named indexes to find rows in the table.
MariaDB documentation Force Index here says this
FORCE INDEX works by only considering the given indexes (like with USE_INDEX) but in addition, it tells the optimizer to regard a table scan as something very expensive. However, if none of the 'forced' indexes can be used, then a table scan will be used anyway.
Use of the index is not mandatory. Since you have only specified one condition - the time, it can choose to use some other index for the fetch. I would suggest that you use another condition for the select in the where clause or add an order by
order by pair, price, time
I ended up creating another index (just for the time column) and it did the trick, running at ~0.002s now. Setting the LIMIT clause had no effect since I was always getting 423 rows (for 423 coin pairs).
Bottom line, I probably needed a more specific index, although the weird part is that the first index worked great on PMA but not through PHP, but the second one now applies to both approaches.
Thank you all for the kind replies :)
I have been learning query optimization, increase query performance and all but in general if we create a query how can we know if this is a wise query.
I know we can see the execution time below, But this time will not give a clear indication without a good amount of data. And usually, when we create a new query we don't have much data to check.
I have learned about clauses and commands performance. But is there is anything by which we can check the performance of the query? Performance here is not execution time, it means that whether a query is "ok" or not, without data dependency.
As we cannot create that much data that would be in live database.
General performance of a query can be checked using the EXPLAIN command in MySQL. See https://dev.mysql.com/doc/refman/5.7/en/using-explain.html
It shows you how MySQL engine plans to execute the query and allows you to do some basic sanity checks i.e. if the engine will use keys and indexes to execute the query, see how MySQL will execute the joins (i.e. if foreign keys aren't missing) and many more.
You can find some general tips about how to use EXPLAIN for optimizing queries here (along with some nice samples): http://www.sitepoint.com/using-explain-to-write-better-mysql-queries/
As mentioned above, Right query is always data-dependent. Up to some level you can use the below methods to check the performance
You can use Explain to understand the Query Execution Plan and that may help you to correct some stuffs. For more info :
Refer Documentation Optimizing Queries with EXPLAIN
You can use Query Analyzer. Refer MySQL Query Analyzer
I like to throw my cookbook at Newbies because they often do not understand how important INDEXes are, or don't know some of the subtleties.
When experimenting with multiple choices of query/schema, I like to use
FLUSH STATUS;
SELECT ...;
SHOW SESSION STATUS LIKE 'Handler%';
That counts low level actions, such as "read next record". It essentially eliminates caching issues, disk speed, etc, and is very reproducible. Often there is a counter in that output (or multiple counters) that match the number of rows in the table (sometimes +/-1) -- that tells me there are table scan(s). This is usually not as good as if some INDEX were being used. If the query has a LIMIT, that value may show up in some Handler.
A really bad query, such as a CROSS JOIN, would show a value of N*M, where N and M are the row counts for the two tables.
I used the Handler technique to 'prove' that virtually all published "get me a random row" techniques require a table scan. Then I could experiment with small tables and Handlers to come up with a list of faster random routines.
Another tip when timing... Turn off the Query_cache (or use SELECT SQL_NO_CACHE).
I have a giant MYSQL table and the hostgator shared server I use has 4gb RAM. I am trying to execute the following query simple using phpmyadmin:
DELETE FROM Table1_main where date = '2009-12-31'
However, this query just times out because there is insufficient RAM. How can I execute this query without buying a higher performance server?
Is there an index on that column? That's usually the way to speed up a simple query like this.
Here's how to create an index on that column:
CREATE INDEX ON table1_main (date);
NOTE: THIS IS A HACK IF you just want it to get it done
I bet there is a better way to do this.
DELETE FROM Table1_main where date = '2009-12-31' limit 1000000
try to increase the number until it breaks.
If you have MySQL greater than or equal to version 5.1, then you can use partitions.
The other solution would be to use a cycle to delete N records at a time until the total is deleted.
Please take a look at insights on both approaches at:
http://mysql.rjweb.org/doc.php/deletebig
Hope that helps,
I have a set of quite complex SELECT queries which use a lot of disk space (I see this from df -h while running). Is there a way to estimate the temporary disk space required for a query before starting it?
You can use the EXPLAIN keyword to describe how your joins will effect the number of rows that will be joined together. This will also assist you in properly using keys if they are not already present. Explain will tell you when it thinks it will need to use temp tables (disk space). Based on the size of the rows being joined, you can then roughly estimate your disk space need.
See the docs on explain here:
http://dev.mysql.com/doc/refman/5.0/en/explain.html
Basically though, just prepend "Explain" to your select query to get info output. I believe you can also do this programatically if needed and use the results in your actual code, say for instance you needed to calculate(estimate) a large query run time and display it to the user before proceeding.
I am cleaning up duplicate indices from (innodb)tables in a mysql database. The database has about 50 tables.
While I can check for duplicate keys using pt-duplicate-key-checker, I was wondering if there is a tool that could help me find out least recently used indices from all the tables.
For eg. , if table "A" has three indices defined "index_1", "index_2" and "index_3", and out of them "index_3" is used least frequently , assume its used every 1/10000 queries made on the table, then the output of the script or tool should be "index_3".
Is there a good way or a tool that could help me run this analysis on the database?
Thanks in advance.
Starting with MySQL 5.6, the performance_schema instruments table io, and computes aggregated statistics by table, and by index.
See table performance_schema.table_io_waits_summary_by_index_usage:
http://dev.mysql.com/doc/refman/5.6/en/table-waits-summary-tables.html#table-io-waits-summary-by-index-usage-table
Finding the least recently used index involves time and timestamps.
What the performance schema measure is counting io against an index, so it can be used to find the least often used index, which in practice should be pretty close.
Full example here:
http://sqlfiddle.com/#!9/c129c/4
Note: This question is also duplicated in
https://dba.stackexchange.com/questions/25406/find-least-recently-used-mysql-table-index