How to find the inefficient queries in mysql database ? I want to do performance tuning on my queries , but i coudn't find where my queries are located ? Please suggest me where can i find mysql queries for my tables .
Thanks
Prabhakaran.R
You can enable the general log and slow query logs.
Enabling general query log will log all the queries and might be heavy if you have many reads/writes. In slow query log, you can mention a threshold and only queries taking time beyond some time will be logged. Post that, you can manually analyze it or you can use tools provided( Percona has great tools)
Have you analyzed your queries with explain plans? You may be able to find a query that will return the result set you wish that isn't as heavy a load on the query engine. Remember to only select the columns you actually need (try to avoid SELECT *), well planned indexing and to use inner/outer joins in favour of a huge list of WHERE clause filters.
Good luck.
RP
In addition to what the others said, use pt-query-digest (from percona.com) to summarize the slowlog. That will tell you the worst queries.
Performance tuning often involves knowing what indexes to put on tables. My index cookbook is one place to learn how to build an INDEX for a given SELECT.
Related
We use two MySQL servers with the same setup and master-master replication. They are located behind a loadbalancer and get almost the same traffic, statements etc.
On server1, there are some cron jobs running additionally, which is the only difference.
However, we've seen that the query optimizer in some cases behaves different with the same query on both servers.
In those cases we have to use FORCE_INDEX to get the best results on both servers.
The main questions are:
Is there any metadata that is stored somewhere on the server that is used by the Query Optimizer?
If we have to backup and restore a database (with XtraBackup), will the query optimizer behave in the same way or is it built from scratch?
Thanks for any reply
Joachim
The Optimizer bases its actions partially on "statistics". Those statistics come from explicit ANALYZE TABLE or from certain changes to the table, such as "adding more than 10% to the table".
There is nothing to sync the stats between the two Masters, so they can drift apart. Even running ANALYZE TABLE on both will not necessarily bring them in sync. This is because of the "random probes" that are made into the table to concoct the stats.
FORCE INDEX is risky because "what helps today may hurt tomorrow".
Probably this is what is happening: You have a query that is on the borderline between picking one query plan and another. The Optimizer's analysis says they are about equivalent, but (for various reasons) they are not. This could lead to one server doing one thing (fast query plan) and the other doing something else (poor query plan).
There is no 'reliable' and 'consistent' way to solve your problem.
Let's try to tackle it a different way. Provide the query, the EXPLAIN, and SHOW CREATE TABLE. There may be a better index and/or a reformulation of the query that avoids the problem completely, and possibly runs faster than either of your current query plans.
I have been learning query optimization, increase query performance and all but in general if we create a query how can we know if this is a wise query.
I know we can see the execution time below, But this time will not give a clear indication without a good amount of data. And usually, when we create a new query we don't have much data to check.
I have learned about clauses and commands performance. But is there is anything by which we can check the performance of the query? Performance here is not execution time, it means that whether a query is "ok" or not, without data dependency.
As we cannot create that much data that would be in live database.
General performance of a query can be checked using the EXPLAIN command in MySQL. See https://dev.mysql.com/doc/refman/5.7/en/using-explain.html
It shows you how MySQL engine plans to execute the query and allows you to do some basic sanity checks i.e. if the engine will use keys and indexes to execute the query, see how MySQL will execute the joins (i.e. if foreign keys aren't missing) and many more.
You can find some general tips about how to use EXPLAIN for optimizing queries here (along with some nice samples): http://www.sitepoint.com/using-explain-to-write-better-mysql-queries/
As mentioned above, Right query is always data-dependent. Up to some level you can use the below methods to check the performance
You can use Explain to understand the Query Execution Plan and that may help you to correct some stuffs. For more info :
Refer Documentation Optimizing Queries with EXPLAIN
You can use Query Analyzer. Refer MySQL Query Analyzer
I like to throw my cookbook at Newbies because they often do not understand how important INDEXes are, or don't know some of the subtleties.
When experimenting with multiple choices of query/schema, I like to use
FLUSH STATUS;
SELECT ...;
SHOW SESSION STATUS LIKE 'Handler%';
That counts low level actions, such as "read next record". It essentially eliminates caching issues, disk speed, etc, and is very reproducible. Often there is a counter in that output (or multiple counters) that match the number of rows in the table (sometimes +/-1) -- that tells me there are table scan(s). This is usually not as good as if some INDEX were being used. If the query has a LIMIT, that value may show up in some Handler.
A really bad query, such as a CROSS JOIN, would show a value of N*M, where N and M are the row counts for the two tables.
I used the Handler technique to 'prove' that virtually all published "get me a random row" techniques require a table scan. Then I could experiment with small tables and Handlers to come up with a list of faster random routines.
Another tip when timing... Turn off the Query_cache (or use SELECT SQL_NO_CACHE).
My site is experiencing a really slow loading time. I have suspected that it might be javascript or php that causes longer loading time but i have tested my site in YSlow and its grade is B which i think is not bad.
Now i want to check my database if something is wrong with queries, database indexing that causes my site to load slower.
Is there some tutorials or tricks i might read or try to test database to figure out if there is slow queries? Any tips for database management?
I always fall back to 2 mantras for faster query execution -
Indexes, indexes and indexes.
Try to get rid of JOINS as much as possible.
There are some tried and tested methods to weed out slow queries. You need to turn on slow query log. This logs all those queries which take more than x seconds to execute. x is specified by you in mysql.conf.
Once the slow queries start logging in the log. You can analyse each query using EXPLAIN and appropriately add indexes to speedy the query execution.
I have a thin database abstraction layer on top of PDO (formerly on top of MySQL) that I've baked simple query logging into which I can switch on and off - it allows me to get a report of the queries and how long each one took. Thus rather than having a simple cut-off - something is either a long query or it's not - I get to see all my query times.
MySQL's slow query log is good, but its one-second resolution is not enough for my needs. To me, a lot of the time, a query that takes 200 milliseconds is an indication of something wrong.
I'm showing my age here. After a quick check of the MySQL manual, it turns out that MySQL's long_query_time can be specified down to microsecond resolution since MySQL 5.1.27! Nonetheless, my method is still handy.
Good tutorial for sql performance - focused on MySql. For joins, if you can't get rid of them - use the right one: LEFT JOIN, RIGHT, INNER, OUTER
I'm wondering if anyone has a suggestion for my situation:
I have a process that runs many tens of thousands of queries. The whole process takes between 5 and 10 minutes. I want to know which queries are running slower than the rest, but I know that none of them are running for more than say, 5 seconds (with this many queries, that would be very noticeable in my logs). How should I find out which ones are taking the most time, and are the ones that, if optimized, would provide the best results?
MORE DETAILS:
My queries run single-threaded and synchronous, and I'd say 70% SELECT and 30% INSERT/UPDATE. I'd have to get some heads together and determine if the work can be split up into different units that can be run simultaneously - I'm not sure...
All the queries are either simple INSERT statements, single-property UPDATE statements, or SELECT statements on either a primary or foreign key or a two-field ANDed restriction.
DESCRIPTION OF THE ISSUE:
What I'm doing is basically copying a complex directed graph structure, in its entirety. Nodes are database entries, and adjacencies represent essentially foreign keys, but not strictly-speaking (they could be a two-field combination, where the first says what table the second is the id for).
Take a look at MySQL's slow query log. You can configure the threshold of what is regarded as "slow".
Basically, you track time from query start to query end, e.g.:
function getmicrotime() {
list($usec, $sec) = explode(" ",microtime());
return ((float)$usec + (float)$sec);
}
$query_start = getmicrotime();
$query = 'SELECT ...';
mysql_query($query, $connect);
$query_end = getmicrotime();
if ($query_end - $query_start > 2) {
// add query to log
your_slow_queries_logger($query);
}
I'd sugest to make some kind of a wrapper for mysql_query function that would take care of logging slow queries.
I, personally, log my slow queries using syslog
If you serious in optimizing MySQL, have a read at this book
High Performance MySQL: Optimization, Backups, Replication, and More
Some of the contents may be outdated for the newest MySQL versions, but should be sufficient as it covers MySQL 5.1. It also has an extensive chapter on benchmarking along with techniques and methods and guiding you to create a benchmark plan that suits your needs.
It also has chapters on Index and Query optimizations which would be very helpful to you if you need to optimize the slow queries identified with the benchmarks.
The other day I found the FOUND_ROWS() (here) function in MySQL and it's corresponding SQL_CALC_FOUND_ROWS option. The later looks especially useful (instead of running a second query to get the row count).
I'm wondering what speed impact there is by adding SQL_CALC_FOUND_ROWS to a query?
I'm guessing it will be much faster than runnning a second query to count the rows, but will it be a lot different. Also, I have found limiting a query to make it much faster (for example when you get the first 10 rows of 1000). Will adding SQL_CALC_FOUND_ROWS to a query with a small limit cause the query to run much slower?
I know I can test this, but I'm wondering about general practices here.
When I was at the MySQL Conference in 2008, part of one session was dedicated to exactly this - benchmarks between SQL_CALC_FOUND_ROWS and doing a separate SELECT.
I believe the result was that there was no benefit to SQL_CALC_FOUND_ROWS - it wasn't faster, in fact it may have been slower. There was also a 3rd way.
Additionally, you don't always need this information, so I would go the extra query route.
I'll try to find the slides...
Edit: Hrm, google tells me that I actually liveblogged from that session: http://beerpla.net/2008/04/16/mysql-conference-liveblogging-mysql-performance-under-a-microscope-the-tobias-and-jay-show-wednesday-200pm/. Google wins when memory fails.
To calculate SQL_CALC_FOUND_ROWS the query will be execute as if no LIMIT was set, but the result set sent to the client will obey the LIMIT.
Update: for COUNT(*) operations which would be using only the index, SQL_CALC_FOUND_ROWS is slower (reference).
I assume it would be slightly faster for queries that you need the number of rows know, but would incur and overhead for queries that you don't need to know.
The best advice I could give is to try it out on your development server and benchmark the difference. Every setup is different.
I would advise to use as few proprietary SQL extensions as possible when developing an application (or actually not using SQL queries at all). Doing a separate query is portable, and actually I don't think MySql could do better at getting the actual information than re-querying. Btw. as the page mentions the command has some drawbacks too when used in replicated environments.