Logging inefficient queries and query frequency in MySQL
We have a pour performing mysql DB. And I'm interested in logging queries and their frequency to assess which ones my time is best spent on. We have slow query logging but these queries aren't the ones hammering the DB. I've been told statspack for Oracle does exactly that with some sort of weighting on the queries.
Any ideas or suggestions would be appreciated.
The MySQL general query log will show all statements being executed. On a busy server, this will grow quickly, and will slow performance, so you likely want to enable it briefly, and then disable it again. That will give you every statement executed, as well as client connects and disconnects.
http://dev.mysql.com/doc/refman/5.5/en/query-log.html
The slow query log can also be a useful tool.
There are some third party tools that can assist with analyzing the output from the general log, the slow query log, and the output from continually running a SHOW FULL PROCESSLIST. One example is MONyog.
Related
If the database is overloaded you'll get an increased number of queries running. Occasional spikes are OK for very short period of time. Too many active threads indicate that:
MySQL is taking too much time to process you requests.
You are continuously retrieving/updating large datasets.
Make sure that queries are tuned to use indexes. Execute SHOW FULL PROCESSLIST of find queries that are getting locked continuously. Try
isolating long running queries by enabling the slow query log.
any help please
I wish to analyze poor MYSQL Query Performance which was happened in different time stamp.
In Oracle, I used to use sqlt (SQLTXPLAIN) report when it's required to analyze poor query performance before and after major version upgrade or executes a query with drastic execution variance in same environment with similar server load. It could provide why SQL is not performing as expected and crucial information (Performance history for the SQL statement, DB parameters, State of CBO stats, Changes on histograms, indexes compare including state, execution plan analysis and all) to actually find root causes before trying to fix performance issue.
Sample Oracle SQLT Report for the reference - http://carlossierradotnet1.files.wordpress.com/2013/04/sqlt_s60032_main.pdf
Can I produce similar report for the MYSQL query which executes with best execution time and worst performance occasionally in the same server environment?
Update:
Capturing long running query and analyze execution plan for a query doesn't work to resolve the problem that I have mentioned in the question. Query doesn't have any problem as it runs without any issue(less than a sec), rarely it behaves with worst performance(hangs sometimes). I am pretty sure, we could pin point the root cause if we get details of Performance history for the SQL statement, State of CBO stats, Changes on histograms, indexes compare including state, execution plan analysis and all for the same query in best and worst timestamp.
I don't have particular query here as it can happen with any complex query if they are running in a big batch, I'd like to provide scenario which might be helpful to understand the problem:
QUERY BATCH#1 - Complex Query#1; Query#2;....Query#10....Complex Query#N;
Situation#1:
All queries are perfectly tuned and runs smoothly when they are executed one by one in a MYSQL prompt.
Situation#2:
QUERY BATCH#1 executing smoothly every night smoothly on 01/05, 02/05, 04/05, 06/05/2014 but Query#2 was taking unacceptable execution time only on 03/05 and Query#10 was taking unacceptable execution time on 05/05/2014. Please be informed that issue was happened with different queries and resource usage (cpu, ram, n/w i/o, storage i/o, etc) and number of db connections are same every day.
Is there any way to check why optimizer took unacceptable time for Query#2 and Query#10 only on 03/05 and 05/05 respectively? Now a days QUERY BATCH#1 runs smoothly every day without applying any changes in the database/system/application.
Does mysql server keeps records of queries executed on it, if it does so , is it possible to retrieve those queries.
Thanks.
You can use the General Query Log, which logs all queries in plain text.
We have this enabled on our development environment only. It shouldn't be a problem to turn this on temporarily on a production server during off-peak times.
If you're looking for slow queries, you may as well use the Slow Query Log.
If you want to keep record of all queries that are executed, you can enable either the General Query Log or the Slow Query Log and set the threshold to 0 seconds (any query that takes more than 0 seconds will be logged).
Another option is the Binary log. However, binary log does not keep record of queries like SELECT or SHOW that do not modify data.
Note that these logs get pretty big pretty fast in a server with traffic. And that both might contain passwords so they have to be protected from unauthorized eyes.
You can use MySQL Proxy which stands between client app and RDBMS itself.
http://forge.mysql.com/wiki/MySQL_Proxy
You can see online queries and also it allows you to rewrite queries based on rules.
There's another option - use a profiler.
For instance: http://www.jetprofiler.com/
i have mysql that is used on production server for php webshop application.
sometimes it works very slow. so, i will change indexes for several tables.
but before that, i have to make some kind of "snapshot" of current performances (several times per day). after that, i will change indexes, and create new "performance snapshot". then i will made some more changes in database, and made another "performance snapshot".
how can i make that "performance snapshot"? is it possible to use some kind of tool, or to ckeck some logs, or...?
if you can help me how to do that.
thank you in advance!
If you want to buy a commercial product, there is the MySQL Query Analyzer
Otherwise, you could use the SQL Profiler which is already included with MySQL.
The SQL Profiler is built into the database server and can be dynamically enabled/disabled via the MySQL client utility. To begin profiling one or more SQL queries, simply issue the following command:
mysql> set profiling=1;
Thereafter, you will see the duration of each of your queries as you run them.
Slow query log and queries not using indexes
query cache hit rate
innodb monitor
and of course your database hard-disk I/O, memory usage ...
Can anyone suggest a good MYSQL optimization tool which helps in finding the bottlenecks in a long query and hence help in optimization?? I am looking for a query profiler.
thanks...
Well, You mean Query Optimization? I guess EXPLAIN <query> is excellent in giving hits as to where the bottlenecks are. After which redefine you indexes & ...
UPDATE1: You could check out - MySQL optimization tools
UPDATE2: After digging up in my code, I see that I used to do 2 things for query optimization.
Turn on Slow Query Log - MySQL can record expensive SQL queries in the slow query log. You can define your expectations in seconds using parameter long_query_time.
mysqldumpslow command - After logging is turned on you can analyze the log contents using mysqldumpslow command. mysqldumpslow /path/to/your/mysql-slow-queries.log -t 10. This will show you top 10 performance killers. For each statement in the output you can see the number of identical calls, execution time in seconds, rows affected and the statement itself.