If the database is overloaded you'll get an increased number of queries running. Occasional spikes are OK for very short period of time. Too many active threads indicate that:
MySQL is taking too much time to process you requests.
You are continuously retrieving/updating large datasets.
Make sure that queries are tuned to use indexes. Execute SHOW FULL PROCESSLIST of find queries that are getting locked continuously. Try
isolating long running queries by enabling the slow query log.
any help please
Related
I am loading data into Mysql DB(8.0.16) through pentaho(8.3) jobs, from the last 2months process was going fine(jobs usually complete in 1hr), Since last night the process is going very slow(executing for more than 12hrs), I have checked the processlist the queries are taking for ever for executing(even the count on small volume tables). am unable to figureout the issue.
any suggestions to findout the bottlenecks.
I wish to analyze poor MYSQL Query Performance which was happened in different time stamp.
In Oracle, I used to use sqlt (SQLTXPLAIN) report when it's required to analyze poor query performance before and after major version upgrade or executes a query with drastic execution variance in same environment with similar server load. It could provide why SQL is not performing as expected and crucial information (Performance history for the SQL statement, DB parameters, State of CBO stats, Changes on histograms, indexes compare including state, execution plan analysis and all) to actually find root causes before trying to fix performance issue.
Sample Oracle SQLT Report for the reference - http://carlossierradotnet1.files.wordpress.com/2013/04/sqlt_s60032_main.pdf
Can I produce similar report for the MYSQL query which executes with best execution time and worst performance occasionally in the same server environment?
Update:
Capturing long running query and analyze execution plan for a query doesn't work to resolve the problem that I have mentioned in the question. Query doesn't have any problem as it runs without any issue(less than a sec), rarely it behaves with worst performance(hangs sometimes). I am pretty sure, we could pin point the root cause if we get details of Performance history for the SQL statement, State of CBO stats, Changes on histograms, indexes compare including state, execution plan analysis and all for the same query in best and worst timestamp.
I don't have particular query here as it can happen with any complex query if they are running in a big batch, I'd like to provide scenario which might be helpful to understand the problem:
QUERY BATCH#1 - Complex Query#1; Query#2;....Query#10....Complex Query#N;
Situation#1:
All queries are perfectly tuned and runs smoothly when they are executed one by one in a MYSQL prompt.
Situation#2:
QUERY BATCH#1 executing smoothly every night smoothly on 01/05, 02/05, 04/05, 06/05/2014 but Query#2 was taking unacceptable execution time only on 03/05 and Query#10 was taking unacceptable execution time on 05/05/2014. Please be informed that issue was happened with different queries and resource usage (cpu, ram, n/w i/o, storage i/o, etc) and number of db connections are same every day.
Is there any way to check why optimizer took unacceptable time for Query#2 and Query#10 only on 03/05 and 05/05 respectively? Now a days QUERY BATCH#1 runs smoothly every day without applying any changes in the database/system/application.
Logging inefficient queries and query frequency in MySQL
We have a pour performing mysql DB. And I'm interested in logging queries and their frequency to assess which ones my time is best spent on. We have slow query logging but these queries aren't the ones hammering the DB. I've been told statspack for Oracle does exactly that with some sort of weighting on the queries.
Any ideas or suggestions would be appreciated.
The MySQL general query log will show all statements being executed. On a busy server, this will grow quickly, and will slow performance, so you likely want to enable it briefly, and then disable it again. That will give you every statement executed, as well as client connects and disconnects.
http://dev.mysql.com/doc/refman/5.5/en/query-log.html
The slow query log can also be a useful tool.
There are some third party tools that can assist with analyzing the output from the general log, the slow query log, and the output from continually running a SHOW FULL PROCESSLIST. One example is MONyog.
Is there any way by which i can calculate the time that would take for a set of Mysql Queries to execute, may be based on the number of records or something, before executing the query?
No. There are many factors that go into how long a query takes to execute, including:
Hardware specs
Network configuration, bandwidth, and load
MySQL Server configuration and server load
MySQL index usage and execution plan
Size of data
Size of index
Size of result set
The best way to determine how long a query will take is to run it. You should disable the query cache using SQL_NO_CACHE so that the query cache does not skew the results.
Use SHOW PROFILE to see where MySQL is spending its time.
Use EXPLAIN to see the execution plan so that you can optimize the execution plan.
Using EXPLAIN, you can get a feel for whether the query is efficient or not, but you will still have to run it to see how long it takes in your environment.
Lately we are seeing some queries in mysql(master) logs but no idea why they are shown there:
Queries are select/update table where id = <some integer>.
There is index on id
table size is below 100 000
Rows scanned are in hundreds (sometimes < 100)
Server is running on extremely good hardware
there are no joins involved
We do not see any heavy activity running on database at that time
tables are innodb
the same queries generally don’t even take 50ms, but sometimes all the execution of these queries take about 4-8 seconds
One observation was all the similar "non-slow-but-weirdly-taking-high-time" queries take almost the same amount of time for some duration . I.e. queries like stated in the top will all take about 4.35 seconds with variation of 0.05 seconds.
Does the network latency/packets-drop affect mysql query timing?
show processlist;
show global status like '%onnect%';
show global status like '%open%';
Is anything backed up? Is it waiting in a queue? waiting for file handles? What are your max_connections, open-files-limit, thread_concurrency ?
One side question: does the network latency/packets-drop affect mysql query timing?
Yes, the timeout must occur before the query is resent by the client
Do you see these problems locally or over the network? If the latter, then obviously packet drops can affect your performance if you are measuring from the client.
Is it running in a virtual machine that can affect the performance?
Disk problems?
How is serialization set up? Can it be a contention problem by many processes accessing the same row?
You may want to enable the query/slow query logs to see if there is any sort of pattern that causes this.
Mysql slow log is not representative source to learn about your slow queries. If something makes server work slow all queries usually go to slow log.
E.g. if you have some slow blocking select on MyISAM a lot of PK updates will go to slow log.
You need to search for other slow queries or server problems. What about load average on this particular machine? Isn't mysql displaced into swap memory? Other applications? Queries per second?