I have around 3 websites residing in a server which is being shared with other teams. I have been notified that there is a huge increase in CPU usage and we need to lower it down. I doubt my websites are causing this.
I have been using SHOW FULL PROCESSLIST in MySQL and 90% of the time shows queries from other databases. But I think executing multiple SHOW FULL PROCESSLIST commands is not enough proof. How can I verify that my databases aren't CPU hogs?
If you suspect that its a MySQL query, try inspecting the MySQL Slow Query Log. The log will help you identify queries that take long time to execute. You can then copy-paste the query text into a MySQL query session, execute it and observe the CPU usage.
You may also want to install different MySQL release, that allows to track exact user statistics and some other things:
http://www.mysqlperformanceblog.com/2008/07/16/mysql-releases-with-percona-patches/
Related
Scenario - you have hundreds of reports running on a slave machine. These reports are either scheduled by MySQL's event scheduler or are called via a Python/R or Shell script. Apart from that, there are fifty odd users who are connecting to MySQL slave running random queries. These people don't really know how to write good queries and that's fair. They are not supposed to. So, every now and then (read every day), you see some queries which are stuck because of read/write locks. How do you fix that.
What you do is that you don't kill whatever is being written. Instead, you kill all the read queries. Now, that is also tricky because, if you kill all the read queries, you will also let go off OUTFILE queries, which are actually write queries (they just don't write to MySQL, but write to disk).
Why killing is necessary (I'm only speaking for MySQL, do not take this out of context)
I have got two words for you - Slave lag. We don't want that to happen, because if that happens, all users, reports, consumers suffer.
I have written the following to kill processes in MySQL based on three questions
how long has the query been running?
who is running the query?
do you want to kill write/modify queries too?
What I have intentionally not done yet is that I have not maintained a history of the processes that have been killed. One should do that so as to analyse and find out who is running all the bad queries. But there are other ways to find that out.
I have create a procedure for this. Haven't spend much time on this. So, please suggest if this is a good way to do it or not.
GitHub Gist
Switch to MariaDB. Versions 10.0 and 10.1 implement several limits and timeouts: https://mariadb.com/kb/en/library/query-limits-and-timeouts/
Then write an API between what the users write and actually hitting the database. In this layer, add the appropriate limitations.
can someone help me with the following sentence from the MySQL Homepage.
In addition, profiling is per process and not per thread. This means that activity on threads within the server other than your own may affect the timing information that you see.
https://dev.mysql.com/doc/refman/5.7/en/show-profile.html
I want to get the duration of a query. Simultaneously 9 other querys from other clients try to get information from the database.
Do I only get the information for my query or do i also see the information of the other querys?
Thanks for your help.
This disclaimer ( profiling is per process and not per thread ) means the information about your query's reported performance will be affected by the other workload on the MySQL server. The server tries to sort out your query's resource usage from the other workload, but due to various limitations it cannot do that perfectly accurately.
It warns you that profiling on a busy MySQL server will not yield completely reproducible results. This isn't generally surprising. As long as you take your profile measurements as appropximations, you probably won't have trouble.
I'm using MONyog to track all queries from user, but when config connect to mysql server. I can not see any query after click start? How can I see queries in MONyog?
Reference MONyog
I guess you are referring to MONyog's Real-Time feature? In Real-Time tab MONyog executes SHOW FULL PROCESSLIST at every 1 second to get queries being executed on the MySQL server. Hence, short-lived queries (queries which takes less than 1 second) may or may not be recorded.
You can enable Performance Schema based sniffer which will record all the queries(even short lived queries). Performance Schema based sniffer makes use of the performance_schema database of the MySQL server(which is available in v5.6+).
I have two mysql databases that are identical. One is running locally and one is running on a production server. The one that runs locally is consistently 3x faster on all queries, even though the production server is larger more memory and better CPU than my local machine.
I did SHOW PROFILES and it doesn't show anything definitive, most of the items are the same, though sometimes the local machine is ~10-20% better. How would I determine where this difference is coming from? Perhaps it has to do with networking performance, but I'm not sure how I would profile that with mysql queries.
What would you suggest as to how to figure this out and improve the production db performance?
Profile the query to see where the time is being spent. A long "sending data" stage will show if it's network latency.
Always use SQL_NO_CACHE when measuring query time.
my site started dragging lately, the queries taking exceptionally longer than I would expect with properly tuned indexes. I just restarted the mysql server after 31 days uptime and every query is now substantially faster and the whole site renders 3-4 times faster.
Would there be anything that jumps out at you as to why this may have been? Improper settings on my.cnf perhaps? Any ideas as to what I can start looking at to try and pinpoint why?
thanks
updated note: I have a 16GB dedicated db box and mysql runs at about 71% of memory after a week or so.
Try to execute show processlist;, maybe there are some long lasting threads that were not killed for some reason.
Similarly execute SHOW SESSION STATUS LIKE 'Created%'; to check if mysql hasn't created to many temporary tables.
Server restart automatically closes all open temp tables and kills threads, so the application might run quicker.
Do you have temporary table(s) that might not be getting cleared/collected?
I would suggest using MySQL Enterprise for analysis purposes. It comes with a 30 day trial. We just used it. We got alerts such as :
CRITICAL Alert - Table Scans Excessive
The target server does not appear to be using indexes efficiently.
CRITICAL Alert - Connection Usage Excessive
CRITICAL Alert - CPU Usage Excessive
WARNING Alert - MyISAM Key Cache Has Sub-Optimal Hit Rate
Just something to explore!