We only have two Junior DBAs, no Senior DBAs, no one to guide us. We need to review each and every query for almost 30 large projects. Each query took between 5 minutes and two hours depending on the query. As a reviewer, it was a real drag on my time.
My question is, do MySQL can do automatic EXPLAIN on each query, without having to run EXPLAIN by hand and log the result of un-optimized queries? Or maybe by cronjob? If possible, how to do this?
If using mariadb there is log-slow-verbosity=query_plan,explain.
Alternately pt-query-digest has an --explain option to point to the server to run the explain query.
Related
My site is experiencing a really slow loading time. I have suspected that it might be javascript or php that causes longer loading time but i have tested my site in YSlow and its grade is B which i think is not bad.
Now i want to check my database if something is wrong with queries, database indexing that causes my site to load slower.
Is there some tutorials or tricks i might read or try to test database to figure out if there is slow queries? Any tips for database management?
I always fall back to 2 mantras for faster query execution -
Indexes, indexes and indexes.
Try to get rid of JOINS as much as possible.
There are some tried and tested methods to weed out slow queries. You need to turn on slow query log. This logs all those queries which take more than x seconds to execute. x is specified by you in mysql.conf.
Once the slow queries start logging in the log. You can analyse each query using EXPLAIN and appropriately add indexes to speedy the query execution.
I have a thin database abstraction layer on top of PDO (formerly on top of MySQL) that I've baked simple query logging into which I can switch on and off - it allows me to get a report of the queries and how long each one took. Thus rather than having a simple cut-off - something is either a long query or it's not - I get to see all my query times.
MySQL's slow query log is good, but its one-second resolution is not enough for my needs. To me, a lot of the time, a query that takes 200 milliseconds is an indication of something wrong.
I'm showing my age here. After a quick check of the MySQL manual, it turns out that MySQL's long_query_time can be specified down to microsecond resolution since MySQL 5.1.27! Nonetheless, my method is still handy.
Good tutorial for sql performance - focused on MySql. For joins, if you can't get rid of them - use the right one: LEFT JOIN, RIGHT, INNER, OUTER
I am using MySQL Workbench on Windows. How do I find out the time taken to execute a query like
Select * from employers where employerid > 200
Is there a statement I can enter that returns the execution time for this query?
Thanks
MySQL workbench is the way to go for things like this (optimization, benchmarking of queries, etc.). You say you're not familiar with it, so I'd recommend reading a tutorial from the good folks at MySQL
Recently I was pulled into the boss-man's office and told that one of my queries was slowing down the system. I then was told that it was because my WHERE clause began with 1 = 1. In my script I was just appending each of the search terms to the query so I added the 1 = 1 so that I could just append AND before each search term. I was told that this is causing the query to do a full table scan before proceeding to narrow the results down.
I decided to test this. We have a user table with around 14,000 records. The queries were ran five times each using both phpmyadmin and PuTTY. In phpmyadmin I limited the queries to 500 but in PuTTY there was no limit. I tried a few different basic queries and tried clocking the times on them. I found that the 1 = 1 seemed to cause the query to be faster than just a query with no WHERE clause at all. This is on a live database but it seemed the results were fairly consistent.
I was hoping to post on here and see if someone could either break down the results for me or explain to me the logic for either side of this.
Well, your boss-man and his information source are both idiots. Adding 1=1 to a query does not cause a full table scan. The only thing it does is make query parsing take a miniscule amount longer. Any decent query plan generator (including the mysql one) will realize this condition is a NOP and drop it.
I tried this on my own database (solar panel historical data), nothing interesting out of the noise.
mysql> select sum(KWHTODAY) from Samples where Timestamp >= '2010-01-01';
seconds: 5.73, 5.54, 5.65, 5.95, 5.49
mysql> select sum(KWHTODAY) from Samples where Timestamp >= '2010-01-01' and 1=1;
seconds: 6.01, 5.74, 5.83, 5.51, 5.83
Note I used ajreal's query cache disabling.
First at all, did you set session query_cache_type=off; during both testing?
Secondly, both your testing queries on PHPmyadmin and Putty (mysql client) are so different, how to verify?
You should apply same query on both site.
Also, you can not assume PHPmyadmin is query cache off. The time display on the phpmyadmin is including PHP processing, which you should avoid as well.
Therefore, you should just do the testing on mysql client instead.
This isn't a really accurate way to determine what's going on inside MySQL. Things like caching and network variations could skew your results.
You should look into using "explain" to find out what query plan MySQL is using for your queries with and without your 1=1. A DBA will be more interested in those results. Also, if your 1=1 is causing a full table scan, you will know for sure.
The explain syntax is here: http://dev.mysql.com/doc/refman/5.0/en/explain.html
How to interpret the results are here: http://dev.mysql.com/doc/refman/5.0/en/explain-output.html
Can anyone suggest a good MYSQL optimization tool which helps in finding the bottlenecks in a long query and hence help in optimization?? I am looking for a query profiler.
thanks...
Well, You mean Query Optimization? I guess EXPLAIN <query> is excellent in giving hits as to where the bottlenecks are. After which redefine you indexes & ...
UPDATE1: You could check out - MySQL optimization tools
UPDATE2: After digging up in my code, I see that I used to do 2 things for query optimization.
Turn on Slow Query Log - MySQL can record expensive SQL queries in the slow query log. You can define your expectations in seconds using parameter long_query_time.
mysqldumpslow command - After logging is turned on you can analyze the log contents using mysqldumpslow command. mysqldumpslow /path/to/your/mysql-slow-queries.log -t 10. This will show you top 10 performance killers. For each statement in the output you can see the number of identical calls, execution time in seconds, rows affected and the statement itself.
I'm using SQL Server 2008 and just recently started having an intermittent problem while querying a database.
At least once a day I'm having timeouts with many of our applications because of a slow query. There is no particular time this happens; sometimes in the morning, sometimes afternoon. Every time I begin troubleshooting the problem, it fixes itself within minutes.
Normally I use this query:
SELECT Name FROM Demographics WHERE Name IS NOT NULL
and it runs in < 1 second. However, during these "problem times" the query will take around 3 minutes. Once the query goes through, I can run it again and it works just fine (almost instantly).
Also, while the query above is running, I can use this:
SELECT Name FROM Demographics
and it runs perfectly. No delay. The only difference is the WHERE clause. So, where do I begin troubleshooting? What tools should I be using to find the cause?
Thanks in advance.
The first thing to do is to look at the execution plan of the query. To do this, open a query window in Management Studio, and then choose Include Actual Execution Plan in the Query menu. Run your query, then go to the Execution Plan tab and save the plan.
When you see the performance problem, repeat these steps. Then, load both execution plans, and compare them to see what is different. If there are differences, they will probably point you in the right direction to find the problem.
Look to see if you are being blocked by another process during the trouble periods.