How can I view a list/history of all queries in a database/cluster in MemSQL or MySQL including completed and on-going running queries? I would like to see the status of any query such as if it completed or if it is running or if it has been aborted. Is there a query that I can run to view this? Thank you.
MemSQL has information_schema views to get info about running and completed/failed queries. Take a look at https://docs.memsql.com/concepts/v6.0/workload-profiling/.
For example the following query will show all the queries that have been run in the last 10 minutes:
select query_text,success_count,failure_count from information_schema.mv_activities_cumulative join information_schema.mv_queries using (activity_name) where last_finished_timestamp > now() - interval '10' minute;
You can also use these views to drill deeper to understand the resource usage of queries.
According to official memsql documentation on Management View Reference they say "mv_activities determines recent resource use by computing the change in mv_activities_cumulative over an interval of time. This interval is controlled by the value of the activities_delta_sleep_s session variable."
So something like this if I read the documentation correctly should upon execution set the variable activities_delta_sleep_s to 30 seconds and then when the query is run you will see all database activities in decreasing cpu time. My problem with it is that I don't seem to see all of the activities I perform and a lot of the query text is blank. The activities_delta_sleep_s according to memsql forums indicates that the variable simple makes all management view calls sleep for this variable amount of time before returning results. I think the idea there is that this would allow the for the aggregation of activities in nodes and network traffic.
#set the lookback period desired
set activities_delta_sleep_s = 30;
#This query is supposed to show resource costs of server activities, but not sure how to ID specific queries.
SELECT * #look at QUERY_TEXT
from information_schema.MV_ACTIVITIES a
LEFT JOIN information_schema.mv_queries q ON a.ACTIVITY_NAME = q.ACTIVITY_NAME
order by 4 DESC;
Related
We are using Spring (Eclipselink) on MariaDB. Our SQL via ORM result in a long lasting DB query. Therefore I need to refine it into a nativequery - which is no big deal by itself. Nevertheless the Resultset is limited by a LIMIT and I need a total counter for all found records. For querying the total counter I found for MariaSQL the following solution.
My question:
Is it save to query the two SQL command separately or should I send them once with a UNION combined?
The question arises due to the fact that between my query and the SELECT FOUND_ROWS() the another query might interfere (with a request from the same Microservice) and dilute the result.
If both queries are done in the same transaction, the MVCC of INNODB should guarantee, that the results will not be influenced by other transactions.
see: https://dev.mysql.com/doc/refman/8.0/en/innodb-multi-versioning.html
I am using MySQL workbench 6.3 CE. I want to take the snapshots of MySQL status variables.
I want to store the values of status variables after every 1 second during the execution of query.
I can simply show the variables using 'show global status'. But I want to execute it automatically after every 1 second.
You can run a procedure and a query at the same time by having two separate connections. Workbench is a handy tool, but you should learn to use the mysql commandline tool, too.
The query is rather simple. INDEX(l_shipdate) is likely to be the best for it.
The real way to speed up the query (assuming that is your ultimate goal) is to build and maintain a "Summary table" of daily or monthly subtotals. Then sum the sums and sum the counts. Avg is (SUM(sums)/SUM(counts)).
More discussion: http://mysql.rjweb.org/doc.php/summarytables
Be cautious about running (via EVENT or cron) any code that might take longer than the interval time. If it gets behind, it is likely to cascade and bring the server down, or at least slow things down severely. For that reason, I much prefer the WHILE loop.
I am currently trying to run a JOIN between two tables in a local MySQL database and it's not working. Below is the query, I am even limiting the query to 10 rows just to run a test. After running this query for 15-20 minutes, it tells me "Error Code" 2013. Lost connection to MySQL server during query". My computer is not going to sleep, and I'm not doing anything to interrupt the connection.
SELECT rd_allid.CreateDate, rd_allid.SrceId, adobe.Date, adobe.Id
FROM rd_allid JOIN adobe
ON rd_allid.SrceId = adobe.Id
LIMIT 10
The rd_allid table has 17 million rows of data and the adobe table has 10 million. I know this is a lot, but I have a strong computer. My processor is an i7 6700 3.4GHz and I have 32GB of ram. I'm also running this on a solid state drive.
Any ideas why I cannot run this query?
"Why I cannot run this query?"
There's not enough information to determine definitively what is happening. We can only make guesses and speculations. And offer some suggestions.
I suspect MySQL is attempting to materialize the entire resultset before the LIMIT 10 clause is applied. For this query, there's no optimization for the LIMIT clause.
And we might guess that there is not a suitable index for the JOIN operation, which is causing MySQL to perform a nested loops join.
We also suspect that MySQL is encountering some resource limitation which is causing the session to be terminated. Possibly filling up all space in /tmp (that usually throws an error, something like "invalid/corrupted myisam table '#tmpNNN'", something of that ilk. Or it could be some other resource constraint. Without doing an analysis, we're just guessing.
It's possible MySQL wrote something to the error log (hostname.err). I'd check there.
But whatever condition MySQL is running into (the answer to the question "Why I cannot run this query")
I'm seriously questioning the purpose of the query. Why is that query being run? Why is returning that particular resultset important?
There are several possible queries we could execute. Some of those will run a long time, and some will be much more performant.
One of the best ways to investigate query performance is to use MySQL EXPLAIN. That will show us the query execution plan, revealing the operations that MySQL will perform, and in what order, and indexes will be used.
We can make some suggestions as to some possible indexes to add, based on the query shown e.g. on adobe (id, date).
And we can make some suggestions about modifications to the query (e.g. adding a WHERE clause, using a LEFT JOIN, incorporate inline views, etc. But we don't have enough of a specification to recommend a suitable alternative.
You can try something like:
SELECT rd_allidT.CreateDate, rd_allidT.SrceId, adobe.Date, adobe.Id
FROM
(SELECT CreateDate, SrceId FROM rd_allid ORDER BY SrceId LIMIT 1000) rd_allidT
INNER JOIN
(SELECT Id FROM adobe ORDER BY Id LIMIT 1000) adobeT ON adobeT.id = rd_allidT.SrceId;
This may help you get a faster response times.
Also if you are not interested in all the relation you can also put some WHERE clauses that will be executed before the INNER JOIN making the query faster also.
As per my research I thought of using the mysqldumpslow utility to parse the log and extract the results, but not able to figure out how to use it. I want to get the count of number of queries logged in the slow query log for an interval of 10 minutes, so that the values can be compared for analysis.
Thanks
You could use logrotate to create a new slow.log every 10 minutes and analyze them one after another. Implying you are using Linux. Be aware that your example shows that your mysql instance is configured to "log-queries-not-using-indexes" hence you will also get those SELECT's that dont use an index in your log file too.
Update :
Since i still dont know what OS you are using, a more general aproach to your problem would be redirecting the slow log into mysql itself following the mysql docs and get all records from the slow log table like :
SELECT COUNT(*) FROM slow_log;
Which gives you the total amount of Querys logged. Follwed by a :
TRUNCATE TABLE slow_log;
Having a script in place doing this every 10 minutes would output the desired information.
Recently I was pulled into the boss-man's office and told that one of my queries was slowing down the system. I then was told that it was because my WHERE clause began with 1 = 1. In my script I was just appending each of the search terms to the query so I added the 1 = 1 so that I could just append AND before each search term. I was told that this is causing the query to do a full table scan before proceeding to narrow the results down.
I decided to test this. We have a user table with around 14,000 records. The queries were ran five times each using both phpmyadmin and PuTTY. In phpmyadmin I limited the queries to 500 but in PuTTY there was no limit. I tried a few different basic queries and tried clocking the times on them. I found that the 1 = 1 seemed to cause the query to be faster than just a query with no WHERE clause at all. This is on a live database but it seemed the results were fairly consistent.
I was hoping to post on here and see if someone could either break down the results for me or explain to me the logic for either side of this.
Well, your boss-man and his information source are both idiots. Adding 1=1 to a query does not cause a full table scan. The only thing it does is make query parsing take a miniscule amount longer. Any decent query plan generator (including the mysql one) will realize this condition is a NOP and drop it.
I tried this on my own database (solar panel historical data), nothing interesting out of the noise.
mysql> select sum(KWHTODAY) from Samples where Timestamp >= '2010-01-01';
seconds: 5.73, 5.54, 5.65, 5.95, 5.49
mysql> select sum(KWHTODAY) from Samples where Timestamp >= '2010-01-01' and 1=1;
seconds: 6.01, 5.74, 5.83, 5.51, 5.83
Note I used ajreal's query cache disabling.
First at all, did you set session query_cache_type=off; during both testing?
Secondly, both your testing queries on PHPmyadmin and Putty (mysql client) are so different, how to verify?
You should apply same query on both site.
Also, you can not assume PHPmyadmin is query cache off. The time display on the phpmyadmin is including PHP processing, which you should avoid as well.
Therefore, you should just do the testing on mysql client instead.
This isn't a really accurate way to determine what's going on inside MySQL. Things like caching and network variations could skew your results.
You should look into using "explain" to find out what query plan MySQL is using for your queries with and without your 1=1. A DBA will be more interested in those results. Also, if your 1=1 is causing a full table scan, you will know for sure.
The explain syntax is here: http://dev.mysql.com/doc/refman/5.0/en/explain.html
How to interpret the results are here: http://dev.mysql.com/doc/refman/5.0/en/explain-output.html