I have been trying to check slow query in process-list but its showing null & time is huge. Is any other way to check slow queries without enable the logs.
| ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO | TIME_MS | ROWS_SENT | ROWS_EXAMINED | ROWS_READ |
| 1149040 | test | 127.0.0.1:58759 | NULL | Sleep | 206 | | NULL | 206054 | 0 | 0 | 6447 |
Thanks
Use:
select * from information_schema.processlist where command!='sleep' and time>10;
This will return queries take more than 10 seconds and is not in sleep mode.
Null value indicates no query is being executed.
Related
Recently in my application which uses MariaDB 10.6, I am facing some weird issues where the same query took more than the expected time and consumes more IO at random times.
Enabled slow query to trace the same where we see a query stuck more than 9min and consumes more IO.
# Time: 230119 15:25:02
# User#Host: user[user] # [192.*.*.*]
# Thread_id: 125616 Schema: DB QC_hit: No
# Query_time: 567.099806 Lock_time: 0.000500 Rows_sent: 48 Rows_examined: 10859204
# Rows_affected: 0 Bytes_sent: 0
SET timestamp=1674152702;
select column1,column2....columnN where column1=v1 and column2=v2 and column3=v3 and column4=v4 and column5=v5;
On seeing the DB processlist, more no of query are in state "Waiting for table metadata lock" and end up in bigger issues.
| 106804 | userx | IP | DB | Query | 4239 | Sending data | Q1 | 0.000 |
| 106838 | userx | IP | DB | Query | 1980 | Waiting for table metadata lock | Q2 | 0.000 |
| 107066 | userx | IP | DB | Sleep | 0 | | NULL | 0.000 |
| 107196 | userx | IP | DB | Sleep | 1 | | NULL | 0.000 |
| 107223 | userx | IP | DB | Query | 4363 | Sending data | Q3 | 0.000 |
| 107277 | userx | IP | DB | Query | 3221 | Sending data | Q4 | 0.000 |
| 107299 | userx | IP | DB | Sleep | 26 | | NULL | 0.000 |
| 107324 | userx | IP | DB | Sleep | 54 | | NULL | 0.000 |
| 107355 | userx | IP | DB | Sleep | 0 | | NULL | 0.000 |
| 107357 | userx | IP | DB | Sleep | 1 | | NULL | 0.000 |
| 107417 | userx | IP | DB | Query | 1969 | Waiting for table metadata lock | | 0.000 |
| 107462 | userx | IP | DB | Sleep | 55 | | NULL | 0.000 |
| 107489 | userx | IP | DB | Query | 1979 | Waiting for table metadata lock | Q5 | 0.000 |
| 107492 | userx | IP | DB | Sleep | 25 | | NULL | 0.000 |
| 107519 | userx | IP | DB | Query | 1981 | Waiting for table metadata lock | Q6 | 0.000 |
Currently, the manual killing of the suspected query using KILL query cmd unblocks the other query to complete, and via MariaDB property, we can use max_statement_time to terminate the long-running query.
But Is there a way to check what was killed by the max_statement_time? Unable to find any traces in error.log.
The actual query should return around 1765 records while the slow query reports row_sent as 48.
Is it a problem with scanning the table or record fetched got stuck after some time?
Or Am I misinterpreting the slow query output Row_send record count
127.0.0.1:3307> select column1,column2....columnN where column1=v1 and column2=v2 and column3=v3 and column4=v4 and column5=v5;
+----------+
| count(*) |
+----------+
| 1756 |
+----------+
1 row in set (0.006 sec)
---EDITED---
Missed adding column 5 added now in the query.
The table is indexed and let me explain the statement.
127.0.0.1:3307> explain extended select..... from Tablename where column1=v1 and column2=v2 and column3=v3 and column4=v4 and column5=v5;
+------+-------------+-------+------+---------------+---------+---------+-------------------------------+------+----------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+------+-------------+-------+------+---------------+---------+---------+-------------------------------+------+----------+-------+
| 1 | SIMPLE | s | ref | PRIMARY | PRIMARY | 7 | const,const,const,const,const | 73 | 100.00 | |
+------+-------------+-------+------+---------------+---------+---------+-------------------------------+------+----------+-------+
1 row in set, 1 warning (0.007 sec)
Assuming that v1..v4 are literal constants (numeric, string, date, etc), and assuming you really mean = in all 4 cases, then
INDEX(col1, col2, col3, col4)
(in any order) is optimal for that one SELECT. This will also minimize I/O for that query.
There are many reasons for "high" I/O. I'll cover a few of them.
Data is bigger than the cache, the size of which is controlled by innodb_buffer_pool_size. How big is the table?
That setting should be configured to be about 70% of available RAM. How much RAM do you have?
If any items in the query are TEXT or BLOB, there may be extra I/O.
Currently you do not have a good index, and it had to check 10859204 rows to find the 48 you needed. With the index, above, only 49 rows need to be fetched.
"Waiting for table metadata lock" -- This implies that you are doing some serious blocking or LOCKing or ALTERing. What other queries are going on?
Since the buffer_pool is shared among all MySQL processes, some other query may be at fault.
Adding the index should eliminate the 48 vs 1765 puzzle and any need for max-query-time.
You should use ENGINE=InnoDB; MyISAM has more locking/blocking issues.
If you provide more specifics, I may have more tips.
(after your edit)
The "Rows" in EXPLAIN is an estimate; it can easily be off by a factor of 2 either way.
Rows_sent in the slowlog is exact.
I'm trying to add a column to a specific table. Every time I run the migration up or down a SLEEP command pops up and blocks everything. Much like many other people who have run into this problem, I kill the blocking process and everything works as expected.
Ex:MySql alter table hangs
I've tried running the migration against a different table and have no issues. Seems to be something specific with this particular table.
Where or what should I be looking for? Why would this issue occur so consistently?
Thanks.
mysql> show processlist;
+-----+------+-----------------+--------+---------+------+---------------------------------+----------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-----+------+-----------------+--------+---------+------+---------------------------------+----------------------------------------------+
| 351 | root | localhost:54691 | database | Sleep | 25 | | NULL |
| 352 | root | localhost:54692 | NULL | Sleep | 54 | | NULL |
| 377 | root | localhost | database | Query | 0 | starting | show processlist |
| 381 | root | localhost:54858 | database | Query | 5 | Waiting for table metadata lock | ALTER TABLE company ADD COLUMN active |
| 382 | root | localhost:54860 | database | Sleep | 5 | | NULL |
+-----+------+-----------------+--------+---------+------+---------------------------------+----------------------------------------------+
5 rows in set (0.00 sec)
The offending process in this case was 382.
The answers to many other similar posts say to kill the SLEEP process and then "debug". My problem was related to a circular import issue in Python, not related to MySQL.
I am using Server version: 5.6.27-log MySQL Community Server (GPL) and I have a problem with a table.
I tried to update some fields with a GUI software, but when I came back to the command line, the lines I tried to update where not updated.
I tried to see if the table was locked using SHOW OPEN TABLES as stated in various other questions. But my table does not appear to be locked:
+--------------------+-------------------------------------------------+--------+-------------+
| Database | Table | In_use | Name_locked |
+--------------------+-------------------------------------------------+--------+-------------+
| arcdev | SCHEDULED_COMMAND | 0 | 0 |
And as soon as I try to make an update like:
UPDATE SCHEDULED_COMMAND SET field = 1;
The server just keeps loading and nothing happen. I tried on other tables and everything worked just fine.
I also tried some DELETE requests and even a DROP TABLE and nothing work so far...
What am I missing?
Thank you for your precious help!
EDIT: Here is the result of the SHOW PROCESSLIST command while a request is hanging:
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
| 282588 | rdsadmin | localhost:17966 | mysql | Sleep | 2 | | NULL |
| 534575 | arc | XXXXXX:49376 | arcdev | Sleep | 17 | | NULL |
| 534579 | arc | XXXXXX:49443 | arcdev | Query | 0 | init | SHOW PROCESSLIST |
| 534659 | arc | XXXXXX:49836 | arcdev | Query | 14 | updating | DELETE FROM SCHEDULED_COMMAND |
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
I have MySQL 5.6.19 community server running on my Ubuntu 14.04. Lately I have been trying to monitor MySQL server using performance_schema. Previously I used to do this by running SHOW FULL PROCESSLIST query every 5 second using a cronjob.
I configured performance_schema.setup_consumers table as follows.
mysql> use performance_schema;
mysql> select * from setup_consumers;
+--------------------------------+---------+
| NAME | ENABLED |
+--------------------------------+---------+
| events_stages_current | NO |
| events_stages_history | NO |
| events_stages_history_long | NO |
| events_statements_current | YES |
| events_statements_history | NO |
| events_statements_history_long | NO |
| events_waits_current | NO |
| events_waits_history | NO |
| events_waits_history_long | NO |
| global_instrumentation | YES |
| thread_instrumentation | YES |
| statements_digest | YES |
+--------------------------------+---------+
By default the events_statements_history is not enabled. I enabled it as follows.
mysql> update setup_consumers set enabled="YES" where name="events_statements_history";
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * from setup_consumers;
+--------------------------------+---------+
| NAME | ENABLED |
+--------------------------------+---------+
| events_stages_current | NO |
| events_stages_history | NO |
| events_stages_history_long | NO |
| events_statements_current | YES |
| events_statements_history | YES |
| events_statements_history_long | NO |
| events_waits_current | NO |
| events_waits_history | NO |
| events_waits_history_long | NO |
| global_instrumentation | YES |
| thread_instrumentation | YES |
| statements_digest | YES |
+--------------------------------+---------+
Now enabled, the events_statements_history table should now be able to capture queries but somehow it is not. Following is the result.
mysql> select count(sql_text) from events_statements_history;
+-----------------+
| count(sql_text) |
+-----------------+
| 0 |
+-----------------+
I do not understand as to why it is not logging in any queries?
The performance schema records data when three conditions are met:
The running thread is instrumented
The instrument currently executed is enabled
The consumers for the data to record are enabled
Threads
By default, every thread (1) is instrumented.
To check if threads are instrumented:
select THREAD_ID, NAME, PROCESSLIST_ID, PROCESSLIST_USER, PROCESSLIST_HOST, INSTRUMENTED from threads;
To change which threads are instrumented by default, use table setup_actors.
Instruments
By default, instruments for statements (aka, queries), are enabled (2).
To see which queries are not instrumented:
select * from setup_instruments where name like "statement/%" and enabled='NO';
To instrument all queries:
update setup_instruments set enabled="YES" where name like "statement/%";
Consumers
By default, only a few consumers are instrumented (3).
To see which consumers are instrumented, or to change this, use select / update on table setup_consumers.
This part seems fine per the question, so check part (1) and (2).
As a side-note, if there is nothing inside the setup_% tables then you might need to enable the performance schema,
https://dev.mysql.com/doc/refman/5.5/en/performance-schema-quick-start.html
My ~700mb database has been importing for 1hr 27min now, the actual disk activity stopped at around 15min but I just left it running to see if it would finish by itself.
The command I ran:
mysqldump DB1 | pv | mysql DB2
So it's pretty much a straight copy from 1 database to another, with DB2 starting as empty.
I can actually see that the data is already in DB2, but the command refuses to end!
So the question is... Should I let it continue to run? Or can I kill it? :/
Updated:
SHOW PROCESSLIST;
+------+--------------+---------------------+------------+---------+------+--------+------------------------------------------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+--------------+---------------------+------------+---------+------+--------+------------------------------------------------------------------------------------------------------+
| 2762 | user1 | localhost | DB2 | Query | 5298 | Locked | /*!50003 CREATE*/ /*!50020 DEFINER="user2"#"%"*/ /*!50003 FUNCTION "function1" |
| 2763 | user1 | localhost | DB1 | Sleep | 5298 | | NULL |
| 2770 | user2 | localhost | NULL | Query | 3633 | Locked | SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`ROUTINES` WHERE `ROUTINE_SCHEMA`='DB2' AND `ROUTIN |
| 2775 | user2 | localhost | NULL | Query | 381 | Locked | SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`ROUTINES` WHERE `ROUTINE_SCHEMA`='DB2' AND `ROUTIN |
| 2776 | user2 | <ipaddress>:<port> | NULL | Query | 0 | NULL | show processlist |
+------+--------------+---------------------+------------+---------+------+--------+------------------------------------------------------------------------------------------------------+
Some settings I have that are non-standard:
innodb_stats_on_metadata=0
innodb_flush_log_at_trx_commit=2