I have mysql MyISAM table on which I am doing a simple select id from mytable limit 1;. This just freezes the system.
I tried explain select id from mytable limit 1;. Again it freezes my system. Table demographics: 50k records, 10 mbs size, 2 indexes (primary key autoincrement), 8 columns.
I am clueless why the explain statement failed, as it is supposed to display the query plan, nothing else. Neither the table size is enormous nor the number of records, then why is mysql working so slow? Rather, what am I missing here?
It was due a waiting state on mytable. eggyal gave me the clue to use show processlist. It showed:
+-----+---------+-----------------+----------------+---------+------+---------------------------------+----------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-----+---------+-----------------+----------------+---------+------+---------------------------------+----------------------------------------------------+
| 349 | root | localhost:56612 | mydb | Query | 3582 | Waiting for table metadata lock | ALTER TABLE `mytable` ADD INDEX(`fk_to_02`) |
I planted a kill 349 to terminate that wait chain, and now the explain statement works as expected.
Related
My not-so-big table hangs on an ALTER command. What could it be?
Only 150k rows, 42 fields 142 MByte total. InnoDB storage engine and Server version: 5.5.44-MariaDB MariaDB Server. 1 field, 'slotindex', is primary key: bigint(20) and BTREE type.
The command:
MariaDB [mydb]> ALTER TABLE `runs` CHANGE `p_w_trans_x` `p_w_tran_x` FLOAT NOT NULL;
Stage: 1 of 2 'copy to tmp table' 65.7% of stage done
Stage: 2 of 2 'Enabling keys' 0% of stage done
Will completely hang forever in this stage 2.
The processlist is then as follows:
MariaDB [(none)]> show full processlist;
+--------+------+-----------------+-----------+---------+-------+---------------------------------+---------------------------------------------------------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+--------+------+-----------------+-----------+---------+-------+---------------------------------+---------------------------------------------------------------------+----------+
| 274226 | root | localhost:45423 | edc_proxy | Sleep | 16043 | | NULL | 0.000 |
| 274319 | root | localhost | myDB | Query | 99 | Waiting for table metadata lock | ALTER TABLE `runs` CHANGE `p_w_trans_x` `p_w_tran_x` FLOAT NOT NULL | 0.000 |
| 274416 | root | localhost | NULL | Query | 0 | NULL | show full processlist | 0.000 |
+--------+------+-----------------+-----------+---------+-------+---------------------------------+---------------------------------------------------------------------+----------+
This answer suggests checking the information_schema tables, not much there:
MariaDB [INFORMATION_SCHEMA]> SELECT * FROM INNODB_LOCK_WAITS;
Empty set (0.00 sec)
MariaDB [INFORMATION_SCHEMA]> SELECT * FROM INNODB_LOCKS ;
Empty set (0.00 sec)
MariaDB [INFORMATION_SCHEMA]> SELECT * FROM INNODB_TRX;
+----------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+
| trx_id | trx_state | trx_started | trx_requested_lock_id | trx_wait_started | trx_weight | trx_mysql_thread_id | trx_query | trx_operation_state | trx_tables_in_use | trx_tables_locked | trx_lock_structs | trx_lock_memory_bytes | trx_rows_locked | trx_rows_modified | trx_concurrency_tickets | trx_isolation_level | trx_unique_checks | trx_foreign_key_checks | trx_last_foreign_key_error | trx_adaptive_hash_latched | trx_adaptive_hash_timeout |
+----------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+
| 83A8B36E | RUNNING | 2016-12-08 11:13:02 | NULL | NULL | 0 | 274226 | NULL | NULL | 0 | 0 | 0 | 376 | 0 | 0 | 0 | REPEATABLE READ | 1 | 1 | NULL | 0 | 10000 |
+----------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+
1 row in set (0.00 sec)
And the section on transactions from show engine innodb status;:
------------
TRANSACTIONS
------------
Trx id counter 83A8F071
Purge done for trx's n:o < 83A8CA86 undo n:o < 0
History list length 1490
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started
MySQL thread id 274543, OS thread handle 0x7fbb863e6700, query id 85356480 localhost root
show engine innodb status
---TRANSACTION 83A8EB07, not started
mysql tables in use 1, locked 2
MySQL thread id 274542, OS thread handle 0x7fbb843f6700, query id 85354935 localhost root Waiting for table metadata lock
ALTER TABLE `runs` CHANGE `p_w_trans_x` `p_w_tran_x` FLOAT NOT NULL
---TRANSACTION 83A8B36E, ACTIVE 24627 sec
MySQL thread id 274226, OS thread handle 0x7fbb845f5700, query id 85337236 localhost 127.0.0.1 root
Trx read view will not see trx with id >= 83A8B36F, sees < 83A8B36D
----------------------------
END OF INNODB MONITOR OUTPUT
============================
Any pointers for further investigation, for circumventing the problem and for solving are appreciated!
A metadata lock is an implicit (from the user perspective) lock that prevents DDL against the table because something else needs the table to remain in its current form. In this case, it's a transaction that has been left running.
Task 1: Your alter will succeed if you kill the connection on thread 274226.
mysql> KILL 274226;
The problem here, as indicated by information_schema.innodb_trx, is that this thread has left a transaction running for several hours and we can infer that this table has been referenced by that transaction. A table can't be altered until no transactioms still have an MVCC view or any locks involving the table. This transaction holds a view, which we can again infer could impact this table, as shown in the last line:
--TRANSACTION 83A8B36E, ACTIVE 24627 sec
MySQL thread id 274226, OS thread handle 0x7fbb845f5700, query id 85337236 localhost 127.0.0.1 root
Trx read view will not see trx with id >= 83A8B36F, sees < 83A8B36D
Note that Sleep is not a real command, in this context, it's just the placeholder status for any idle connection. All connections are doing something, and in this case the "something" is sleeping -- in other words, idle and waiting for another query. But an idle connection is still a connection, and if your code (or query browser tool) leaves a transaction running, it just keeps running.
Task 2: find the bug or mistake that left that transaction running. In a live application, leaving transactions running potentially make a much bigger mess.
I was having a very similar issue with table lock, but it ended up being MySQL Workbench out of all things. Any RENAME TABLE or ALTER TABLE command in MySQL Workbench would just sit claiming a meta lock. I logged onto the server and was able to execute those types of queries no problem. Current version of Workbench is 8.0.22.
I had sifted through all these queries and was totally stumped when they showed no issues:
SHOW OPEN TABLES;
SHOW ENGINE inndodb STATUS;
SELECT * FROM INNODB_LOCK_WAITS;
SELECT * FROM INNODB_LOCKS;
SELECT * FROM INNODB_TRX;
SHOW FULL PROCESSLIST;
I'm having this problem, here is my sql statements:
select * from tb1 where id > the_max_read order by id
The table tb1 is to monitor some other tables' changes, so it keeps growing.
Variable the_max_read is the max id that program already read.
I'm running this sql via C++ and using mysql's mysql_query function, and save result with mysql_store_result.
DB engine is innodb.
The problem is that it miss some rows sometime, not always but keep happening.
For example, say I have this table:
| -- id | -- name|
| 834370 | name1 |
| 834371 | name2 |
| 834372 | name3 |
| 834373 | name4 |
| 834374 | name5 |
| 834375 | name6 |
and the_max_read=834371, when run the above sql, the result only contains 834374 and 834375.
Though this table may be inserted some new rows by other programs, but I still cannot understand why it just miss some rows, it's almost the simplest sql.
This sounds like it might be a transaction issue, where you read before some of the transactions are committed.
Try read uncommitted data: http://dev.mysql.com/doc/refman/5.1/en/set-transaction.html
e.g.
SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
select * from tb1 where id > the_max_read order by id;
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
Hope that helps.
I am trying to optimize a query, and all looks well when I got to "EXPLAIN" it, but it's still coming up in the "log_queries_not_using_index".
Here is the query:
SELECT t1.id_id,t1.change_id,t1.like_id,t1.dislike_id,t1.user_id,t1.from_id,t1.date_id,t1.type_id,t1.photo_id,t1.mobile_id,t1.mobiletype_id,t1.linked_id
FROM recent AS t1
LEFT JOIN users AS t2 ON t1.user_id = t2.id_id
WHERE t2.active_id=1 AND t1.postedacommenton_id='0' AND t1.type_id!='Friends'
ORDER BY t1.id_id DESC LIMIT 35;
So it grabs like a 'wallpost' data, and then I joined in the USERS table to make sure the user is still an active user (the number 1), and two small other "ANDs".
When I run this with the EXPLAIN in phpmyadmin it shows
id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra
1 | SIMPLE | t1 | index | user_id | PRIMARY | 4 | NULL | 35 | Using where
1 | SIMPLE | t2 | eq_ref | PRIMARY,active_id | PRIMARY | 4 | hnet_user_info.t1.user_id | 1 | Using where
It shows the t1 query found 35 rows using "WHERE", and the t2 query found 1 row (the user), using "WHERE"
So I can't figure out why it's showing up in the log_queries_not_using_index report.
Any tips? I can post more info if you need it.
tldr; ignore the "not using index warning". A query execution time of 1.3 milliseconds is not a problem; there is nothing to optimize here - look at the entire performance profile to find bottlenecks.
Trust the database engine. The database query planner will use indices when it determines that doing so is beneficial. In this case, due to the low cardinality estimates (35x1), the query planner decided that there was no reason to use indices for the actual execution plan. If indices were used in a trivial case like this it could actually increase the query execution time.
As always, use the 97/3 rule.
I have problem related to mysql database. i am linux webserver admin and i am facing a problem with a mysql query. The database is very small. I tried to track in logs and found that a query is taking minimum 5 sec to respond . The first page of site is coming from the database. Client are using cms. when the server gets some number of hits database server starts to give response very slowly and wait time increases from 5 sec to several seconds.
I checked slow query logs
{
Query_time: 11.480138 Lock_time: 0.003837 Rows_sent: 921 Rows_examined: 3333
SET timestamp=1346656767;
SELECT `Tender`.`id`,
`Tender`.`department_id`,
`Tender`.`title_english`,
`Tender`.`content_english`,
`Tender`.`title_hindi`,
`Tender`.`content_hindi`,
`Tender`.`file_name`,
`Tender`.`start_publish`,
`Tender`.`end_publish`,
`Tender`.`publish`,
`Tender`.`status`,
`Tender`.`createdBy`,
`Tender`.`created`,
`Tender`.`modifyBy`,
`Tender`.`modified`
FROM `mcms_tenders` AS `Tender`
WHERE `Tender`.`department_id` IN ( 31, 33, 32, 30 );
}
Every line in the log is same only there is diff in Query time.
Is there any way tweak the performance?
Update: Here is the EXPLAIN result:
+----+-------------+--------+------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+------+---------+------+------+-------------+
| 1 | SIMPLE | Tender | ALL | NULL | NULL | NULL | NULL | 3542 | Using where |
+----+-------------+--------+------+---------------+------+---------+------+------+-------------+
1 row in set, 1 warning (0.00 sec)
client is saying they are using Index so i run the command to check the indexing.
I got following output. Does It means they are using Indexing.
+--------------+------------+----------+--------------+-------------+-----------+------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| mcms_tenders | 0 | PRIMARY | 1 | id | A | 4264 | NULL | NULL | | BTREE | |
+--------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
The normal way to tweak the performance of a query like this is to create an index on department_id.
However, this assumes that Tenders is actually a table and not a view. You should confirm this, since the problem may be in a view.
Also, from what you describe the issue may be the connection from the server to the end users. I would try running the query locally on the server (or checking the execute time strictly on the server) to see if the query is really taking that long.
"when the server gets some number of hits"
Define 'some number'. It makes sense that reading the database is slower when it is more heavily used. Also, MySQL has a query cache that is fully invalidated when changes are made to the data. So every time someone inserts, deletes or modifies a record in this table, the next queries will be slower because the table date is still uncached.
But 11 seconds for a query like this is very slow, so either the load is way too high, the hardware insufficient or broken or your database lacks indexes (I always forget to mention that at first, because I assume adding indexes to be a second nature for anyone working with databases).
I have a MySQL query that is copying data from one table to another for processing. For some reason, this query that normally takes a few seconds locked up overnight and ran for several hours. When I logged in this morning, I tried to kill the query, but it is still listed in the process list.
| Id | User | Host | db | Command | Time | State | Info |
+---------+----------+-----------+------+---------+-------+--------------+--------------------------------------------------------------------------------------+
| 1061763 | tb_admin | localhost | dw | Killed | 45299 | Sending data | INSERT INTO email_data_inno_stage SELECT * FROM email_data_test LIMIT 4480000, 10000 |
| 1062614 | tb_admin | localhost | dw | Killed | 863 | Sending data | INSERT INTO email_data_inno_stage SELECT * FROM email_data_test LIMIT 4480000, 10000 |
What could have caused this, and how can I kill this process so I can get on with my work?
If the table email_data_test is MyISAM and it was locked, that would have held up the the INSERT.
If the table email_data_test is InnoDB, then a lot of MVCC data was being written in ib_logfiles, which may not have occurred yet.
In both cases, you had the LIMIT clause scroll through 4,480,000 rows just to get to 10,000 rows you actually needed to INSERT.
Killing the query only causes the InnoDB table email_data_inno_stage to execute a rollback.