I'm trying to add a column to a specific table. Every time I run the migration up or down a SLEEP command pops up and blocks everything. Much like many other people who have run into this problem, I kill the blocking process and everything works as expected.
Ex:MySql alter table hangs
I've tried running the migration against a different table and have no issues. Seems to be something specific with this particular table.
Where or what should I be looking for? Why would this issue occur so consistently?
Thanks.
mysql> show processlist;
+-----+------+-----------------+--------+---------+------+---------------------------------+----------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-----+------+-----------------+--------+---------+------+---------------------------------+----------------------------------------------+
| 351 | root | localhost:54691 | database | Sleep | 25 | | NULL |
| 352 | root | localhost:54692 | NULL | Sleep | 54 | | NULL |
| 377 | root | localhost | database | Query | 0 | starting | show processlist |
| 381 | root | localhost:54858 | database | Query | 5 | Waiting for table metadata lock | ALTER TABLE company ADD COLUMN active |
| 382 | root | localhost:54860 | database | Sleep | 5 | | NULL |
+-----+------+-----------------+--------+---------+------+---------------------------------+----------------------------------------------+
5 rows in set (0.00 sec)
The offending process in this case was 382.
The answers to many other similar posts say to kill the SLEEP process and then "debug". My problem was related to a circular import issue in Python, not related to MySQL.
Related
I am using Server version: 5.6.27-log MySQL Community Server (GPL) and I have a problem with a table.
I tried to update some fields with a GUI software, but when I came back to the command line, the lines I tried to update where not updated.
I tried to see if the table was locked using SHOW OPEN TABLES as stated in various other questions. But my table does not appear to be locked:
+--------------------+-------------------------------------------------+--------+-------------+
| Database | Table | In_use | Name_locked |
+--------------------+-------------------------------------------------+--------+-------------+
| arcdev | SCHEDULED_COMMAND | 0 | 0 |
And as soon as I try to make an update like:
UPDATE SCHEDULED_COMMAND SET field = 1;
The server just keeps loading and nothing happen. I tried on other tables and everything worked just fine.
I also tried some DELETE requests and even a DROP TABLE and nothing work so far...
What am I missing?
Thank you for your precious help!
EDIT: Here is the result of the SHOW PROCESSLIST command while a request is hanging:
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
| 282588 | rdsadmin | localhost:17966 | mysql | Sleep | 2 | | NULL |
| 534575 | arc | XXXXXX:49376 | arcdev | Sleep | 17 | | NULL |
| 534579 | arc | XXXXXX:49443 | arcdev | Query | 0 | init | SHOW PROCESSLIST |
| 534659 | arc | XXXXXX:49836 | arcdev | Query | 14 | updating | DELETE FROM SCHEDULED_COMMAND |
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
I have a large amount of connections but when I issue a show full processlist I am not showing anything close to the connections I see. Are these connections orphans of some sort? I tried the flush hosts command and the connections persist, even with a reboot of the server and also restarting the mysql server.
I believe these connections are causing issues with making new connections to the database. User's are getting a "server went away" error. How do I clear these?
See commands below:
mysql> show status like '%onn%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| Aborted_connects | 5 |
| Connections | 11743 |
| Max_used_connections | 24 |
| Ssl_client_connects | 0 |
| Ssl_connect_renegotiates | 0 |
| Ssl_finished_connects | 0 |
| Threads_connected | 6 |
+--------------------------+-------+
7 rows in set (0.00 sec)
mysql> show full processlist;
+-------+---------+----------------------+--------------------+---------+-------+-------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-------+---------+----------------------+--------------------+---------+-------+-------+-----------------------+
| 4494 | rode | localhost:43411 | NULL | Sleep | 11159 | | NULL |
| 4506 | rode | localhost:43423 | information_schema | Sleep | 11159 | | NULL |
| 4554 | rode | localhost:43511 | performance_schema | Sleep | 11112 | | NULL |
| 11500 | ass | serv:1243 | Home-Tech | Sleep | 0 | | NULL |
| 11743 | root | localhost | NULL | Query | 0 | NULL | show full processlist |
| 11744 | ass | out:6070 | Home-Tech | Sleep | 4 | | NULL |
| 11745 | ass | out:6074 | HTGlobal | Sleep | 8 | | NULL
The MySQL server has gone away (error 2006) has two main causes
Server timed out and closed the connection. To fix, check that “wait_timeout” mysql variable in your my.cnf configuration file is large enough.
Server dropped an incorrect or too large packet. If mysqld gets a packet that is too large or incorrect, it assumes that something has gone wrong with the client and closes the connection. To fix, you can increase the maximal packet size limit “max_allowed_packet” in my.cnf file, eg. set max_allowed_packet = 128M, then sudo /etc/init.d/mysql restart.
there are two main ways to fix this. if the above change doesn't there may be an issue with your linux or windows mysql database server; you either need to increase ram on your server or watch it's process.
is this on a windows or linux box?
My ~700mb database has been importing for 1hr 27min now, the actual disk activity stopped at around 15min but I just left it running to see if it would finish by itself.
The command I ran:
mysqldump DB1 | pv | mysql DB2
So it's pretty much a straight copy from 1 database to another, with DB2 starting as empty.
I can actually see that the data is already in DB2, but the command refuses to end!
So the question is... Should I let it continue to run? Or can I kill it? :/
Updated:
SHOW PROCESSLIST;
+------+--------------+---------------------+------------+---------+------+--------+------------------------------------------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+--------------+---------------------+------------+---------+------+--------+------------------------------------------------------------------------------------------------------+
| 2762 | user1 | localhost | DB2 | Query | 5298 | Locked | /*!50003 CREATE*/ /*!50020 DEFINER="user2"#"%"*/ /*!50003 FUNCTION "function1" |
| 2763 | user1 | localhost | DB1 | Sleep | 5298 | | NULL |
| 2770 | user2 | localhost | NULL | Query | 3633 | Locked | SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`ROUTINES` WHERE `ROUTINE_SCHEMA`='DB2' AND `ROUTIN |
| 2775 | user2 | localhost | NULL | Query | 381 | Locked | SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`ROUTINES` WHERE `ROUTINE_SCHEMA`='DB2' AND `ROUTIN |
| 2776 | user2 | <ipaddress>:<port> | NULL | Query | 0 | NULL | show processlist |
+------+--------------+---------------------+------------+---------+------+--------+------------------------------------------------------------------------------------------------------+
Some settings I have that are non-standard:
innodb_stats_on_metadata=0
innodb_flush_log_at_trx_commit=2
I created views on the tables
each of them had about 46299 rows
I created them successfully in phpMyadmin
after that I created a join of these two views
which itself is a view.
in mysql in a millisecond I got Query OK, 0 rows affected (0.01 sec)
but I couldn't reach my phpMyAdmin
so I decided to drop the view
in mysql it took about 20 minuets for dropping
again I tried my way and now I can't reach phpMyadmin
I don't know that it is under prcessing or not?!
how much I should wait? and why mysql show me fast query ok but phpmyAdmin is still processing if It is really under process
+-----+------+-----------------+-------+---------+------+--------------+--------
------------------------------------+
| Id | User | Host | db | Command | Time | State | Info
|
+-----+------+-----------------+-------+---------+------+--------------+--------
------------------------------------+
| 7 | root | localhost:38993 | week1 | Sleep | 2188 | | NULL
|
| 20 | root | localhost:39223 | week1 | Query | 0 | NULL | SHOW PR
OCESSLIST |
| 104 | root | localhost:39526 | NULL | Query | 800 | Sending data | SELECT
1 FROM `week1`.`alarm` LIMIT 100000 |
| 105 | root | localhost:39527 | week1 | Query | 800 | Sending data | SELECT
1 FROM `week1`.`alarm` LIMIT 100000 |
+-----+------+-----------------+-------+---------+------+--------------+--------
------------------------------------+
4 rows in set (0.00 sec)
mysql>
You can see if something have locked the table by issuing a SHOW PROCESSLIST command.
When I try to drop a table, MySQL hangs. I don't have any other open sessions. How to resolve this? I have waited for 10 hours and the process has not terminated.
I'm trying easier answer for newbies as i am:
1) run :
SHOW PROCESSLIST
if you get something like:
+----+-----------------+-----------------+--------+------------+-----------+---------------------------------+---------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-----------------+-----------------+--------+------------+-----------+---------------------------------+---------------------------------------------------+
| 4 | event_scheduler | localhost | NULL | Daemon | 580410103 | Waiting on empty queue | NULL |
| 13 | root | localhost:50627 | airbnb | Sleep | 10344 | | NULL |
| 17 | root | localhost:50877 | NULL | Query | 2356 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`characteristics` |
| 18 | root | localhost:50878 | airbnb | Query | 2366 | Waiting for table metadata lock | DROP TABLE `airbnb`.`characteristics` |
| 21 | root | localhost:51281 | airbnb | Query | 2305 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`bed_type` |
| 22 | root | localhost:51282 | airbnb | Query | 2301 | Waiting for table metadata lock | SHOW INDEXES FROM `airbnb`.`characteristics` |
| 23 | root | localhost:51290 | airbnb | Query | 2270 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`property_type` |
| 24 | root | localhost:51296 | airbnb | Query | 2240 | Waiting for table metadata lock | SHOW INDEXES FROM `airbnb`.`property_type` |
| 26 | root | localhost:51303 | NULL | Query | 2212 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`characteristics` |
| 27 | root | localhost:51304 | NULL | Query | 2218 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`bed_type` |
| 29 | root | localhost:51306 | NULL | Query | 2176 | Waiting for table metadata lock | SHOW INDEXES FROM `airbnb`.`characteristics` |
| 30 | root | localhost:51308 | NULL | Query | 2122 | Waiting for table metadata lock | DROP TABLE `airbnb`.`characteristics` |
| 34 | root | localhost:51312 | NULL | Query | 2063 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`characteristics` |
| 35 | root | localhost:51313 | NULL | Query | 2066 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`bed_type` |
| 39 | root | localhost:51338 | NULL | Query | 2004 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`characteristics` |
| 40 | root | localhost:51339 | NULL | Query | 2008 | Waiting for table metadata lock | SHOW FULL COLUMNS FROM `airbnb`.`bed_type` |
| 45 | root | localhost | airbnb | Field List | 997 | Waiting for table metadata lock | |
| 46 | root | localhost | airbnb | Field List | 798 | Waiting for table metadata lock | |
| 53 | root | localhost | airbnb | Query | 0 | starting | SHOW PROCESSLIST |
+----+-----------------+-----------------+--------+------------+-----------+---------------------------------+---------------------------------------------------+
with State : waiting for table metadata lock (as mentioned in official answer)
2) KILL 13 (13 coresponding to the Id).
If it's indeed a deadlock, all the following processes will continue normally.
Waiting for table metadata lock
drop table tableA name
SELECT l1.lat, l1.lon, l2.zipcode FROM tableA l1, tableBl2 where l1.lat = l2.latitude and l1.lon = l2.longitude limit 10
If this is your table, see this link
you have an implicit deadlock. Kill the other transactions to release the drop, or kill the drop to release the other transactions.
You can use KILL thread_id, in sql_plus.
I'm adding further information since I came up with another interesting experience.
Metadata Dead locks may equally happen between a ddl operation on a given table (drop, alter...) and a select query on that table.
Yes, select.
So, if you loop over a cursor in mysql (or php, for example with pdo::fetch), and you run a ddl statement on the same table(s), you will get a deadlock.
One solution to this atypical scenario is to release the implicit locks with a commit statement systematically after any select statement is completely fetched.
Restarting MySQL might not be the prettiest solution but it worked for me:
sudo /etc/init.d/mysql restart
mysqladmin drop YOURDATABASE