I have a mysql 5.1 db running a stored proc with
START TRANSACTION
Insert some rows to table 1
Insert some rows to table 2
COMMIT
Calling this stored procedure often fails with
SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction
According to this page here, if mysql server is not started with innodb_rollback_on_timeout then only the last statement is rolled back but START TRANSACTION itself will set autocommit = 0. Does that mean that our mysql server needs to be started with this parameter so that it doesn't leave the db in an inconsistent state where some rows are inserted into table 1 but not into table 2?
Yes, either that, or declare a handler "FOR '1205'" in your procedure by which you could (eg.) roll back the transaction and interrupt the process.
You can rollback yourself if the calling client checks for errors and rolls back the transaction if an error occurs. As stated in the bug log:
In the event of a row level lock timeout, it can be desirable to allow your application to decide what to do (such as ROLLBACK, retry the statement, etc...) so this behavior was added with an option for backwards compatibility if desired.
Otherwise yes - if you don't check for lock wait errors, and you always want the whole transaction to rollback, then you should set innodb_rollback_on_timeout in your my.cnf.
Related
I did some queries without a commit. Then the application was stopped.
How can I display these open transactions and commit or cancel them?
How can I display these open transactions and commit or cancel them?
There is no open transaction, MySQL will rollback the transaction upon disconnect.
You cannot commit the transaction (IFAIK).
You display threads using
SHOW FULL PROCESSLIST
See: http://dev.mysql.com/doc/refman/5.1/en/thread-information.html
It will not help you, because you cannot commit a transaction from a broken connection.
What happens when a connection breaks
From the MySQL docs: http://dev.mysql.com/doc/refman/5.0/en/mysql-tips.html
4.5.1.6.3. Disabling mysql Auto-Reconnect
If the mysql client loses its connection to the server while sending a statement, it immediately and automatically tries to reconnect once to the server and send the statement again. However, even if mysql succeeds in reconnecting, your first connection has ended and all your previous session objects and settings are lost: temporary tables, the autocommit mode, and user-defined and session variables. Also, any current transaction rolls back.
This behavior may be dangerous for you, as in the following example where the server was shut down and restarted between the first and second statements without you knowing it:
Also see: http://dev.mysql.com/doc/refman/5.0/en/auto-reconnect.html
How to diagnose and fix this
To check for auto-reconnection:
If an automatic reconnection does occur (for example, as a result of calling mysql_ping()), there is no explicit indication of it. To check for reconnection, call mysql_thread_id() to get the original connection identifier before calling mysql_ping(), then call mysql_thread_id() again to see whether the identifier has changed.
Make sure you keep your last query (transaction) in the client so that you can resubmit it if need be.
And disable auto-reconnect mode, because that is dangerous, implement your own reconnect instead, so that you know when a drop occurs and you can resubmit that query.
Although there won't be any remaining transaction in the case, as #Johan said, you can see the current transaction list in InnoDB with the query below if you want.
SELECT * FROM information_schema.innodb_trx\G
From the document:
The INNODB_TRX table contains information about every transaction (excluding read-only transactions) currently executing inside InnoDB, including whether the transaction is waiting for a lock, when the transaction started, and the SQL statement the transaction is executing, if any.
You can use show innodb status (or show engine innodb status for newer versions of mysql) to get a list of all the actions currently pending inside the InnoDB engine. Buried in the wall of output will be the transactions, and what internal process ID they're running under.
You won't be able to force a commit or rollback of those transactions, but you CAN kill the MySQL process running them, which does essentially boil down to a rollback. It kills the processes' connection and causes MySQL to clean up the mess its left.
Here's what you'd want to look for:
------------
TRANSACTIONS
------------
Trx id counter 0 140151
Purge done for trx's n:o < 0 134992 undo n:o < 0 0
History list length 10
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0 0, not started, process no 17004, OS thread id 140621902116624
MySQL thread id 10594, query id 10269885 localhost marc
show innodb status
In this case, there's just one connection to the InnoDB engine right now (my login, running the show query). If that line were an actual connection/stuck transaction you'd want to terminate, you'd then do a kill 10594.
By using this query you can see all open transactions.
List All:
SHOW FULL PROCESSLIST
if you want to kill a hang transaction copy transaction id and kill transaction by using this command:
KILL <id> // e.g KILL 16543
With this query below, you can check how many transactions are running currently:
mysql> SELECT count(*) FROM information_schema.innodb_trx;
+----------+
| count(*) |
+----------+
| 3 |
+----------+
1 row in set (0.00 sec)
I need to make MySQL server to rollback transaction immediately after its client disconnected, because each client works concurrently. The problem can be reproduced like these (using an innodb table type)
On Client A:
START TRANSACTION;
SELECT MAX(ID) FROM tblone FOR UPDATE;
#... then disconnect your connection to the server
On Client B:
START TRANSACTION;
SELECT MAX(ID) FROM tblone FOR UPDATE;
#... lock wait time out will occur here
I had set MySQL's server option like innodb_rollback_on_timeout and using mysql's client mysql --skip-reconnect on both client. I tried this using one server and two client, on a network. I disconnected the network physically (unplug the cable) after SELECT ... FOR UPDATE; line. I need to make other clients to be able to use tblone on a transaction (lock it, update it) immediately, and for that to happen I think the server should rollback the transaction for Client A, after Client A disconnects.
When you are physically disconnecting a client you're not sending a normal disconnect (which would have caused a rollback) and the MySQL protocol isn't very chatty so the server never knows that the client isn't there. I think this is a flaw in the protocol when comparing to other database systems where the client and server talks internally much more.
Anyway. There are two variables that you could change. They basically do the same but for different clients.
The first is wait_timeout and it is used by application clients like java or php.
The other is interactive_timeout and it is used by the mysql client (as in your tests)
In both cases the server to kills the connection after a number of seconds and when doing so rollbacks all transactions and releases all locks.
This is to discuss some of the Comments. Please note that this disagrees with some of the Comments. I'll use an INSERT instead of SELECT..FOR UPDATE because the effect is more visible.
Let's look at some different cases:
(1) No SQL + timeout
START TRANSACTION;
do some SQL statement(s)
do no SQL for more than the timeout (before COMMITing)
Avoid this because of the cases detailed below. Solution: Don't depend on InnoDB to help you with long transactions.
(2) long-running query
START TRANSACTION;
do some SQL statement(s)
run an SQL query that takes more than the timeout
COMMIT;
All is well. The timeout does not apply as long as the server (mysqld) is continuing to perform queries. That is, the timeout 'clock' starts over at the end of each SQL statement
(3) (auto-reconnect)
START TRANSACTION;
INSERT ... VALUES (123);
time passes; no SQL performed for longer than the timeout
disconnect occurs
INSERT ... VALUES (456);
auto-reconnect (because you have it ENabled);
the INSERT proceeds
COMMIT;
123 will be rolled back; 456 will be inserted. (Similarly SELECT..FOR UPDATE would lose the locks.) Not good. The solution is to turn off "auto-reconnect". Instead, check for errors and treat the disconnect error as a fatal error for the Transaction. (Then start over the transaction.)
The INSERT 456 will be running in a new transaction, controlled by autocommit.
(4) (NO auto-reconnect)
START TRANSACTION;
INSERT ... VALUES (123);
time passes; no SQL for longer than the timeout
disconnect occurs
INSERT ... VALUES (456);
NO auto-reconnect (because you have it DISabled)
COMMIT;
123 will be rolled back. The INSERT of 456 will get an error something like "connection lost". Start the transaction over.
I am using aiomysql (https://github.com/aio-libs/aiomysql) and have some problems with the unclosed transaction and locking rows. I use AIO connections pool in my application.
I am NOT using SA context managers for transactions.
My questions:
If I do only and only SELECT's as I understand there are no locks on rows, So do I need to call wait for conn.commit() or I can skip it if I can skip how MySQL should now what the transaction ends?
In the code below, then AIO MySQL start a new transaction? then acquire() connection called or then create cursor called or I should explicitly call "START TRANSACTION"?
The commit needs to be inside the try block as you want to be sure to rollback if there is no commit. However selects do not require commits.
If autocommit is True on your connection then each insert or update is considered a single transaction and implicitly committed. If autocommit is False then you automatically get transactions and must commit after your inserts. You do not need to call START TRANSACTION if autocommit is false.
If you need to call START TRANSACTION you use conn.begin() documented here:
https://aiomysql.readthedocs.io/en/latest/connection.html#connection
A MySQL transaction is used if you have multiple contingent updates that must all be successful together or rolled back. For example a bank transfer that fails on the second update needs to be rolled back:
Withdraw money from account A
Deposit money in account B
You can find a transaction example in the aiomysql github.
https://github.com/aio-libs/aiomysql/tree/master/examples
I need to make MySQL server to rollback transaction immediately after its client disconnected, because each client works concurrently. The problem can be reproduced like these (using an innodb table type)
On Client A:
START TRANSACTION;
SELECT MAX(ID) FROM tblone FOR UPDATE;
#... then disconnect your connection to the server
On Client B:
START TRANSACTION;
SELECT MAX(ID) FROM tblone FOR UPDATE;
#... lock wait time out will occur here
I had set MySQL's server option like innodb_rollback_on_timeout and using mysql's client mysql --skip-reconnect on both client. I tried this using one server and two client, on a network. I disconnected the network physically (unplug the cable) after SELECT ... FOR UPDATE; line. I need to make other clients to be able to use tblone on a transaction (lock it, update it) immediately, and for that to happen I think the server should rollback the transaction for Client A, after Client A disconnects.
When you are physically disconnecting a client you're not sending a normal disconnect (which would have caused a rollback) and the MySQL protocol isn't very chatty so the server never knows that the client isn't there. I think this is a flaw in the protocol when comparing to other database systems where the client and server talks internally much more.
Anyway. There are two variables that you could change. They basically do the same but for different clients.
The first is wait_timeout and it is used by application clients like java or php.
The other is interactive_timeout and it is used by the mysql client (as in your tests)
In both cases the server to kills the connection after a number of seconds and when doing so rollbacks all transactions and releases all locks.
This is to discuss some of the Comments. Please note that this disagrees with some of the Comments. I'll use an INSERT instead of SELECT..FOR UPDATE because the effect is more visible.
Let's look at some different cases:
(1) No SQL + timeout
START TRANSACTION;
do some SQL statement(s)
do no SQL for more than the timeout (before COMMITing)
Avoid this because of the cases detailed below. Solution: Don't depend on InnoDB to help you with long transactions.
(2) long-running query
START TRANSACTION;
do some SQL statement(s)
run an SQL query that takes more than the timeout
COMMIT;
All is well. The timeout does not apply as long as the server (mysqld) is continuing to perform queries. That is, the timeout 'clock' starts over at the end of each SQL statement
(3) (auto-reconnect)
START TRANSACTION;
INSERT ... VALUES (123);
time passes; no SQL performed for longer than the timeout
disconnect occurs
INSERT ... VALUES (456);
auto-reconnect (because you have it ENabled);
the INSERT proceeds
COMMIT;
123 will be rolled back; 456 will be inserted. (Similarly SELECT..FOR UPDATE would lose the locks.) Not good. The solution is to turn off "auto-reconnect". Instead, check for errors and treat the disconnect error as a fatal error for the Transaction. (Then start over the transaction.)
The INSERT 456 will be running in a new transaction, controlled by autocommit.
(4) (NO auto-reconnect)
START TRANSACTION;
INSERT ... VALUES (123);
time passes; no SQL for longer than the timeout
disconnect occurs
INSERT ... VALUES (456);
NO auto-reconnect (because you have it DISabled)
COMMIT;
123 will be rolled back. The INSERT of 456 will get an error something like "connection lost". Start the transaction over.
Consider the following:
START TRANSACTION;
BEGIN;
INSERT INTO prp_property1 (module_name,environment_name,NAME,VALUE) VALUES ('','production','','300000');
/** Assume there is syntax error SQL here...**/
Blah blah blah
DELETE FROM prp_property1 WHERE environment_name = 'production';
COMMIT TRANSACTION;
Question:
I noticed that the transaction automatically rolls back and the record insert attempt fails.
If I don't provide a error handler or error check along with ROLLBACK TRANSACTION as above, is it safe as it seems to be doing the job in an example like above because the COMMIT TRANSACTION never gets executed?
I assume the transaction is rolled back immediately and discarded as soon as a error occurs.
No, transactions are not rolled back as soon as an error occurs. But you may be using a client-application which applies this policy.
For example, if you are using the mysql command-line client, then it normally stops executing when an error occurs and will quit. Quitting while a transaction is in progress does cause it to be rolled back.
When you are writing your own application, you can control the policy on rollback, but there are some exceptions:
Quitting (i.e. disconnecting from the database) always rolls back a transaction in progress
A deadlock or lock-wait timeout implicitly causes a rollback
Other than these conditions, if you invoke a command which generates an error, the error is returned as normal, and you are free to do whatever you like, including committing the transaction anyway.
Use Mysql stored procedure
BEGIN
DECLARE exit handler for sqlexception
BEGIN
ROLLBACK;
END;
DECLARE exit handler for sqlwarning
BEGIN
ROLLBACK;
END;
START TRANSACTION;
INSERT INTO prp_property1 (module_name,environment_name,NAME,VALUE) VALUES ('','production','','300000');
[ERROR]
COMMIT;
END
You can set if warning or error rollback, then you don't need delete, with transaction all entry is deleted.
You may use procedure to do this more effectively.
Transaction with Stored Procedure in MySQL Server
I would like to add to what #MarkR already said. Error Handling, assuming InnoDB engine, happens as described in the Mysql Server Documentation
If you run out of file space in a tablespace, a MySQL Table is full error occurs and InnoDB rolls back the SQL statement.
A transaction deadlock causes InnoDB to roll back the entire transaction.
A duplicate-key error rolls back the SQL statement
A row too long error rolls back the SQL statement.
Other errors are mostly detected by the MySQL layer of code (above the InnoDB storage engine level), and they roll back the corresponding SQL statement
My understanding is also that when the Mysql session ends (when the php scripts ends), anything that is not committed is rolled back. I yet have to find a really reliable source to back this statement so do not take my word for it.
I've tested these three situations; mySQL does not roll back automatically.
A transaction deadlock causes InnoDB to roll back the entire transaction.
A duplicate-key error rolls back the SQL statement
A row too long error rolls back the SQL statement.
Only the affected records fail, the rest of the records succeed unless your application calls "rollback" explicitly.