MySQL - detecting and handling Lock Wait Timeout inside Stored Procedure - mysql

The Question
purely for academic reasons, I'm wondering if you could add a handler to a mysql stored procedure that is able to recover from a lock wait timeout error if one of its queries locks up (such as a SELECT ... FOR UPDATE or UPDATE) query.
The Example
This is assuming an innoDB database, set to issolation level Repeatable read, with an empty users table defined.
1. Example Procedure:
DROP PROCEDURE IF EXISTS `lock_test`;
DELIMITER ;;
CREATE PROCEDURE `lock_test`(OUT status ENUM('success','timeout'))
MODIFIES SQL DATA
BEGIN
START TRANSACTION;
SELECT * FROM `users` FOR UPDATE;
SET status := 'success';
COMMIT;
END;;
DELIMITER ;
2. Run code in mysql terminal 1:
START TRANSACTION;
SELECT * FROM `users` FOR UPDATE;
the contents of users will be displayed, but the transaction will remain open.
3. Run code in mysql terminal 2:
CALL `lock_test`(#out);
SELECT #out;
the transaction will run until it times out (default value of innodb_lock_wait_timeout is 50 seconds)
Is it possible to add a handler inside the lock_test() procedure, so that we can have #out hold 'timeout'?

After spending some time reading through the MySQL Handler Documentation I was able to get what I was looking for:
DROP PROCEDURE IF EXISTS `lock_test`;
DELIMITER ;;
CREATE PROCEDURE `lock_test`(OUT status_out VARCHAR(255))
MODIFIES SQL DATA
BEGIN
DECLARE procedure_attempts INT DEFAULT 5;
DECLARE query_timeout INT DEFAULT FALSE;
SET status_out := 'start';
procedure_loop:
REPEAT
BEGIN
DECLARE CONTINUE HANDLER FOR 1205
-- Error: 1205 SQLSTATE: HY000 (ER_LOCK_WAIT_TIMEOUT)
BEGIN
SET query_timeout := TRUE;
SET status_out := CONCAT(status_out,'-timeout');
END;
IF ( procedure_attempts < 1) THEN
LEAVE procedure_loop;
END IF;
START TRANSACTION;
SELECT * FROM `users` FOR UPDATE;
IF (query_timeout) THEN
SET query_timeout := FALSE;
ELSE
SET status_out := CONCAT(status_out,'-success');
SET procedure_attempts := 0;
END IF;
COMMIT;
SET procedure_attempts := procedure_attempts - 1;
END;
UNTIL FALSE END REPEAT;
-- loop
SET status_out := CONCAT(status_out,'-end');
END;;
DELIMITER ;
When run as follows:
SET ##innodb_lock_wait_timeout:=1;
CALL `lock_test`(#out);
SELECT #out;
The output will be start-timeout-timeout-timeout-timeout-timeout-end after about 10 seconds of running time (which would be much longer if run without setting the timeout to 1 second.
While probably not too practical (or advisable) in most projects, could potentially be useful when debugging timeout issues when running a query from inside another query - I hope it might help someone else in the future.

Related

mysql: retry transaction specified number of times

I have an sql script which deletes large number of records from database. Script deletes records by small chunks:
DROP PROCEDURE IF EXISTS `my_proc`;
DELIMITER $$
CREATE PROCEDURE my_proc()
BEGIN
DECLARE chunk_size INT DEFAULT 1000;
main: LOOP
SET #count= -1;
START TRANSACTION;
DELETE FROM my_table WHERE .... LIMIT chunk_size;
SELECT ROW_COUNT() into #count;
COMMIT;
IF #count = 0 THEN
LEAVE main;
END IF;
DO SLEEP(0.1);
END LOOP main;
END$$
DELIMITER ;
CALL my_proc();
DROP PROCEDURE IF EXISTS `my_proc`;
The problem is that DELETE statement may fail, so I want retry deletion of current chunk 10 times. So I want the script to fail only if the DELETE statement fails 10 times for current chunk.
I try to use DECLARE CONTINUE HANDLER FOR 1213 BEGIN ..... END; but do not quite understand how to make it work. The main problem now is to make the script fail with meaningful (preferably original) error message if the DELETE failed 10 times.

Handling unfound data in mySQL procedure loop

I think I'm narrowing in on my issue. I have a loop that is only firing once:
DELIMITER $$
DROP PROCEDURE IF EXISTS `thread_updates` $$
CREATE PROCEDURE `thread_updates`()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE my_curr_id INT DEFAULT NULL;
-- DECLARE other vars
DECLARE fixer_cursor CURSOR FOR
SELECT DISTINCT(id)
FROM log
WHERE date >= '2018-01-01';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN fixer_cursor;
REPEAT
FETCH fixer_cursor INTO my_curr_id;
IF NOT done THEN
SELECT data FROM table WHERE id = my_curr_id; -- This may not exist
-- Do other stuff with 'data' or NULL from above
END IF;
UNTIL done END REPEAT;
CLOSE fixer_cursor;
END $$
DELIMITER ;
I think the issue may be that inside the IF NOT done THEN loop, I have a few select statements that may be trying to select results that don't exist (not found).
This is fine (for me) as the logic continues along using NULL values in those spots, but I suspect that my CONTINUE HANDLER FOR NOT FOUND is catching the NOT FOUND warning that these selects throw inside the loop and are thus stopping the entire loop prematurely.
How can I listen for NOT FOUND warning on my cursor only?
Or, how can I suppress the NOT FOUND warning in my MAYBE FOUND select statements inside my loop so my loop continues?
I think I have solved the issue by implementing a counter in a loop rather than relying on the NOT FOUND handler:
DELIMITER $$
DROP PROCEDURE IF EXISTS `thread_updates` $$
CREATE PROCEDURE `thread_updates`()
BEGIN
DECLARE my_total INT DEFAULT NULL; -- Declare total
DECLARE my_counter INT DEFAULT 0; -- Declare counter starting at 0
DECLARE my_curr_id INT DEFAULT NULL;
-- DECLARE other vars
DECLARE fixer_cursor CURSOR FOR
SELECT DISTINCT(id)
FROM log
WHERE date >= '2018-01-01';
OPEN fixer_cursor;
SELECT FOUND_ROWS() INTO my_total; -- Get total number of rows
my_fixerloop: LOOP
FETCH fixer_cursor INTO my_curr_id;
IF my_counter >= my_total THEN -- Compare counter to total
CLOSE fixer_cursor;
LEAVE my_fixerloop;
END IF;
SET my_counter = my_counter + 1; -- Increment by one for each record
SELECT data FROM table WHERE id = my_curr_id; -- This may not exist
-- Do other stuff with 'data' or NULL from above
END LOOP;
END $$
DELIMITER ;

commit and rollback for specified cases

can we put a check (sort of) to-
Check if data is committed to database only when the operation is successfully completed.
Data should be rolled back in case of failed transactions.
like this one-
DELIMITER $$
CREATE PROCEDURE `sp_fail`()
BEGIN
DECLARE `_rollback` BOOL DEFAULT 0;
DECLARE CONTINUE HANDLER FOR SQLEXCEPTION SET `_rollback` = 1;
START TRANSACTION;
UPDATE customer SET age= 20 WHERE name='stark';
UPDATE customer SET age= 20 WHERE name='brian'; -- fail as there is no name as brian in the table customer
IF `_rollback` THEN
SELECT 'The transaction has failed' AS 'Result';
ROLLBACK;
ELSE
SELECT 'The transaction was successful' AS 'Result';
COMMIT;
END IF;
END$$
DELIMITER ;
Its mine modified version of one of the answers in stackoverflow itself.I was thinking of using this to solve the above mentioned points but when i call the procedure the call runs successfully but no rows are affected.Why is that ?
Thanks in advance.
There is Autocommit properties in mysql. You will need set AUTOCOMMIT=0 (this set the autocommit false), after that you can issue COMMIT or ROLLBACK at the end of query according to your condition.
1. INSERT INTO table_name ...;
2. COMMIT; -- confirm your changes
OR
3. ROLLBACK; -- undo your changes

Why does MySQL procedure stops executing upon running into an error? How to I allow it to continue?

I have a stored procedure that does many queries. And what I am trying to do is log any error "if any occurred" into a new table I have created then move on to the next query.
Here is a portion of the procedure
DELIMITER $$
CREATE DEFINER=`root`#`10.%` PROCEDURE `prod_create_new_tasks`()
MAIN:
BEGIN
SET #trys = 0;
loop_label: LOOP
SET #trys := #trys+1, #p1 = '', #p2 = '';
DROP TEMPORARY TABLE IF EXISTS su;
CREATE TEMPORARY TABLE su (KEY(user_id)) ENGINE=MEMORY AS
SELECT user_id, fullname, current_team_id, status
FROM mydb.view_users;
SET #total_errors = ##error_count;
GET DIAGNOSTICS CONDITION 1
#p1 = RETURNED_SQLSTATE, #p2 = MESSAGE_TEXT;
IF(#total_errors > 0 OR #trys > 10) THEN
-- log error
INSERT INTO mydb.error_logs(error_no, error_text, procedure_name, stage)
VALUES(#p1, #p2, current_procedure_name, 'loop_label' );
-- if there are too many tries to create the temporary table and it keeps failing Quit!!
IF( #trys > 10) THEN
LEAVE MAIN;
ELSE
-- sleep for 10 seconds and go back to create the table again
SELECT SLEEP(10);
ITERATE loop_label;
END IF;
END IF;
LEAVE loop_label;
END LOOP;
-- set MySQL Safe mode ON on update
SET SQL_SAFE_UPDATES = 1;
END
Following the logic there the code should try to create a temporary table. If there was any error while trying to create the temporary table GET DIAGNOSTICS should capture the error and store it into error_logs table. After 10 seconds, another attempt to create the table should take.
What seems to be happening is as soon the first attempt to create the table fails it stops and write the error to the screen. no error are logged into error_logs table and no other attempt takes place.
It seems that there is a variables that need to be changed for the season to allow the procedure to continue even if it encounter an error.
My question is How can I log the error "if any" into the table and allow the loop to jump to the next run without stopping?
Note, I tried to change the view_users table to something that does not exists on purpose so I can get the error "table does not exists" logged into the table but that did not work.
I finally figured out the solution.
I would like to post the answer to help out anyone that is running into the same issue.
The solution is to DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
Continue Handler will allow the code to run even if there was an error. from there I set the sqlstate and errorno then I store them and move on.
Here is my final code.
DELIMITER $$
CREATE DEFINER=`root`#`10.%` PROCEDURE `prod_create_new_tasksddd`()
MAIN:
BEGIN
DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
BEGIN
GET DIAGNOSTICS CONDITION 1
#err_sqlstate = RETURNED_SQLSTATE,
#err_message = MESSAGE_TEXT,
#err_num = MYSQL_ERRNO;
SET #hasError = TRUE;
END;
SET #current_procedure_name = 'taskddd';
SET #trys = 0;
SET #hasError = FALSE;
loop_label: LOOP
SET #trys := #trys+1;
DROP TEMPORARY TABLE IF EXISTS su;
SET #hasError = FALSE;
-- trigger error 1146
CREATE TEMPORARY TABLE su LIKE t2;
IF(#hasError OR #trys > 10) THEN
IF #trys > 10 THEN
SET #msg = 'Gave Up!';
ELSE
SET #msg = CONCAT('Trying Again ', #trys);
END IF;
INSERT INTO mydb.error_logs(error_no, error_sqlstate, error_text, procedure_name, stage, trys, current_actions) VALUES(#err_num, #err_sqlstate, #err_message, #current_procedure_name, 'loop_label', #trys , #msg);
-- if there are too many tries to create the temporary table and it keeps failing Quit!!
IF( #trys > 10) THEN
LEAVE MAIN;
ELSE
-- sleep for 2 seconds and go back to create the table again
SET #err_sqlstate = '', #err_message = '', #err_num = '', #msg = NULL;
SET #hasError = FALSE;
SELECT SLEEP(2);
ITERATE loop_label;
END IF;
END IF;
LEAVE loop_label;
END LOOP;
END

How to troubleshoot MySQL procedure - Why loop is reaching max runs?

I have MySQL stored procedure that will create temporary table from remote server using FEDERATED tables. Then the procedure used these temporary table to do some work.
I am seeing some issue where I get an error like
Unable to connect to foreign data source
Got timeout reading communication packets
Unable to connect to foreign data source
Got error 1430 from storage engine
The problem here is that if these temporary table failed to create the entire procedure fails because the tables do not exists.
So I thought I can do some checking after the attempt to create temporary table and if the attempt return error then I go back and try it again. to stop an infinite loop I added a condition to quite after 10 tries.
The issue that i am running into is that the script quits and it does not run all the way to the end.
here is a portion of the procedure where the procedure is quitting and not sure why.
DELIMITER $$
CREATE DEFINER=`root`#`10.%` PROCEDURE `act`()
MAIN:
BEGIN
DECLARE current_procedure_name CHAR(60) DEFAULT 'activities';
DECLARE last_run_time DATETIME DEFAULT NULL;
DECLARE current_run_time_start DATETIME DEFAULT NOW();
-- set the SQL mode to ''
SET SQL_MODE = '';
-- set MySQL Safe mode OFF on update
SET SQL_SAFE_UPDATES = 0;
SET #trys = 0;
loop_label: LOOP
SET #trys := #trys+1;
-- db.view_users is a federated table
DROP TEMPORARY TABLE IF EXISTS view_users1, view_users2, view_users3;
CREATE TEMPORARY TABLE view_users1 (KEY(user_id)) ENGINE=MEMORY AS
SELECT user_id, fullname
FROM db.view_users;
IF(##error_count > 0 OR #trys > 10) THEN
-- if there are too many tries to create the temporary table and it keeps failing Quit!!
IF( #trys > 10) THEN
LEAVE MAIN;
ELSE
-- sleep for 10 seconds and go back to create the table again
SELECT SLEEP(10);
ITERATE loop_label;
END IF;
END IF;
CREATE TEMPORARY TABLE view_users2 (KEY(user_id)) ENGINE=MEMORY AS
SELECT * FROM view_users1;
IF(##error_count > 0) THEN
ITERATE loop_label;
END IF;
CREATE TEMPORARY TABLE view_users3 (KEY(user_id)) ENGINE=MEMORY AS
SELECT * FROM view_users1;
IF(##error_count > 0) THEN
ITERATE loop_label;
END IF;
END LOOP;
-- set MySQL Safe mode back ON on update
SET SQL_SAFE_UPDATES = 1;
END
How can I find out why it is quitting? it seem that #trys is reaching 11 and it quits but I don't understand why would it?
I have tried to run this code outside the procedure and the second line returns 0 error;
DROP TEMPORARY TABLE IF EXISTS view_users1, view_users2, view_users3;
CREATE TEMPORARY TABLE view_users1 (KEY(user_id)) ENGINE=MEMORY AS
SELECT user_id, fullname
FROM db.view_users;
SELECT ##error_count;
A second question, is there a better approach for this problem? it is important that this procedure runs all the way.
I finally figure out the cause of the issue.
from the manual 13.6.5.5 LOOP Syntax
The statements within the loop are repeated until the loop is
terminated. Usually, this is accomplished with a LEAVE statement.
Within a stored function, RETURN can also be used, which exits the
function entirely
at the end just before END LOOP; I needed to add LEAVE loop_label to end the LOOP other wise it will continue until #trys reaches 11;