How to troubleshoot MySQL procedure - Why loop is reaching max runs? - mysql

I have MySQL stored procedure that will create temporary table from remote server using FEDERATED tables. Then the procedure used these temporary table to do some work.
I am seeing some issue where I get an error like
Unable to connect to foreign data source
Got timeout reading communication packets
Unable to connect to foreign data source
Got error 1430 from storage engine
The problem here is that if these temporary table failed to create the entire procedure fails because the tables do not exists.
So I thought I can do some checking after the attempt to create temporary table and if the attempt return error then I go back and try it again. to stop an infinite loop I added a condition to quite after 10 tries.
The issue that i am running into is that the script quits and it does not run all the way to the end.
here is a portion of the procedure where the procedure is quitting and not sure why.
DELIMITER $$
CREATE DEFINER=`root`#`10.%` PROCEDURE `act`()
MAIN:
BEGIN
DECLARE current_procedure_name CHAR(60) DEFAULT 'activities';
DECLARE last_run_time DATETIME DEFAULT NULL;
DECLARE current_run_time_start DATETIME DEFAULT NOW();
-- set the SQL mode to ''
SET SQL_MODE = '';
-- set MySQL Safe mode OFF on update
SET SQL_SAFE_UPDATES = 0;
SET #trys = 0;
loop_label: LOOP
SET #trys := #trys+1;
-- db.view_users is a federated table
DROP TEMPORARY TABLE IF EXISTS view_users1, view_users2, view_users3;
CREATE TEMPORARY TABLE view_users1 (KEY(user_id)) ENGINE=MEMORY AS
SELECT user_id, fullname
FROM db.view_users;
IF(##error_count > 0 OR #trys > 10) THEN
-- if there are too many tries to create the temporary table and it keeps failing Quit!!
IF( #trys > 10) THEN
LEAVE MAIN;
ELSE
-- sleep for 10 seconds and go back to create the table again
SELECT SLEEP(10);
ITERATE loop_label;
END IF;
END IF;
CREATE TEMPORARY TABLE view_users2 (KEY(user_id)) ENGINE=MEMORY AS
SELECT * FROM view_users1;
IF(##error_count > 0) THEN
ITERATE loop_label;
END IF;
CREATE TEMPORARY TABLE view_users3 (KEY(user_id)) ENGINE=MEMORY AS
SELECT * FROM view_users1;
IF(##error_count > 0) THEN
ITERATE loop_label;
END IF;
END LOOP;
-- set MySQL Safe mode back ON on update
SET SQL_SAFE_UPDATES = 1;
END
How can I find out why it is quitting? it seem that #trys is reaching 11 and it quits but I don't understand why would it?
I have tried to run this code outside the procedure and the second line returns 0 error;
DROP TEMPORARY TABLE IF EXISTS view_users1, view_users2, view_users3;
CREATE TEMPORARY TABLE view_users1 (KEY(user_id)) ENGINE=MEMORY AS
SELECT user_id, fullname
FROM db.view_users;
SELECT ##error_count;
A second question, is there a better approach for this problem? it is important that this procedure runs all the way.

I finally figure out the cause of the issue.
from the manual 13.6.5.5 LOOP Syntax
The statements within the loop are repeated until the loop is
terminated. Usually, this is accomplished with a LEAVE statement.
Within a stored function, RETURN can also be used, which exits the
function entirely
at the end just before END LOOP; I needed to add LEAVE loop_label to end the LOOP other wise it will continue until #trys reaches 11;

Related

mysql: retry transaction specified number of times

I have an sql script which deletes large number of records from database. Script deletes records by small chunks:
DROP PROCEDURE IF EXISTS `my_proc`;
DELIMITER $$
CREATE PROCEDURE my_proc()
BEGIN
DECLARE chunk_size INT DEFAULT 1000;
main: LOOP
SET #count= -1;
START TRANSACTION;
DELETE FROM my_table WHERE .... LIMIT chunk_size;
SELECT ROW_COUNT() into #count;
COMMIT;
IF #count = 0 THEN
LEAVE main;
END IF;
DO SLEEP(0.1);
END LOOP main;
END$$
DELIMITER ;
CALL my_proc();
DROP PROCEDURE IF EXISTS `my_proc`;
The problem is that DELETE statement may fail, so I want retry deletion of current chunk 10 times. So I want the script to fail only if the DELETE statement fails 10 times for current chunk.
I try to use DECLARE CONTINUE HANDLER FOR 1213 BEGIN ..... END; but do not quite understand how to make it work. The main problem now is to make the script fail with meaningful (preferably original) error message if the DELETE failed 10 times.

Why does MySQL procedure stops executing upon running into an error? How to I allow it to continue?

I have a stored procedure that does many queries. And what I am trying to do is log any error "if any occurred" into a new table I have created then move on to the next query.
Here is a portion of the procedure
DELIMITER $$
CREATE DEFINER=`root`#`10.%` PROCEDURE `prod_create_new_tasks`()
MAIN:
BEGIN
SET #trys = 0;
loop_label: LOOP
SET #trys := #trys+1, #p1 = '', #p2 = '';
DROP TEMPORARY TABLE IF EXISTS su;
CREATE TEMPORARY TABLE su (KEY(user_id)) ENGINE=MEMORY AS
SELECT user_id, fullname, current_team_id, status
FROM mydb.view_users;
SET #total_errors = ##error_count;
GET DIAGNOSTICS CONDITION 1
#p1 = RETURNED_SQLSTATE, #p2 = MESSAGE_TEXT;
IF(#total_errors > 0 OR #trys > 10) THEN
-- log error
INSERT INTO mydb.error_logs(error_no, error_text, procedure_name, stage)
VALUES(#p1, #p2, current_procedure_name, 'loop_label' );
-- if there are too many tries to create the temporary table and it keeps failing Quit!!
IF( #trys > 10) THEN
LEAVE MAIN;
ELSE
-- sleep for 10 seconds and go back to create the table again
SELECT SLEEP(10);
ITERATE loop_label;
END IF;
END IF;
LEAVE loop_label;
END LOOP;
-- set MySQL Safe mode ON on update
SET SQL_SAFE_UPDATES = 1;
END
Following the logic there the code should try to create a temporary table. If there was any error while trying to create the temporary table GET DIAGNOSTICS should capture the error and store it into error_logs table. After 10 seconds, another attempt to create the table should take.
What seems to be happening is as soon the first attempt to create the table fails it stops and write the error to the screen. no error are logged into error_logs table and no other attempt takes place.
It seems that there is a variables that need to be changed for the season to allow the procedure to continue even if it encounter an error.
My question is How can I log the error "if any" into the table and allow the loop to jump to the next run without stopping?
Note, I tried to change the view_users table to something that does not exists on purpose so I can get the error "table does not exists" logged into the table but that did not work.
I finally figured out the solution.
I would like to post the answer to help out anyone that is running into the same issue.
The solution is to DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
Continue Handler will allow the code to run even if there was an error. from there I set the sqlstate and errorno then I store them and move on.
Here is my final code.
DELIMITER $$
CREATE DEFINER=`root`#`10.%` PROCEDURE `prod_create_new_tasksddd`()
MAIN:
BEGIN
DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
BEGIN
GET DIAGNOSTICS CONDITION 1
#err_sqlstate = RETURNED_SQLSTATE,
#err_message = MESSAGE_TEXT,
#err_num = MYSQL_ERRNO;
SET #hasError = TRUE;
END;
SET #current_procedure_name = 'taskddd';
SET #trys = 0;
SET #hasError = FALSE;
loop_label: LOOP
SET #trys := #trys+1;
DROP TEMPORARY TABLE IF EXISTS su;
SET #hasError = FALSE;
-- trigger error 1146
CREATE TEMPORARY TABLE su LIKE t2;
IF(#hasError OR #trys > 10) THEN
IF #trys > 10 THEN
SET #msg = 'Gave Up!';
ELSE
SET #msg = CONCAT('Trying Again ', #trys);
END IF;
INSERT INTO mydb.error_logs(error_no, error_sqlstate, error_text, procedure_name, stage, trys, current_actions) VALUES(#err_num, #err_sqlstate, #err_message, #current_procedure_name, 'loop_label', #trys , #msg);
-- if there are too many tries to create the temporary table and it keeps failing Quit!!
IF( #trys > 10) THEN
LEAVE MAIN;
ELSE
-- sleep for 2 seconds and go back to create the table again
SET #err_sqlstate = '', #err_message = '', #err_num = '', #msg = NULL;
SET #hasError = FALSE;
SELECT SLEEP(2);
ITERATE loop_label;
END IF;
END IF;
LEAVE loop_label;
END LOOP;
END

MySQL - detecting and handling Lock Wait Timeout inside Stored Procedure

The Question
purely for academic reasons, I'm wondering if you could add a handler to a mysql stored procedure that is able to recover from a lock wait timeout error if one of its queries locks up (such as a SELECT ... FOR UPDATE or UPDATE) query.
The Example
This is assuming an innoDB database, set to issolation level Repeatable read, with an empty users table defined.
1. Example Procedure:
DROP PROCEDURE IF EXISTS `lock_test`;
DELIMITER ;;
CREATE PROCEDURE `lock_test`(OUT status ENUM('success','timeout'))
MODIFIES SQL DATA
BEGIN
START TRANSACTION;
SELECT * FROM `users` FOR UPDATE;
SET status := 'success';
COMMIT;
END;;
DELIMITER ;
2. Run code in mysql terminal 1:
START TRANSACTION;
SELECT * FROM `users` FOR UPDATE;
the contents of users will be displayed, but the transaction will remain open.
3. Run code in mysql terminal 2:
CALL `lock_test`(#out);
SELECT #out;
the transaction will run until it times out (default value of innodb_lock_wait_timeout is 50 seconds)
Is it possible to add a handler inside the lock_test() procedure, so that we can have #out hold 'timeout'?
After spending some time reading through the MySQL Handler Documentation I was able to get what I was looking for:
DROP PROCEDURE IF EXISTS `lock_test`;
DELIMITER ;;
CREATE PROCEDURE `lock_test`(OUT status_out VARCHAR(255))
MODIFIES SQL DATA
BEGIN
DECLARE procedure_attempts INT DEFAULT 5;
DECLARE query_timeout INT DEFAULT FALSE;
SET status_out := 'start';
procedure_loop:
REPEAT
BEGIN
DECLARE CONTINUE HANDLER FOR 1205
-- Error: 1205 SQLSTATE: HY000 (ER_LOCK_WAIT_TIMEOUT)
BEGIN
SET query_timeout := TRUE;
SET status_out := CONCAT(status_out,'-timeout');
END;
IF ( procedure_attempts < 1) THEN
LEAVE procedure_loop;
END IF;
START TRANSACTION;
SELECT * FROM `users` FOR UPDATE;
IF (query_timeout) THEN
SET query_timeout := FALSE;
ELSE
SET status_out := CONCAT(status_out,'-success');
SET procedure_attempts := 0;
END IF;
COMMIT;
SET procedure_attempts := procedure_attempts - 1;
END;
UNTIL FALSE END REPEAT;
-- loop
SET status_out := CONCAT(status_out,'-end');
END;;
DELIMITER ;
When run as follows:
SET ##innodb_lock_wait_timeout:=1;
CALL `lock_test`(#out);
SELECT #out;
The output will be start-timeout-timeout-timeout-timeout-timeout-end after about 10 seconds of running time (which would be much longer if run without setting the timeout to 1 second.
While probably not too practical (or advisable) in most projects, could potentially be useful when debugging timeout issues when running a query from inside another query - I hope it might help someone else in the future.

while loop terminating after fetching some data

I have written a procedure that creates a temporary table and executes a query by fetching the rows from the temporary table.I have around 13486 rows in the temporary table.But when i am calling the procedure i observed that the procedure is getting terminated after fetching 107 rows from the temporary table.Moreover i also observed that this value is not constant..Sometimes it is 107 the other time it is 114 and some other time it is just 100.Why this happens?Please need help?Somebody please..Here is my procedure.And i came to know that while loop will terminate for >1000 iterations.Please suggest me a method to overcome this.
DELIMITER $$
DROP PROCEDURE IF EXISTS `lookup`.`test` $$
CREATE PROCEDURE `lookup`.`test` ()
BEGIN
CREATE TEMPORARY TABLE lookup.airportname(id int AUTO_INCREMENT,PRIMARY KEY(id))
AS (select distinct airport_id from lookup.airport);
SET #num=0;
SET #arpt=NULL;
SELECT count(*) INTO #num FROM airportname;
SET #i=0;
while #i<#num do
SELECT airport_id INTO #arpt FROM airportname WHERE id=#i;
select #arpt,#i;
set #i=#i+1;
end while;
END $$
DELIMITER ;
I am using mysql query browser.Thank you.
If you want to insert value into id Then it should not be auto_increment. - Remove auto_increment from table definition, it should work
delimiter |
create procedure employee_select()
Begin
Declare empno,done int(9);
Declare emp_select Cursor for select emp_no from gross_salary ;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
Open emp_select; // cursor opening
read_loop: loop // start looping all the datas one by one
fetch emp_select into empno; // fetching the select value into variable empno
//note :variable name should not be same as columne name in select statement"
IF done THEN
LEAVE read_loop; // if no more rows, this makes it to leave the loop"
END IF;
//Enter the code you want do for each row
end loop;
close emp_select;
End |
delimiter ;

Can I run a loop in MySQL without using a procedure/function?

For testing, is it possible to run a loop from MySQL workbench or similar tool? I tried but got an error.
If it is possible, please supply a simple example I can run.
You can't do a for loop in an SQL editor without a stored procedure. I use TOAD for MySQL.
A quick stored procedure should do the job:
DELIMITER $$
DROP PROCEDURE IF EXISTS proc_loop_test$$
CREATE PROCEDURE proc_loop_test()
BEGIN
DECLARE int_val INT DEFAULT 0;
test_loop : LOOP
IF (int_val = 10) THEN
LEAVE test_loop;
END IF;
SET int_val = int_val +1;
SELECT int_val;
END LOOP;
END$$
DELIMITER ;
There's a trick with limited use-cases that is "loop-like".
I wanted to create a large (1~2 million) row table of random integers for a test:
INSERT INTO test_table (num) VALUES(ROUND(RAND() * 1E6));
-- calling this will insert once for every row in test_table
INSERT INTO test_table (num)
SELECT ROUND(RAND() * 1E6)
FROM test_table;
So I quickly just kept doubling the number of rows until I had what I needed.
Supposed that you already have an arbitrary table myOldTable which is sufficiently long you could use the following trick:
set #counter = 0;
select (#counter := #counter+1), #counter*#counter from myOldTable limit 1000;
If it is that you only want to block the current thread then use select sleep(seconds); otherwise you can use a stored procedure (if there's something you want to loop over) or a UDF (user defined function).