This is a very simple MySQL stored procedure. Cursor "commission" has only 3000 records, but the procedure call takes more than 30 seconds to run. Why is that?
DELIMITER //
DROP PROCEDURE IF EXISTS apply_credit//
CREATE PROCEDURE apply_credit()
BEGIN
DECLARE done tinyint DEFAULT 0;
DECLARE _pk_id INT;
DECLARE _eid, _source VARCHAR(255);
DECLARE _lh_revenue, _acc_revenue, _project_carrier_expense, _carrier_lh, _carrier_acc, _gross_margin, _fsc_revenue, _revenue, _load_count DECIMAL;
DECLARE commission CURSOR FOR
SELECT pk_id, eid, source, lh_revenue, acc_revenue, project_carrier_expense, carrier_lh, carrier_acc, gross_margin, fsc_revenue, revenue, load_count FROM ct_sales_commission;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
DELETE FROM debug;
OPEN commission;
REPEAT
FETCH commission INTO
_pk_id, _eid, _source, _lh_revenue, _acc_revenue, _project_carrier_expense, _carrier_lh, _carrier_acc, _gross_margin, _fsc_revenue, _revenue, _load_count;
INSERT INTO debug VALUES(concat('row ', _pk_id));
UNTIL done = 1 END REPEAT;
CLOSE commission;
END//
DELIMITER ;
CALL apply_credit();
SELECT * FROM debug;
If you select some datas, and insert into another table, you can do this:
INSERT INTO debug
SELECT concat('row ', _pk_id)
FROM ct_sales_commission;
It's faster than using a cursor.
Some minor turning:
Remove all indexes on the table debug.
Replace the DELETE FROM into TRUNCATE TABLE.
Add DELAYED to the insert statement.
INSERT DELAYED INTO ... VALUES(....)
The database is hosted in a data centre very far away from my MySQL client.
Connected to a MySQL client which is closely located with the MySQL server makes execution time almost 60 times faster (it takes less than one second for the procedure to complete).
I suspect that MySQL client CLI has an issue handling a remote data connection like that.
Related
I'm trying to use a call a stored procedure with a set of values that i'm receiving from another query and i would like to know on how i can i call another procedure using the value from a query . here is my code
CREATE DEFINER=`root`#`localhost` PROCEDURE `temp`(IN u_id int)
BEGIN
#below query will give me all the u_id values that i need to use(ex : 2,8,9)
Declare cur cursor for select r_id from temp.usr_rl where u_id in (u_id);
#below i would like to use the u_id values and run the below procedure in a loop for each value in u_id
open cur;
repeat
fetch cur into a;
if not done then
call get_r(a);
end if;
until done end repeat;
close cur;
END
That would cause you huge performance hit since you are using cursor to process every record and calling another procedure. Thus you are actually doubling the effect. Instead, fetch and store all the values in a Temporary Table using CTAS like below and access that temporary table from within your procedure call get_r for whatever further processing you are doing.
create temporary table myTemp
as select r_id from temp.usr_rl where u_id = u_id;
I am trying to write a stored procedure in MySQL which will perform a somewhat simple select query, and then loop over the results in order to decide whether to perform additional queries, data transformations, or discard the data altogether. Effectively, I want to implement this:
$result = mysql_query("SELECT something FROM somewhere WHERE some stuff");
while ($row = mysql_fetch_assoc($result)) {
// check values of certain fields, decide to perform more queries, or not
// tack it all into the returning result set
}
Only, I want it only in MySQL, so it can be called as a procedure. I know that for triggers, there is the FOR EACH ROW ... syntax, but I can't find mention of anything like this for use outside of the CREATE TRIGGER ... syntax. I have read through some of the looping mechanisms in MySQL, but so far all I can imagine is that I would be implementing something like this:
SET #S = 1;
LOOP
SELECT * FROM somewhere WHERE some_conditions LIMIT #S, 1
-- IF NO RESULTS THEN
LEAVE
-- DO SOMETHING
SET #S = #S + 1;
END LOOP
Although even this is somewhat hazy in my mind.
For reference, though I don't think it's necessarily relevant, the initial query will be joining four tables together to form a model of hierarchal permissions, and then based on how high up the chain a specific permission is, it will retrieve additional information about the children to which that permission should be inherited.
Something like this should do the trick (However, read after the snippet for more info)
CREATE PROCEDURE GetFilteredData()
BEGIN
DECLARE bDone INT;
DECLARE var1 CHAR(16); -- or approriate type
DECLARE var2 INT;
DECLARE var3 VARCHAR(50);
DECLARE curs CURSOR FOR SELECT something FROM somewhere WHERE some stuff;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET bDone = 1;
DROP TEMPORARY TABLE IF EXISTS tblResults;
CREATE TEMPORARY TABLE IF NOT EXISTS tblResults (
--Fld1 type,
--Fld2 type,
--...
);
OPEN curs;
SET bDone = 0;
REPEAT
FETCH curs INTO var1, var2, var3;
IF whatever_filtering_desired
-- here for whatever_transformation_may_be_desired
INSERT INTO tblResults VALUES (var1, var2, var3);
END IF;
UNTIL bDone END REPEAT;
CLOSE curs;
SELECT * FROM tblResults;
END
A few things to consider...
Concerning the snippet above:
may want to pass part of the query to the Stored Procedure, maybe particularly the search criteria, to make it more generic.
If this method is to be called by multiple sessions etc. may want to pass a Session ID of sort to create a unique temporary table name (actually unnecessary concern since different sessions do not share the same temporary file namespace; see comment by Gruber, below)
A few parts such as the variable declarations, the SELECT query etc. need to be properly specified
More generally: trying to avoid needing a cursor.
I purposely named the cursor variable curs[e], because cursors are a mixed blessing. They can help us implement complicated business rules that may be difficult to express in the declarative form of SQL, but it then brings us to use the procedural (imperative) form of SQL, which is a general feature of SQL which is neither very friendly/expressive, programming-wise, and often less efficient performance-wise.
Maybe you can look into expressing the transformation and filtering desired in the context of a "plain" (declarative) SQL query.
Use cursors.
A cursor can be thought of like a buffered reader, when reading through a document. If you think of each row as a line in a document, then you would read the next line, perform your operations, and then advance the cursor.
Using a cursor within a stored procedure.
Prepare the SQL Query
SELECT id FROM employee where department_id = 1;
Create the cursor which will hold the result set returned by the SQL Query.
DECLARE BonusDistributionCursor CURSOR FOR SELECT id FROM employee where department_id = 1;
To have a safe exit when fetching a row from cursor does not return any result then declare a handler called NOT FOUND and set value to a declared variable
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
Open the Cursor before you can fetch the next row from the cursor.
OPEN BonusDistributionCursor;
Fetch the next row pointed by the cursor and move the cursor to next row after that.
FETCH BonusDistributionCursor INTO employeeId;
Run the desired business logic according to the usecase required.
DELIMITER $$
CREATE PROCEDURE distributeYearlyBonus (IN departmentId VARCHAR(2))
BEGIN
DECLARE finished INTEGER DEFAULT 0;
DECLARE empId VARCHAR(TEXT) DEFAULT "";
DECLARE BonusDistributionCursor CURSOR FOR SELECT id FROM employee where department_id = departmentId;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN BonusDistributionCursor;
addBonus: LOOP
FETCH BonusDistributionCursor INTO empId;
IF finished = 1 THEN
LEAVE addBonus;
END IF;
INSERT INTO `bonus_paid_details` (`employee_id`, `year`, `datetime`) VALUES (empId, YEAR(CURDATE());, now());
END LOOP addBonus;
CLOSE BonusDistributionCursor;
END$$
DELIMITER ;
Execute the above script and you will find a new Stored Procedure created.
Call or Invoke the Stored Procedure by inputing the departmentId which will receive the bonus amount.
CALL BonusDistributionCursor(1);
Hope this explains "How to iterate using Cursors used within Stored Procedure"
So I have 2 tables, communication,and movement.
communication has columns fromID,timestamp that has ID of caller, and time the call was made. Then I have another table movement that has ID,timestamp,x,y, that has the ID of a person, their location (x,y), and the time that they are at that location.
I want to write a query that looks something like this:
For every single row of communication(R)
SELECT * FROM movement m
WHERE m.ID = R.fromID && m.timestamp <= R.timestamp
ORDER BY timestamp
Basically, what this is doing is finding the closest movement timestamp for a given communication timestamp. After that, eventually, I want to find the location (x,y) of a call, based on the movement data.
How would I do this? I know there's a set based approach, but I don't want to do it that way. I looked into cursors, but I get the feeling that the performance is terrible on that.
So is there anyway to do this with a loop? I essentially want to loop through every single row of the communication, and get the result.
I tried something like this:
DELMITER $$
CREATE PROCEDURE findClosestTimestamp()
BEGIN
DECLARE commRowCount DEFAULT 0;
DECLARE i DEFAULT 0;
DECLARE ctimestamp DEFAULT 0;
SELECT COUNT(*) FROM communication INTO commRowCount;
SET i = 0;
WHILE i < commRowCount DO
SELECT timestamp INTO ctimestamp FROM communication c
SELECT * FROM movement m
WHERE m.vID = c.fromID && m.timestamp <= R.timestamp
END$$
DELIMITER ;
But I know that's completely wrong.
Is the only way to do this cursors? I just can't find an example of this anywhere on the internet, and I'm completely new to procedures in SQL.
Any guidance would be greatly appreciated, thank you!!
Let's see if I can point you in the right direction using cursors:
delimiter $$
create procedure findClosestTimeStamp()
begin
-- Variables to hold values from the communications table
declare cFromId int;
declare cTimeStamp datetime;
-- Variables related to cursor:
-- 1. 'done' will be used to check if all the rows in the cursor
-- have been read
-- 2. 'curComm' will be the cursor: it will fetch each row
-- 3. The 'continue' handler will update the 'done' variable
declare done int default false;
declare curComm cursor for
select fromId, timestamp from communication; -- This is the query used by the cursor.
declare continue handler for not found -- This handler will be executed if no row is found in the cursor (for example, if all rows have been read).
set done = true;
-- Open the cursor: This will put the cursor on the first row of its
-- rowset.
open curComm;
-- Begin the loop (that 'loop_comm' is a label for the loop)
loop_comm: loop
-- When you fetch a row from the cursor, the data from the current
-- row is read into the variables, and the cursor advances to the
-- next row. If there's no next row, the 'continue handler for not found'
-- will set the 'done' variable to 'TRUE'
fetch curComm into cFromId, cTimeStamp;
-- Exit the loop if you're done
if done then
leave loop_comm;
end if;
-- Execute your desired query.
-- As an example, I'm putting a SELECT statement, but it may be
-- anything.
select *
from movement as m
where m.vID = cFromId and m.timeStamp <= cTimeStamp
order by timestampdiff(SECOND, cTimeStamp, m.timeStamp)
limit 1;
end loop;
-- Don't forget to close the cursor when you finish
close curComm;
end $$
delimiter ;
References:
MySQL Reference: Cursors
MySQL Reference: Date and time functions - timestampdiff()
I create a Mysql procedure using cursor, but it's run too slow... It's get between 40 and 60 lines by second.. See:
DELIMITER $$
CREATE PROCEDURE sp_create(IN v_idsorteio INT,OUT afetados INT)
BEGIN
DECLARE done INT default 0;
DECLARE vc_idsocio INT;
DECLARE z INT;
DECLARE cur1 CURSOR FOR select IdSocio from socios where Sorteio=1 and Finalizado='S' and CodClientes IS NOT NULL;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done=1;
SET z=1;
OPEN cur1;
FETCH cur1 INTO vc_idsocio;
WHILE done=0 DO
-- SELECT register as t;
insert INTO socios_numeros_sorteio (IdSocio,IdSorteio,NumerodeSorteio) VALUES (vc_idsocio,v_idsorteio,z);
FETCH cur1 INTO vc_idsocio;
SET z = z+1;
END WHILE;
CLOSE cur1;
Select z-1 as total INTO afetados;
END$$
DELIMITER ;
how can I to improve that?
This is slow because you are looping through a resultset, row by row, and performing individual insert statements for each row returned. That's why it's gonna be slow.
Let's briefly summarize what you are doing. First, you are running a query:
select IdSocio
from socios
where Sorteio=1
and Finalizado='S'
and CodClientes IS NOT NULL;
(Apparently the order these rows are returned in is not important.)
Then for each row returned from that query, you want to insert a row into another table.
insert INTO socios_numeros_sorteio
(IdSocio
,IdSorteio
,NumerodeSorteio
) VALUES
(vc_idsocio
,v_idsorteio
,z);
The value for the first column is coming from a value returned by the query.
The value for the second column is being assigned a value passed as an argument to the procedure.
And the value for the third column is from a counter that starts at 1 and is being incremented by 1 for each row.
MySQL is optimized to perform an operation like this. But it's NOT optimized to do this using a stored procedure that loops through a cursor row by row.
If you are looking to get some reasonable performance, you need to SIGNIFICANTLY REDUCE the number of individual INSERT statements you run, and instead think in terms of processing data in "sets" rather than individual rows. One approach is batch the rows up into "extended insert" statements, which can insert multiple rows at a time. (The number rows you can insert in one statement is effectively limited by max_allowed_packet.)
That approach will significantly improve performance, but it doesn't avoid the overhead of the cursor, fetching each row into procedure variables.
Something like this (in the body of your procedure) is likely to perform much, much better, because it takes the result set from your select and inserts all of the rows into the destination table in one fell swoop, without bothering to mess with updating the values of variables in the procedure.
BEGIN
SET #idsorteio = v_idsorteio;
INSERT INTO socios_numeros_sorteio
( IdSocio
, IdSorteio
, NumerodeSorteio
)
SELECT s.IdSocio AS IdSocio
, #idsorteio AS IdSorteio
, #z := #z+1 AS NumerodeSorteio
FROM socios s
JOIN (SELECT #z := 0) z
WHERE s.Sorteio=1
AND s.Finalizado='S'
AND s.CodClientes IS NOT NULL;
SELECT ROW_NUMBER() INTO afetados;
END$$
Another simple solution is only to change the engine of the table to MyISAM by running the below query,
ALTER TABLE `socios_numeros_sorteio`
ENGINE=MyISAM;
Then CALL the procedure again.
note: MyISAM make the insertion process very fast
I need to perform the same procedure for several tables in my DB. The poblem is that that procedure contains the following line:
DECLARE tableIt CURSOR FOR select id from table where column=inputParam ;
table is the table the procedure works with. And I can't find a way to make that table name to be dynamic, i.e. to read it from an input parameter.
Right now I have 8 different procedures (one for each table) which differentiate from each other only by one word (the name of the table).
That is really a pain since I have to make every change to the procedure 8 times.
Is it possible to parameterize the select statement for the CURSOR so I can have only one procedure??
Dynamic cursors does not seem to be supported in Mysql.
http://dev.mysql.com/doc/refman/5.1/en/cursors.html
You can work around it
http://forums.mysql.com/read.php?98,133197,149099#msg-149099
"DROP VIEW IF EXISTS v1;
SET #stmt_text=CONCAT("CREATE VIEW v1 AS SELECT c_text FROM ", t_name);
PREPARE stmt FROM #stmt_text;
EXECUTE stmt;
BEGIN
DECLARE v_text VARCHAR(45);
DECLARE done INT DEFAULT 0;
DECLARE c cursor FOR SELECT c_text FROM v1;"