SQL - Looping through ever row of table in MySQL? - mysql

So I have 2 tables, communication,and movement.
communication has columns fromID,timestamp that has ID of caller, and time the call was made. Then I have another table movement that has ID,timestamp,x,y, that has the ID of a person, their location (x,y), and the time that they are at that location.
I want to write a query that looks something like this:
For every single row of communication(R)
SELECT * FROM movement m
WHERE m.ID = R.fromID && m.timestamp <= R.timestamp
ORDER BY timestamp
Basically, what this is doing is finding the closest movement timestamp for a given communication timestamp. After that, eventually, I want to find the location (x,y) of a call, based on the movement data.
How would I do this? I know there's a set based approach, but I don't want to do it that way. I looked into cursors, but I get the feeling that the performance is terrible on that.
So is there anyway to do this with a loop? I essentially want to loop through every single row of the communication, and get the result.
I tried something like this:
DELMITER $$
CREATE PROCEDURE findClosestTimestamp()
BEGIN
DECLARE commRowCount DEFAULT 0;
DECLARE i DEFAULT 0;
DECLARE ctimestamp DEFAULT 0;
SELECT COUNT(*) FROM communication INTO commRowCount;
SET i = 0;
WHILE i < commRowCount DO
SELECT timestamp INTO ctimestamp FROM communication c
SELECT * FROM movement m
WHERE m.vID = c.fromID && m.timestamp <= R.timestamp
END$$
DELIMITER ;
But I know that's completely wrong.
Is the only way to do this cursors? I just can't find an example of this anywhere on the internet, and I'm completely new to procedures in SQL.
Any guidance would be greatly appreciated, thank you!!

Let's see if I can point you in the right direction using cursors:
delimiter $$
create procedure findClosestTimeStamp()
begin
-- Variables to hold values from the communications table
declare cFromId int;
declare cTimeStamp datetime;
-- Variables related to cursor:
-- 1. 'done' will be used to check if all the rows in the cursor
-- have been read
-- 2. 'curComm' will be the cursor: it will fetch each row
-- 3. The 'continue' handler will update the 'done' variable
declare done int default false;
declare curComm cursor for
select fromId, timestamp from communication; -- This is the query used by the cursor.
declare continue handler for not found -- This handler will be executed if no row is found in the cursor (for example, if all rows have been read).
set done = true;
-- Open the cursor: This will put the cursor on the first row of its
-- rowset.
open curComm;
-- Begin the loop (that 'loop_comm' is a label for the loop)
loop_comm: loop
-- When you fetch a row from the cursor, the data from the current
-- row is read into the variables, and the cursor advances to the
-- next row. If there's no next row, the 'continue handler for not found'
-- will set the 'done' variable to 'TRUE'
fetch curComm into cFromId, cTimeStamp;
-- Exit the loop if you're done
if done then
leave loop_comm;
end if;
-- Execute your desired query.
-- As an example, I'm putting a SELECT statement, but it may be
-- anything.
select *
from movement as m
where m.vID = cFromId and m.timeStamp <= cTimeStamp
order by timestampdiff(SECOND, cTimeStamp, m.timeStamp)
limit 1;
end loop;
-- Don't forget to close the cursor when you finish
close curComm;
end $$
delimiter ;
References:
MySQL Reference: Cursors
MySQL Reference: Date and time functions - timestampdiff()

Related

MySQL: Run a stored procedure getting parameter from a query inside another stored procedure

I am new to MySQL.
I am developing a system where many users are assigned to specific tasks. When they are inactive for a certain period of time (lets say more than 10 minutes) I would like the system automatically clear their assignments so that others can work on them.
To achieve that I have created a table called tblactivitytracker for activity tracking. Assignments are in a table called tblinquiries. I have created a stored procedure to get the inactive users.
Here is an sqlfiddle example: Get Inactive Users
In the above example I get 3 inactive users: auditor1, auditor2 and auditor3.
I have created a stored procedure to clear assignment of a single user which does the job perfectly.
CREATE PROCEDURE `spClearAssignedInquiry`(IN `pAssignedTo` VARCHAR(50))
UPDATE
tblinquiries
SET
AuditStatus='Check', AssignedTo=NULL, Result=NULL,
ResultCategories=NULL, AuditBy=NULL,
Remarks=NULL, StartTime=NULL, EndTime=NULL
WHERE
AssignedTo=pAssignedTo AND
AuditStatus='Assigned' AND EndTime IS NULL
If I pass auditor1 as a parameter in the above procedure it will clear the user's assignment.
To pass all inactive users and clear the assignments in a single go I tried the below procedure following this stackoverflow solution:
CREATE PROCEDURE `spInactiveUsers`()
BEGIN
DECLARE done BOOLEAN DEFAULT FALSE;
DECLARE AssignedTo VARCHAR(50);
DECLARE cur CURSOR FOR
SELECT
q1.AssignedTo AS AssignedTo
FROM
(SELECT
InquiryId, AssignedTo
FROM
tblinquiries
WHERE
AuditStatus='Assigned' AND StartTime IS NOT NULL AND EndTime IS NULL
ORDER BY
AssignedTo ASC
) q1
RIGHT JOIN
(SELECT
UserId, MAX(LastActivity) AS LastActivity, ROUND(TIME_TO_SEC(TIMEDIFF(MAX(LastActivity),CURRENT_TIMESTAMP()))/60,0) AS InactiveMinutes
FROM
tblactivitytracker
GROUP BY
UserId
ORDER BY
LastActivity ASC
) q2
ON
q2.UserId=q1.AssignedTo
WHERE
q2.InactiveMinutes>10;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done := TRUE;
OPEN cur;
testLoop: LOOP
FETCH cur INTO AssignedTo;
IF done THEN
LEAVE testLoop;
END IF;
CALL spClearAssignedInquiry(AssignedTo);
END LOOP testLoop;
CLOSE cur;
END
But it does not clear any of the assignments.
I am banging my head to the wall for the last couple of days. Any help would be much appreciated. Thanks in advance.
You are using a variable name that is also the name of a column. The value of the variable will take precedence over the column value, see the documentation:
A local variable should not have the same name as a table column. If an SQL statement, such as a SELECT ... INTO statement, contains a reference to a column and a declared local variable with the same name, MySQL currently interprets the reference as the name of a variable.
So in
...
FROM
(SELECT
InquiryId, AssignedTo
...
you are selecting the variable AssignedTo (which is null), not the column from your table.
Just rename it (in declare and the loop), or, less advised, explicitly state the tablename to set the scope, e.g. SELECT InquiryId, tblinquiries.AssignedTo .... order by tblinquiries.AssignedTo.
There is another (minor) problem is your use of TIMEDIFF in TIMEDIFF(MAX(LastActivity), CURRENT_TIMESTAMP()). It requires the later time in the first argument if you want to get a positive number (as in q2.InactiveMinutes>10).

Iterate through a column to give input to a stored procedure in MySQL [duplicate]

I am trying to write a stored procedure in MySQL which will perform a somewhat simple select query, and then loop over the results in order to decide whether to perform additional queries, data transformations, or discard the data altogether. Effectively, I want to implement this:
$result = mysql_query("SELECT something FROM somewhere WHERE some stuff");
while ($row = mysql_fetch_assoc($result)) {
// check values of certain fields, decide to perform more queries, or not
// tack it all into the returning result set
}
Only, I want it only in MySQL, so it can be called as a procedure. I know that for triggers, there is the FOR EACH ROW ... syntax, but I can't find mention of anything like this for use outside of the CREATE TRIGGER ... syntax. I have read through some of the looping mechanisms in MySQL, but so far all I can imagine is that I would be implementing something like this:
SET #S = 1;
LOOP
SELECT * FROM somewhere WHERE some_conditions LIMIT #S, 1
-- IF NO RESULTS THEN
LEAVE
-- DO SOMETHING
SET #S = #S + 1;
END LOOP
Although even this is somewhat hazy in my mind.
For reference, though I don't think it's necessarily relevant, the initial query will be joining four tables together to form a model of hierarchal permissions, and then based on how high up the chain a specific permission is, it will retrieve additional information about the children to which that permission should be inherited.
Something like this should do the trick (However, read after the snippet for more info)
CREATE PROCEDURE GetFilteredData()
BEGIN
DECLARE bDone INT;
DECLARE var1 CHAR(16); -- or approriate type
DECLARE var2 INT;
DECLARE var3 VARCHAR(50);
DECLARE curs CURSOR FOR SELECT something FROM somewhere WHERE some stuff;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET bDone = 1;
DROP TEMPORARY TABLE IF EXISTS tblResults;
CREATE TEMPORARY TABLE IF NOT EXISTS tblResults (
--Fld1 type,
--Fld2 type,
--...
);
OPEN curs;
SET bDone = 0;
REPEAT
FETCH curs INTO var1, var2, var3;
IF whatever_filtering_desired
-- here for whatever_transformation_may_be_desired
INSERT INTO tblResults VALUES (var1, var2, var3);
END IF;
UNTIL bDone END REPEAT;
CLOSE curs;
SELECT * FROM tblResults;
END
A few things to consider...
Concerning the snippet above:
may want to pass part of the query to the Stored Procedure, maybe particularly the search criteria, to make it more generic.
If this method is to be called by multiple sessions etc. may want to pass a Session ID of sort to create a unique temporary table name (actually unnecessary concern since different sessions do not share the same temporary file namespace; see comment by Gruber, below)
A few parts such as the variable declarations, the SELECT query etc. need to be properly specified
More generally: trying to avoid needing a cursor.
I purposely named the cursor variable curs[e], because cursors are a mixed blessing. They can help us implement complicated business rules that may be difficult to express in the declarative form of SQL, but it then brings us to use the procedural (imperative) form of SQL, which is a general feature of SQL which is neither very friendly/expressive, programming-wise, and often less efficient performance-wise.
Maybe you can look into expressing the transformation and filtering desired in the context of a "plain" (declarative) SQL query.
Use cursors.
A cursor can be thought of like a buffered reader, when reading through a document. If you think of each row as a line in a document, then you would read the next line, perform your operations, and then advance the cursor.
Using a cursor within a stored procedure.
Prepare the SQL Query
SELECT id FROM employee where department_id = 1;
Create the cursor which will hold the result set returned by the SQL Query.
DECLARE BonusDistributionCursor CURSOR FOR SELECT id FROM employee where department_id = 1;
To have a safe exit when fetching a row from cursor does not return any result then declare a handler called NOT FOUND and set value to a declared variable
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
Open the Cursor before you can fetch the next row from the cursor.
OPEN BonusDistributionCursor;
Fetch the next row pointed by the cursor and move the cursor to next row after that.
FETCH BonusDistributionCursor INTO employeeId;
Run the desired business logic according to the usecase required.
DELIMITER $$
CREATE PROCEDURE distributeYearlyBonus (IN departmentId VARCHAR(2))
BEGIN
DECLARE finished INTEGER DEFAULT 0;
DECLARE empId VARCHAR(TEXT) DEFAULT "";
DECLARE BonusDistributionCursor CURSOR FOR SELECT id FROM employee where department_id = departmentId;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN BonusDistributionCursor;
addBonus: LOOP
FETCH BonusDistributionCursor INTO empId;
IF finished = 1 THEN
LEAVE addBonus;
END IF;
INSERT INTO `bonus_paid_details` (`employee_id`, `year`, `datetime`) VALUES (empId, YEAR(CURDATE());, now());
END LOOP addBonus;
CLOSE BonusDistributionCursor;
END$$
DELIMITER ;
Execute the above script and you will find a new Stored Procedure created.
Call or Invoke the Stored Procedure by inputing the departmentId which will receive the bonus amount.
CALL BonusDistributionCursor(1);
Hope this explains "How to iterate using Cursors used within Stored Procedure"

MySQL Result consisted of more than one row on stored procedure

This stored procedure that I'm working on errors out some times. I am getting a Result consisted of more than one row error, but only for certain JOB_ID_INPUT values. I understand what causes this error, and so I have tried to be really careful to make sure that my return values are scalar when they should be. Its tough to see into the stored procedure, so I'm not sure where the error could be generated. Since the error is thrown conditionally, it has me thinking memory could be an issue, or cursor reuse. I don't work with cursors that often so I'm not sure. Thank you to anyone who helps.
DROP PROCEDURE IF EXISTS export_job_candidates;
DELIMITER $$
CREATE PROCEDURE export_job_candidates (IN JOB_ID_INPUT INT(11))
BEGIN
DECLARE candidate_count INT(11) DEFAULT 0;
DECLARE candidate_id INT(11) DEFAULT 0;
# these are the ib variables
DECLARE _overall_score DECIMAL(5, 2) DEFAULT 0.0;
# declare the cursor that will be needed for this SP
DECLARE curs CURSOR FOR SELECT user_id FROM job_application WHERE job_id = JOB_ID_INPUT;
# this table stores all of the data that will be returned from the various tables that will be joined together to build the final export
CREATE TEMPORARY TABLE IF NOT EXISTS candidate_stats_temp_table (
overall_score_ib DECIMAL(5, 2) DEFAULT 0.0
) engine = memory;
SELECT COUNT(job_application.id) INTO candidate_count FROM job_application WHERE job_id = JOB_ID_INPUT;
OPEN curs;
# loop controlling the insert of data into the temp table that is retuned by this function
insert_loop: LOOP
# end the loop if there is no more computation that needs to be done
IF candidate_count = 0 THEN
LEAVE insert_loop;
END IF;
FETCH curs INTO candidate_id;
# get the ib data that may exist for this user
SELECT
tests.overall_score
INTO
_overall_score
FROM
tests
WHERE
user_id = candidate_id;
#build the insert for the table that is being constructed via this loop
INSERT INTO candidate_stats_temp_table (
overall_score
) VALUES (
_overall_score
);
SET candidate_count = candidate_count - 1;
END LOOP;
CLOSE curs;
SELECT * FROM candidate_stats_temp_table WHERE 1;
END $$
DELIMITER ;
The WHERE 1 (as pointed out by #cdonner) definitely doesn't look right, but I'm pretty sure this error is happening because one of your SELECT ... INTO commands is returning more than one row.
This one should be OK because it's an aggregate without a GROUP BY, which always returns one row:
SELECT COUNT(job_application.id) INTO candidate_count
FROM job_application WHERE job_id = JOB_ID_INPUT;
So it's probably this one:
# get the ib data that may exist for this user
SELECT
tests.overall_score
INTO
_overall_score
FROM
tests
WHERE
user_id = candidate_id;
Try to figure out if it's possible for this query to return more than one row, and if so, how do you work around it. One way might be to MAX the overall score:
SELECT MAX(tests.overall_sore) INTO _overall_score
FROM tests
WHERE user_id = candidate_id
I think you want to use
LIMIT 1
in your select, not
WHERE 1
Aside from using this safety net, you should understand your data to figure out why you are getting multiple results. Without seeing the data, it is difficult for me to take a guess.

MySQL Stored Procedure: Can I update directly with the cursor?

It's probably a really easy question, but I couldn't find an answer to it myself:
When I have a cursor pointing to the row I want to update, is there a faster way than to issue an UPDATE statement?
DECLARE current_id, current_product_id, current_price, current_position INT DEFAULT 0;
DECLARE new_position INT DEFAULT 0;
DECLARE cursor_offers CURSOR FOR
SELECT id, product_id, price, position FROM offers
ORDER BY product_id, price ASC;
OPEN cursor_offers;
offers_loop: LOOP
FETCH cursor_offers INTO current_id, current_product_id, current_price, current_position;
# Loop control omitted
# Conditional statements omitted, that calculate new_position
UPDATE offers SET position = new_position WHERE id = current_id; # <--- This line
END LOOP offers_loop;
I figure that as MySQL already has a pointer to that line via the cursor, it would be inefficient to find it again in the UPDATE statement.
Read the documentation. It says that cursors are read-only and can not be used to update.
So I'd say it's not possible to directly update using a cursor. You'll have to issue an update statement.

Procedure mysql with cursor is too slow.. Why?

I create a Mysql procedure using cursor, but it's run too slow... It's get between 40 and 60 lines by second.. See:
DELIMITER $$
CREATE PROCEDURE sp_create(IN v_idsorteio INT,OUT afetados INT)
BEGIN
DECLARE done INT default 0;
DECLARE vc_idsocio INT;
DECLARE z INT;
DECLARE cur1 CURSOR FOR select IdSocio from socios where Sorteio=1 and Finalizado='S' and CodClientes IS NOT NULL;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done=1;
SET z=1;
OPEN cur1;
FETCH cur1 INTO vc_idsocio;
WHILE done=0 DO
-- SELECT register as t;
insert INTO socios_numeros_sorteio (IdSocio,IdSorteio,NumerodeSorteio) VALUES (vc_idsocio,v_idsorteio,z);
FETCH cur1 INTO vc_idsocio;
SET z = z+1;
END WHILE;
CLOSE cur1;
Select z-1 as total INTO afetados;
END$$
DELIMITER ;
how can I to improve that?
This is slow because you are looping through a resultset, row by row, and performing individual insert statements for each row returned. That's why it's gonna be slow.
Let's briefly summarize what you are doing. First, you are running a query:
select IdSocio
from socios
where Sorteio=1
and Finalizado='S'
and CodClientes IS NOT NULL;
(Apparently the order these rows are returned in is not important.)
Then for each row returned from that query, you want to insert a row into another table.
insert INTO socios_numeros_sorteio
(IdSocio
,IdSorteio
,NumerodeSorteio
) VALUES
(vc_idsocio
,v_idsorteio
,z);
The value for the first column is coming from a value returned by the query.
The value for the second column is being assigned a value passed as an argument to the procedure.
And the value for the third column is from a counter that starts at 1 and is being incremented by 1 for each row.
MySQL is optimized to perform an operation like this. But it's NOT optimized to do this using a stored procedure that loops through a cursor row by row.
If you are looking to get some reasonable performance, you need to SIGNIFICANTLY REDUCE the number of individual INSERT statements you run, and instead think in terms of processing data in "sets" rather than individual rows. One approach is batch the rows up into "extended insert" statements, which can insert multiple rows at a time. (The number rows you can insert in one statement is effectively limited by max_allowed_packet.)
That approach will significantly improve performance, but it doesn't avoid the overhead of the cursor, fetching each row into procedure variables.
Something like this (in the body of your procedure) is likely to perform much, much better, because it takes the result set from your select and inserts all of the rows into the destination table in one fell swoop, without bothering to mess with updating the values of variables in the procedure.
BEGIN
SET #idsorteio = v_idsorteio;
INSERT INTO socios_numeros_sorteio
( IdSocio
, IdSorteio
, NumerodeSorteio
)
SELECT s.IdSocio AS IdSocio
, #idsorteio AS IdSorteio
, #z := #z+1 AS NumerodeSorteio
FROM socios s
JOIN (SELECT #z := 0) z
WHERE s.Sorteio=1
AND s.Finalizado='S'
AND s.CodClientes IS NOT NULL;
SELECT ROW_NUMBER() INTO afetados;
END$$
Another simple solution is only to change the engine of the table to MyISAM by running the below query,
ALTER TABLE `socios_numeros_sorteio`
ENGINE=MyISAM;
Then CALL the procedure again.
note: MyISAM make the insertion process very fast