I have a problem fetching values from a MySQL cursor.
I create a temporary table which is a mere copy of another table (the original table has a variable name, that's passed as the procedure's parameter, and because MySQL doesn't support variable table names, I have to create a copy - can't use the original directly).
The temporary table creation goes just fine, all data that are supposed to be in it are there. Then I define a cursor to iterate through my temporary table... but when I try to fetch from the cursor in a while loop my variables are not filled with data from the "cursored" table... most of them are just NULL, only last 2 seem to have correct values inside.
Here is the chunk of my code:
-- variables to be filled from the cursor
DECLARE id,rain,snow,hs,clouds,rain2,cape,status,done int;
DECLARE v_v,v_u double;
-- cursor declaration
DECLARE iter CURSOR FOR (SELECT id,cape,rain,snow,hstones,clouds,raining,wind_u,wind_v FROM temp_tbl);
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
-- drop the old temporary table if any
DROP TABLE IF EXISTS temp_tbl;
-- a statement to create the temporary table from a table with the specified name
-- (table_name is a parameter of the stored procedure this chunk is from)
SET #temp_table = CONCAT('CREATE TABLE temp_tbl AS SELECT * FROM ', table_name, ' WHERE 1');
-- prepare, execute and deallocate the statement
PREPARE ctmp FROM #temp_table;
EXECUTE ctmp;
DEALLOCATE PREPARE ctmp;
-- now the temp_table exists, open the cursor
OPEN iter;
WHILE NOT done DO
-- fetch the values
FETCH iter INTO id,cape,rain,snow,hs,clouds,rain2,v_u,v_v;
-- fetch doesnt work, only v_u and v_v variables are fetched correctly, others are null
-- ... further statements go here
END WHILE;
CLOSE iter;
Is there any type-checking within the FETCH statement that might cause such problem? The columns in my temporary table (which is derived from the original one) are just small-ints or tiny-ints, so these should be perfectly compatible with ints I use in the fetch statement. Those two last ones are doubles, but weird that only these two doubles are fetched. Even the ID int column, which is the primary key isn't fetched.
I work with the dbForge Studio to step into and debug my procedures, but that shouldn't be the problem.
In MySQL functions, when the parameter or variable names conflict with the field names, the parameter or variable names are used.
In these statements:
DECLARE id,rain,snow,hs,clouds,rain2,cape,status,done int;
DECLARE iter CURSOR FOR (SELECT id,cape,rain,snow,hstones,clouds,raining,wind_u,wind_v FROM temp_tbl);
you select the uninitialized variables, not the fields. It is the same as doing:
DECLARE iter CURSOR FOR (SELECT NULL, NULL, NULL, NULL, NULL, NULL, NULL, wind_u,wind_v FROM temp_tbl);
The last two field name do not conflict and are selected correctly.
Prepend the variable names with an underscore:
DECLARE _id, _rain, _snow, _hs, _clouds, _rain2, _cape, _status, _done INT;
Related
I'm trying to write a MySQL stored proceedure that loops through all existing tables in my database and creates a copy/clone of each table. I'm using a cursor to loop through the table names then create a new table like this:
DELIMITER //
CREATE PROCEDURE CopyTables()
BEGIN
DECLARE finished INT DEFAULT 0;
DECLARE tableName VARCHAR(100);
DECLARE copyTableName VARCHAR(100);
DECLARE curTables
CURSOR FOR
SELECT table_name FROM INFORMATION_SCHEMA.TABLES;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN curTables;
create_loop: LOOP
FETCH curTables INTO tableName;
IF finished THEN LEAVE create_loop; END IF;
SELECT concat('Processing table ', tableName);
SET copyTableName = CONCAT('copy_',tableName);
SELECT concat('Creating table ', copyTableName);
CREATE TABLE copyTableName LIKE tableName;
END LOOP;
CLOSE curTables;
END //
DELIMITER;
But I get an error when calling the stored procedure:
> call CopyTables()
[2020-12-08 18:16:03] 1 row retrieved starting from 1 in 77 ms (execution: 15 ms, fetching: 62 ms)
[2020-12-08 18:16:03] [S1000] Attempt to close streaming result set com.mysql.cj.protocol.a.result.ResultsetRowsStreaming#7a714591 that was not registered. Only one streaming result set may be open and in use per-connection. Ensure that you have called .close() on any active result sets before attempting more queries.
Is the result set exception effectively complaining because I'm creating new tables which is effectively messing with the cursor/select? I've got additional table changes on both the original and copied table to perform, like adding new columns, creating triggers, modifying constraints.
The list of table names is not static, and this should be able to run on whatever database I need it.
Can you suggest another way to achieve this without the cursor perhaps?
The problem is that the procedure is returning multiple result sets, but your Java client is not handling that correctly.
Refer to How do you get multiple resultset from a single CallableStatement?
Another problem with your procedure is that you aren't creating tables the way you think you are.
This statement:
CREATE TABLE copyTableName LIKE tableName;
will only create a table named literally copyTableName that is like another table that is literally tableName. It will NOT use the values of variables by those names.
To do what you want, you need to use a prepared statement:
SET #sql = CONCAT('CREATE TABLE `', copyTableName, '` LIKE `', tableName, '`');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
This way the value of your variables is concatenated into an SQL statement.
Note that PREPARE only accepts a user-defined session variable, the type with the # sigil. It doesn't work with local variables you create in your procedure with DECLARE. Read https://dev.mysql.com/doc/refman/8.0/en/prepare.html and https://dev.mysql.com/doc/refman/8.0/en/user-variables.html
I am trying to write a stored procedure in MySQL which will perform a somewhat simple select query, and then loop over the results in order to decide whether to perform additional queries, data transformations, or discard the data altogether. Effectively, I want to implement this:
$result = mysql_query("SELECT something FROM somewhere WHERE some stuff");
while ($row = mysql_fetch_assoc($result)) {
// check values of certain fields, decide to perform more queries, or not
// tack it all into the returning result set
}
Only, I want it only in MySQL, so it can be called as a procedure. I know that for triggers, there is the FOR EACH ROW ... syntax, but I can't find mention of anything like this for use outside of the CREATE TRIGGER ... syntax. I have read through some of the looping mechanisms in MySQL, but so far all I can imagine is that I would be implementing something like this:
SET #S = 1;
LOOP
SELECT * FROM somewhere WHERE some_conditions LIMIT #S, 1
-- IF NO RESULTS THEN
LEAVE
-- DO SOMETHING
SET #S = #S + 1;
END LOOP
Although even this is somewhat hazy in my mind.
For reference, though I don't think it's necessarily relevant, the initial query will be joining four tables together to form a model of hierarchal permissions, and then based on how high up the chain a specific permission is, it will retrieve additional information about the children to which that permission should be inherited.
Something like this should do the trick (However, read after the snippet for more info)
CREATE PROCEDURE GetFilteredData()
BEGIN
DECLARE bDone INT;
DECLARE var1 CHAR(16); -- or approriate type
DECLARE var2 INT;
DECLARE var3 VARCHAR(50);
DECLARE curs CURSOR FOR SELECT something FROM somewhere WHERE some stuff;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET bDone = 1;
DROP TEMPORARY TABLE IF EXISTS tblResults;
CREATE TEMPORARY TABLE IF NOT EXISTS tblResults (
--Fld1 type,
--Fld2 type,
--...
);
OPEN curs;
SET bDone = 0;
REPEAT
FETCH curs INTO var1, var2, var3;
IF whatever_filtering_desired
-- here for whatever_transformation_may_be_desired
INSERT INTO tblResults VALUES (var1, var2, var3);
END IF;
UNTIL bDone END REPEAT;
CLOSE curs;
SELECT * FROM tblResults;
END
A few things to consider...
Concerning the snippet above:
may want to pass part of the query to the Stored Procedure, maybe particularly the search criteria, to make it more generic.
If this method is to be called by multiple sessions etc. may want to pass a Session ID of sort to create a unique temporary table name (actually unnecessary concern since different sessions do not share the same temporary file namespace; see comment by Gruber, below)
A few parts such as the variable declarations, the SELECT query etc. need to be properly specified
More generally: trying to avoid needing a cursor.
I purposely named the cursor variable curs[e], because cursors are a mixed blessing. They can help us implement complicated business rules that may be difficult to express in the declarative form of SQL, but it then brings us to use the procedural (imperative) form of SQL, which is a general feature of SQL which is neither very friendly/expressive, programming-wise, and often less efficient performance-wise.
Maybe you can look into expressing the transformation and filtering desired in the context of a "plain" (declarative) SQL query.
Use cursors.
A cursor can be thought of like a buffered reader, when reading through a document. If you think of each row as a line in a document, then you would read the next line, perform your operations, and then advance the cursor.
Using a cursor within a stored procedure.
Prepare the SQL Query
SELECT id FROM employee where department_id = 1;
Create the cursor which will hold the result set returned by the SQL Query.
DECLARE BonusDistributionCursor CURSOR FOR SELECT id FROM employee where department_id = 1;
To have a safe exit when fetching a row from cursor does not return any result then declare a handler called NOT FOUND and set value to a declared variable
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
Open the Cursor before you can fetch the next row from the cursor.
OPEN BonusDistributionCursor;
Fetch the next row pointed by the cursor and move the cursor to next row after that.
FETCH BonusDistributionCursor INTO employeeId;
Run the desired business logic according to the usecase required.
DELIMITER $$
CREATE PROCEDURE distributeYearlyBonus (IN departmentId VARCHAR(2))
BEGIN
DECLARE finished INTEGER DEFAULT 0;
DECLARE empId VARCHAR(TEXT) DEFAULT "";
DECLARE BonusDistributionCursor CURSOR FOR SELECT id FROM employee where department_id = departmentId;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN BonusDistributionCursor;
addBonus: LOOP
FETCH BonusDistributionCursor INTO empId;
IF finished = 1 THEN
LEAVE addBonus;
END IF;
INSERT INTO `bonus_paid_details` (`employee_id`, `year`, `datetime`) VALUES (empId, YEAR(CURDATE());, now());
END LOOP addBonus;
CLOSE BonusDistributionCursor;
END$$
DELIMITER ;
Execute the above script and you will find a new Stored Procedure created.
Call or Invoke the Stored Procedure by inputing the departmentId which will receive the bonus amount.
CALL BonusDistributionCursor(1);
Hope this explains "How to iterate using Cursors used within Stored Procedure"
I create a Mysql procedure using cursor, but it's run too slow... It's get between 40 and 60 lines by second.. See:
DELIMITER $$
CREATE PROCEDURE sp_create(IN v_idsorteio INT,OUT afetados INT)
BEGIN
DECLARE done INT default 0;
DECLARE vc_idsocio INT;
DECLARE z INT;
DECLARE cur1 CURSOR FOR select IdSocio from socios where Sorteio=1 and Finalizado='S' and CodClientes IS NOT NULL;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done=1;
SET z=1;
OPEN cur1;
FETCH cur1 INTO vc_idsocio;
WHILE done=0 DO
-- SELECT register as t;
insert INTO socios_numeros_sorteio (IdSocio,IdSorteio,NumerodeSorteio) VALUES (vc_idsocio,v_idsorteio,z);
FETCH cur1 INTO vc_idsocio;
SET z = z+1;
END WHILE;
CLOSE cur1;
Select z-1 as total INTO afetados;
END$$
DELIMITER ;
how can I to improve that?
This is slow because you are looping through a resultset, row by row, and performing individual insert statements for each row returned. That's why it's gonna be slow.
Let's briefly summarize what you are doing. First, you are running a query:
select IdSocio
from socios
where Sorteio=1
and Finalizado='S'
and CodClientes IS NOT NULL;
(Apparently the order these rows are returned in is not important.)
Then for each row returned from that query, you want to insert a row into another table.
insert INTO socios_numeros_sorteio
(IdSocio
,IdSorteio
,NumerodeSorteio
) VALUES
(vc_idsocio
,v_idsorteio
,z);
The value for the first column is coming from a value returned by the query.
The value for the second column is being assigned a value passed as an argument to the procedure.
And the value for the third column is from a counter that starts at 1 and is being incremented by 1 for each row.
MySQL is optimized to perform an operation like this. But it's NOT optimized to do this using a stored procedure that loops through a cursor row by row.
If you are looking to get some reasonable performance, you need to SIGNIFICANTLY REDUCE the number of individual INSERT statements you run, and instead think in terms of processing data in "sets" rather than individual rows. One approach is batch the rows up into "extended insert" statements, which can insert multiple rows at a time. (The number rows you can insert in one statement is effectively limited by max_allowed_packet.)
That approach will significantly improve performance, but it doesn't avoid the overhead of the cursor, fetching each row into procedure variables.
Something like this (in the body of your procedure) is likely to perform much, much better, because it takes the result set from your select and inserts all of the rows into the destination table in one fell swoop, without bothering to mess with updating the values of variables in the procedure.
BEGIN
SET #idsorteio = v_idsorteio;
INSERT INTO socios_numeros_sorteio
( IdSocio
, IdSorteio
, NumerodeSorteio
)
SELECT s.IdSocio AS IdSocio
, #idsorteio AS IdSorteio
, #z := #z+1 AS NumerodeSorteio
FROM socios s
JOIN (SELECT #z := 0) z
WHERE s.Sorteio=1
AND s.Finalizado='S'
AND s.CodClientes IS NOT NULL;
SELECT ROW_NUMBER() INTO afetados;
END$$
Another simple solution is only to change the engine of the table to MyISAM by running the below query,
ALTER TABLE `socios_numeros_sorteio`
ENGINE=MyISAM;
Then CALL the procedure again.
note: MyISAM make the insertion process very fast
I have a table with lots of different columns, like
resolution, brightness, type, width, camera_type, projection_type etc
I need to get the names of only those columns, that are relevant to, say LED tv's to build filters, - which means I need to somehow
SHOW COLUMNS FROM Goods WHERE category_id = 5 AND [there is at least one value in the column for a row that meets the category_id = 5 condition]
Is it even possible?
I'm guessing what you want is the column names that have a non-null value where category_id = 5. The difficulty here is finding out which columns have a non-null value where category_id = 5. This will have to be accomplished on a per column basis but can be done.
To be honest... I don't work in MySql but I'm certain you can follow along and make any adjustments needed.
-- create a temp table to hold the results
CREATE TEMPORARY TABLE TempTable (columnName varchar(100));
-- create a bunch of dynamic SQL to insert its name into the temp table when category_id = 5 and its value is not null
DECLARE column_name VARCHAR(100);
DECLARE no_more_rows BOOLEAN;
DECLARE loop_cntr INT DEFAULT 0;
DECLARE num_rows INT DEFAULT 0;
-- Declare the cursor
DECLARE columns_cur CURSOR FOR
SELECT
column_name
FROM information_schema.columns
WHERE TABLE_NAME = 'Goods';
-- Declare 'handlers' for exceptions
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
OPEN columns_cur;
select FOUND_ROWS() into num_rows;
the_loop: LOOP
FETCH columns_cur
INTO column_name;
IF no_more_rows THEN
CLOSE columns_cur;
LEAVE the_loop;
END IF;
PREPARE stmt FROM 'IF(EXISTS(SELECT * FROM Goods WHERE ? IS NOT NULL AND catetory_id = 5)) INSERT INTO TempTable (column_name) VALUES ('?');'
EXECUTE stmt USING #column_name;
DEALLOCATE PREPARE stmt;
END LOOP the_loop;
You can put condition on the rows that you want to fetch, the where clause is responsible for it. The columns that you want are to be specified in the select statement only, there is no way to specify which column to be selected. Once you have the results you can decide which columns to use, depending upon the value.
I need to perform the same procedure for several tables in my DB. The poblem is that that procedure contains the following line:
DECLARE tableIt CURSOR FOR select id from table where column=inputParam ;
table is the table the procedure works with. And I can't find a way to make that table name to be dynamic, i.e. to read it from an input parameter.
Right now I have 8 different procedures (one for each table) which differentiate from each other only by one word (the name of the table).
That is really a pain since I have to make every change to the procedure 8 times.
Is it possible to parameterize the select statement for the CURSOR so I can have only one procedure??
Dynamic cursors does not seem to be supported in Mysql.
http://dev.mysql.com/doc/refman/5.1/en/cursors.html
You can work around it
http://forums.mysql.com/read.php?98,133197,149099#msg-149099
"DROP VIEW IF EXISTS v1;
SET #stmt_text=CONCAT("CREATE VIEW v1 AS SELECT c_text FROM ", t_name);
PREPARE stmt FROM #stmt_text;
EXECUTE stmt;
BEGIN
DECLARE v_text VARCHAR(45);
DECLARE done INT DEFAULT 0;
DECLARE c cursor FOR SELECT c_text FROM v1;"