Why is the cursor exiting after one iteration? - mysql

I have a table that lists tables that I want to get counts off of:
|dq_tbl_id|dq_tbl_tbl_name |dq_tbl_use_batch_stamp|
|---------|--------------------------|----------------------|
|1 |dev_credly_badges |1 |
|... |... |... |
|18 |tbl18 |[NULL] |
I can't figure out why the procedure exits after one iteration:
CREATE DEFINER=`channel`#`%` PROCEDURE `dq_job_dev_count`()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE proc_dq_tbl_id SMALLINT;
DECLARE tbl_count INT DEFAULT 0;
DECLARE tbl_name VARCHAR(35);
DECLARE dq_tbl_use_batch_stamp BOOL DEFAULT FALSE;
DECLARE dq_tbl_ids
CURSOR FOR
SELECT dq_tbl_id FROM ref_dq_tbl_count;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN dq_tbl_ids;
WHILE done = 0 DO
FETCH NEXT FROM dq_tbl_ids INTO proc_dq_tbl_id;
IF done = 0 THEN
SELECT dq_tbl_tbl_name, dq_tbl_use_batch_stamp INTO #tbl_name, #dq_tbl_use_batch_stamp FROM ref_dq_tbl_count WHERE dq_tbl_id = #proc_dq_tbl_id;
IF #dq_tbl_use_batch_stamp = 1 THEN
SET #sql_stmt = CONCAT('SELECT COUNT(*) INTO #tbl_count FROM ', #tbl_name, ' WHERE DATE(batch_stamp) = CURDATE()');
ELSE
SET #sql_stmt = CONCAT('SELECT COUNT(*) INTO #tbl_count FROM ', #tbl_name);
END IF;
PREPARE stmt FROM #sql_stmt;
EXECUTE stmt;
INSERT INTO prod_dq_tbl_load_counts (tbl_name, row_count) VALUES (#tbl_name, #tbl_count);
END IF;
END WHILE;
CLOSE dq_tbl_ids;
END

You are fetching the cursor result into a local variable:
FETCH NEXT FROM dq_tbl_ids INTO proc_dq_tbl_id;
Subsequently you use a user-defined variable #proc_dq_tbl_id under the mistaken assumption that it contains the value you fetched.
In MySQL stored procedures, a variable with the # sigil is a user-defined variable. It is literally a different variable than the local variable you created with DECLARE.
You can fetch the cursor into a user-defined variable (with the # sigil) or you can fetch the cursor into a local variable. But then you must use the same variable in subsequent expressions.
Since #proc_dq_tbl_id is likely to be NULL, the SELECT statement you use it in will never match any rows. Your cursor is probably looping plenty, but the SELECT statement in each loop iteration is finding no matches.
This treatment of variables in MySQL is different than other brands of SQL database (e.g. Microsoft SQL Server). It is sometimes confusing to developers who are accustomed to using other brands.

Related

want to Write a Stored Procedure in MYSQL [duplicate]

I have made a stored procedure. I want it to filter the data by different parameters. If I pass one parameter, it should be filtered by one; if I pass two, it should be filtered by two, and so on, but it is not working.
Can anyone help me please?
DROP PROCEDURE IF EXISTS medatabase.SP_rptProvince2;
CREATE PROCEDURE medatabase.`SP_rptProvince2`(
IN e_Region VARCHAR(45)
)
BEGIN
DECLARE strQuery VARCHAR(1024);
DECLARE stmtp VARCHAR(1024);
SET #strQuery = CONCAT('SELECT * FROM alldata where 1=1');
IF e_region IS NOT NULL THEN
SET #strQuery = CONCAT(#strQuery, ' AND (regionName)'=e_Region);
END IF;
PREPARE stmtp FROM #strQuery;
EXECUTE stmtp;
END;
AFAIK, you can't have a variable argument list like that. You can do one of a couple of things:
Take a fixed maximum number of parameters, and check them for null-ness before concatenating:
CREATE PROCEDURE SP_rptProvince2(a1 VARCHAR(45), a2 VARCHAR(45), ...)
...
IF a1 IS NOT NULL THEN
SET #strQuery = CONCAT(#strQuery, ' AND ', a2);
END IF;
If you need predetermined fields to which the criteria in the argument apply (like the e_Region parameter in your existing code), then you modify the CONCAT operation appropriately.
Possible invocation:
CALL SP_rptProvince2('''North''', 'column3 = ''South''')
Take a single parameter that is much bigger than just 45 characters, and simply append it to the query (assuming it is not null).
Clearly, this places the onus on the user to provide the correct SQL code.
Possible invocation:
CALL SP_rptProvince2('RegionName = ''North'' AND column3 = ''South''')
There's not a lot to choose between the two. Either can be made to work; neither is entirely satisfactory.
You might note that there was a need to protect the strings in the arguments with extra quotes; that is the sort of thing that makes this problematic.
I found a JSON-based approach which works with the latest MySQL/MariaDB systems. Check the link below (Original Author is Federico Razzoli): https://federico-razzoli.com/variable-number-of-parameters-and-optional-parameters-in-mysql-mariadb-procedures
Basically, you take a BLOB parameter which is actually a JSON object and then do JSON_UNQUOTE(JSON_EXTRACT(json object, key)) as appropriate.
Lifted an extract here:
CREATE FUNCTION table_exists(params BLOB)
RETURNS BOOL
NOT DETERMINISTIC
READS SQL DATA
COMMENT '
Return whether a table exists.
Parameters must be passed in a JSON document:
* schema (optional). : Schema that could contain the table.
By default, the schema containing this procedure.
* table : Name of the table to check.
'
BEGIN
DECLARE v_table VARCHAR(64)
DEFAULT JSON_UNQUOTE(JSON_EXTRACT(params, '$.table'));
DECLARE v_schema VARCHAR(64)
DEFAULT JSON_UNQUOTE(JSON_EXTRACT(params, '$.schema'));
IF v_schema IS NULL THEN
RETURN EXISTS (
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE
TABLE_SCHEMA = SCHEMA()
AND TABLE_NAME = v_table
);
ELSE
RETURN EXISTS (
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE
TABLE_SCHEMA = v_schema
AND TABLE_NAME = v_table
);
END IF;
END;

Differentiate between no results returned and end of rows for cursor

I'm trying to implement a cursor in MYSQL. Inside that cursor, I have a select statement that might or might not return any rows.
I'm facing a problem when that query does not return any rows.
According to the MYSQL documentation,
NOT FOUND: Shorthand for the class of SQLSTATE values that begin with '02'. This is relevant within the context of cursors and is used to control what happens when a cursor reaches the end of a data set. If no more rows are available, a No Data condition occurs with SQLSTATE value '02000'. You can set up a handler for it or a NOT-FOUND condition to detect this condition. For another example, see Section 13.6.6, “Cursors”. The NOT FOUND condition also occurs for SELECT ... INTO var_list statements that retrieve no rows.
My termination of the cursor is implemented as usual.
DECLARE CONTINUE HANDLER
FOR NOT FOUND SET FINISHED = 1;
The problem is that the termination is triggered because of the query inside the cursor returning 0 rows. Is there a way to differentiate between those two cases so that the cursor continues despite my query returning 0 results?
Thanks in advance.
DROP PROCEDURE IF EXISTS CURSOR_PLACEHOLDER;
DELIMITER $$
CREATE PROCEDURE CURSOR_PLACEHOLDER()
BEGIN
DECLARE FINISHED INTEGER DEFAULT 0;
DECLARE CURRENT_ROW_ID VARCHAR(256);
DEClARE CURRENT_ROW
CURSOR FOR
SELECT ID
FROM A
WHERE ID_P IN (SELECT ID_P
FROM A
GROUP BY ID_P
HAVING COUNT(ID_P) > 1);
DECLARE CONTINUE HANDLER
FOR NOT FOUND SET FINISHED = 1;
OPEN CURRENT_ROW;
GET_ACTION:
LOOP
FETCH CURRENT_ROW INTO CURRENT_ROW_ID;
IF FINISHED = 1 THEN
LEAVE GET_ACTION;
END IF;
SET #CURRENT_P_ID := (SELECT ID_P FROM A WHERE ID = CURRENT_ROW_ID);
SELECT a.ID_U, a.ID_R, a.A, a.FROMDATE, a.TODATE
INTO #ID_U, #ID_R, #A, #FROM_DATE, #TO_DATE
FROM ASSIG a
WHERE a.ID_G = #CURRENT_P_ID; # <- Query that returns 0 rows and terminates the cursor
IF (#ID_U IS NOT NULL) THEN
SET #ASSIG_ID := GENERATE_ID();
INSERT INTO ASSIG
VALUES (#ASSIG_ID,
#G_ID,
#ID_U,
#ID_R,
#A,
#FROM_DATE,
#TO_DATE);
END IF;
UPDATE A
SET ID_P = #G_ID
WHERE ID = CURRENT_ROW_ID;
END LOOP GET_ACTION;
CLOSE CURRENT_ROW;
END
$$
DELIMITER ;

How to count values from the database table column?

I want to convert game tag data found here into a dimension table of a star schema.
But the problem is that steamspy_tag_data table is organised as such every column name is tag name of a game and one game can have multiple tags. For example, lets say I have game Warcraft3 with appid 30 it would be in a table like this.
appid|strategy|action|shooter|fantasy|
-----+--------+------+-------+-------
30 6345 1452 0 6340
Column value greater than 0 signifies amount of user votes that voted certain game to be of that game tag. For Warcraft3 game with appid 30 - 6345 users voted it classifies as strategy 1452 users it classifies as action, .. etc.
Some columns for example "abstract" (column) tag has almost all 0 throughout the whole column meaning almost no game uses that tag, so to simplify 372 columns with over 29k row value into something more compact I want to run a query that would count non-zero value per every tag column and put them in the new table "tagovi" for better visibility which columns(tags) have relatively low game usage count.
so far I came up with this:
CREATE DEFINER=`root`#`localhost` PROCEDURE `zbroji_tagove`()
BEGIN
DECLARE i INTEGER;
DECLARE total_row_count INTEGER;
DECLARE dummy VARCHAR(255);
DECLARE zbrojeno INTEGER;
DECLARE trenutna VARCHAR(255);
DECLARE kursor CURSOR FOR
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'sppi' AND table_name = 'steamspy_tag_data'
ORDER BY ordinal_position;
SELECT FOUND_ROWS() into total_row_count;
open kursor;
FETCH kursor into dummy;
SET i = 1;
ponavljanje: LOOP
IF i > total_row_count THEN
CLOSE kursor;
LEAVE ponavljanje;
END IF;
FETCH kursor INTO trenutna;
SET zbrojeno = 0;
SET zbrojeno = (SELECT COUNT(*) FROM steamspy_tag_data where trenutna <> 0);
INSERT INTO tagovi(kategorija,broj_igra)
VALUES (
(trenutna),(zbrojeno)
);
SET i = i + 1;
end LOOP;
END
New table tagovi has 3 columns (ID auto_increment, kategorija Varchar(255), broj_igra INTEGER).
When I execute my stored procedure "zbroji_tagove"() I get SQL ERROR CODE 1292; Truncated incorrect DOUBLE value 'some_tag_name'. So somehow sql treats variable value as value instead of column at line SET zbrojeno = (SELECT COUNT() FROM steamspy_tag_data where trenutna <> 0*);
Is there a way for me to accomplish what I want inside MySQL environment?
You can't use variables like that, you need a prepared statement
See
#sql = CONCAT("SELECT COUNT(*) INTO zbrojeno FROM steamspy_tag_data where ",trenutna," <> 0);";
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;

MySQL create view across all prefixed databases' table

I have databases named company_abc, company_xyz, etc. Those company_* databases have all the same structure and they contain users table.
What I need to do is to aggregate all users data from just company_* databases and replicate this view to another server. The view would just be something like
COMPANY NAME | USERNAME
abc | user#email.com
abc | user1#email.com
xyz | user2#email.com
company3 | user3#email.com
Is something like that possible in MySQL?
The databases are created dynamically, as well as the users so I can't create a view with just a static set of databases.
As you say you want to create view with dynamic database names - so the result you want to achieve is not possible in current versions of mysql.
So you have example following options:
Option 1
If you want to get result of all databases users tables you could define a stored procedure that uses prepared statement. This procedure needs parameter db_prefix what in your case is company_%. Basicly this procedure selects all tables named as users from information_schema when database name is like db_prefix parameter value. After that it loops through results and creates query string as union all users tables and executes this query. When creating a query string i also add field called source, so i can identify from what database this result is coming. In my example my databases are all in default collation utf8_unicode_ci.
In this case you can define procedure example "getAllUsers"
-- Dumping structure for procedure company_abc1.getAllUsers
DELIMITER //
CREATE DEFINER=`root`#`localhost` PROCEDURE `getAllUsers`(IN `db_prefix` TEXT)
DETERMINISTIC
COMMENT 'test'
BEGIN
DECLARE qStr TEXT DEFAULT '';
DECLARE cursor_VAL VARCHAR(255) DEFAULT '';
DECLARE done INTEGER DEFAULT 0;
DECLARE cursor_i CURSOR FOR SELECT DISTINCT (table_schema) FROM information_schema.tables WHERE table_name = 'users' AND table_schema LIKE db_prefix COLLATE utf8_unicode_ci;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cursor_i;
read_loop: LOOP
FETCH cursor_i INTO cursor_VAL;
IF done = 1 THEN
LEAVE read_loop;
END IF;
IF qStr != '' THEN
SET qStr = CONCAT(qStr, ' UNION ALL ');
END IF;
SET qStr = CONCAT(qStr, ' SELECT *, \'', cursor_VAL ,'\' as source FROM ', cursor_VAL, '.users');
END LOOP;
CLOSE cursor_i;
SET #qStr = qStr;
PREPARE stmt FROM #qStr;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #qStr = NULL;
END//
DELIMITER ;
Now you can get your all users result as:
CALL getAllUsers('company_%');
In my example database it results as:
id name source
1 User 1 company_abc1
2 User 2 company_abc1
3 User 3 company_abc1
1 User 1 company_abc2
2 User 2 company_abc2
3 User 3 company_abc2
1 User 1 company_abc3
2 User 2 company_abc3
3 User 3 company_abc3
1 User 1 company_abc4
2 User 2 company_abc4
3 User 3 company_abc4
1 User 1 company_abc5
2 User 2 company_abc5
3 User 3 company_abc5
Option 2
If you really, really need view then you can modify first procedure and instead of executeing select you can create view. Example like this:
-- Dumping structure for procedure company_abc1.createAllUsersView
DELIMITER //
CREATE DEFINER=`root`#`localhost` PROCEDURE `createAllUsersView`(IN `db_prefix` TEXT)
DETERMINISTIC
COMMENT 'test'
BEGIN
DECLARE qStr TEXT DEFAULT '';
DECLARE cursor_VAL VARCHAR(255) DEFAULT '';
DECLARE done INTEGER DEFAULT 0;
DECLARE cursor_i CURSOR FOR SELECT DISTINCT (table_schema) FROM information_schema.tables WHERE table_name = 'users' AND table_schema LIKE db_prefix COLLATE utf8_unicode_ci;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cursor_i;
read_loop: LOOP
FETCH cursor_i INTO cursor_VAL;
IF done = 1 THEN
LEAVE read_loop;
END IF;
IF qStr != '' THEN
SET qStr = CONCAT(qStr, ' UNION ALL ');
END IF;
SET qStr = CONCAT(qStr, ' SELECT *, \'', cursor_VAL ,'\' as source FROM ', cursor_VAL, '.users');
END LOOP;
CLOSE cursor_i;
SET #qStr = CONCAT('CREATE OR REPLACE VIEW allUsersView AS ', qStr);
PREPARE stmt FROM #qStr;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #qStr = NULL;
END//
DELIMITER ;
In this stored procedure we create/replace view called allUsersView, so basicly every time you will execute this procedure it will updates view.
In my test case it creates view like this:
CREATE OR REPLACE VIEW `allusersview` AS
SELECT *, 'company_abc1' as source FROM company_abc1.users
UNION ALL SELECT *, 'company_abc2' as source FROM company_abc2.users
UNION ALL SELECT *, 'company_abc3' as source FROM company_abc3.users
UNION ALL SELECT *, 'company_abc4' as source FROM company_abc4.users
UNION ALL SELECT *, 'company_abc5' as source FROM company_abc5.users ;
And now you can use view.
SELECT * FROM allusersview
And result is same as in first option.
All tested on:
Mysql 5.6.16
To find the list of database names:
SELECT SCHEMA_NAME
FROM information_schema.`SCHEMATA`
WHERE SCHEMA_NAME LIKE 'company%';
If you can code in something like PHP, the rest is pretty easy -- build a UNION of SELECTs from each database. But, if you must do it just in SQL...
To build the UNION, write a Stored Procedure. It will do the above query in a CURSOR. Inside the loop that walks through the cursor, CONCAT() a constructed SELECT onto a UNION you are building.
When the loop is finished, PREPARE and EXECUTE the constructed UNION. That will deliver something like the output example you had.
But, if you now need to INSERT the results of that into another server, you should leave the confines of the Stored Procedure and use some other language.
OK, OK, if you must stay in SQL, then you need some setup: Create a "Federated" table that connects to the other server. Now, in your SP, concatenate INSERT INTO fed_tbl in front of the UNION. Then the execute should do the entire task.
If you have trouble with the FEDERATED Engine, you may need to switch to FederatedX in MariaDB.
"The details are left as an exercise to the reader."
I already marked this as duplicate of Mysql union from multiple database tables
(SELECT *, 'abc' as COMPANY_NAME from company_abc.users)
union
(SELECT *, 'xyz' as COMPANY_NAME from company_xyz.users)
union
(SELECT *, 'company3' as COMPANY_NAME from company_company3.users)
...
I think that the only method to make this is to write a stored procedure that read all database and table name from information_schema.table, build a string with union select * from company_abc.users union all select * from company_xyz and then execute the command with prepared statement: http://dev.mysql.com/doc/refman/5.0/en/sql-syntax-prepared-statements.html

MySQL Stored Procedures : cursor declaration

Sorry for the vague title, here is my problem. I have stored procedures for DB2 that i try to convert for MySQL. I'd like to know if i can write the SELECT statement in the cursor declaration as a string variable. For example with DB2 i have this :
(...)
-- Declare cursors
DECLARE c_very_init CURSOR WITH RETURN FOR s_very_init;
DECLARE c_date CURSOR WITH RETURN FOR s_date;
DECLARE CONTINUE HANDLER FOR not_found
SET at_end = 1;
-- In case the_date is 0, retrieve the first date
IF the_date = 0 THEN
SET sql_end_date = '
SELECT DATE
FROM ACCOUNTS
WHERE REF = ''' || the_ref || '''
ORDER BY ID ASC FETCH FIRST 1 ROWS ONLY';
PREPARE s_date FROM sql_end_date;
OPEN c_date;
FETCH FROM c_date INTO data_ins;
SET the_last_date = data_ins;
CLOSE c_date;
ELSE
SET the_last_date = the_date;
END IF;
-- Get the 'very' initial value
SET sql_very_init = '
SELECT in, out
FROM MOVEMENTS
WHERE REF = ''' || the_ref || '''
AND DATE < ' || the_last_date;
PREPARE s_very_init FROM sql_very_init;
OPEN c_very_init;
FETCH FROM c_very_init INTO dare, avere;
-- Loop through the results
(...)
I declare a c_very_init cursor, but at the time of the cursor declaration in the SP i still don't know the full select statement because i need to fetch (if necessary) the the_last_date value. It seems i can't do this :
DECLARE c_very_init CURSOR WITH RETURN FOR s_very_init;
with MySQL, the syntax being with the statement directly in the declaration :
DECLARE c_very_init CURSOR FOR SELECT blaablaa...;
Am i wrong?
Thank you.
fabien.
No, you cannot declare cursors in this way. But if 'the_ref' is a variable, you could do it like this -
...
DECLARE the_ref INT DEFAULT 10;
DECLARE cur1 CURSOR FOR SELECT column1 FROM table1 WHERE column1 = the_ref;
...