Related
I am relatively new to MySQL stored Procedures. I have a stored procedure that works fine in certain conditions and not in other. I'm a bit confused that what causes the error. It is a entity processing SP, based on certain values and conditions it either creates or update entity from data in a staging table.
In conditions when it works fine:
When I only process a single entity.
Works for bulk entries when there is nothing in entity table.
Don't works when:
There are entries in entity table and bulk processing is done. - Error in checking up the already existing entity_id in entity and older entity_id is assigned instead a new one should be created.
(I actually need above scenario to work more frequently than others)
I have tried to keep the code to minimum in order to understand the flow of SP. Please consider all variables as declared. SP might not compile.
CREATE DEFINER=`admin`#`%` PROCEDURE `sp_ent`(test_id int)
BEGIN
-- Move code tables to temp tables
-- Declare all the required variables here
DECLARE counter, len INT;
DECLARE var2 INT;
DECLARE var3 TINYINT(1);
DECLARE var4 DECIMAL(18,4);
DECLARE var5 DATE;
DECLARE var6 VARCHAR(1000);
DECLARE var7 VARCHAR(5000);
DROP TEMPORARY TABLE IF EXISTS `temp_ent`;
IF test_id IS NOT NULL THEN
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_ent` AS (SELECT * FROM `ent_st` WHERE processed = 0 and id=test_id);
ELSE
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_ent` AS (SELECT * FROM `ent_st` WHERE processed = 0);
END IF;
ALTER TABLE `temp_ent` ADD PRIMARY KEY(id);
-- SELECT * FROM `temp_ent`;
DROP TEMPORARY TABLE IF EXISTS `temp_exc`; CREATE TEMPORARY TABLE IF NOT EXISTS `temp_exc` AS (SELECT * FROM `code_exc`);
-- A few more like above
SET counter=1, len=(SELECT COUNT(*) FROM `temp_ent`);
WHILE counter <= len DO
SELECT `id`,var1, var2, var3
INTO v_id,var1, var2, var3
FROM `temp_ent` LIMIT 1;-- WHERE `id` = v_id;
BEGIN
DECLARE insufficient_information CONDITION FOR SQLSTATE '45000';
DECLARE CONTINUE HANDLER FOR insufficient_information SET v_proccessed=1;
SET v_status = CASE
WHEN ... THEN ...
ELSE 'valid'
END;
IF v_status <> 'valid' THEN SIGNAL insufficient_information; END IF;
SELECT `entity_id`,`entity`
INTO v_underlying_entity_id,v_underlying_entity_symbol
FROM `entity` WHERE `entity_id` = v_underlying OR `entity` = v_underlying_entity;
SET v_underlying_entity_symbol = COALESCE(v_underlying_entity_symbol,v_underlying_entity);
SET v_entity = (
CASE
WHEN ... THEN ...
WHEN ... THEN ...
.
.
WHEN ... THEN ...
END
);
SELECT `entity_id`,`entity`
INTO v_entity_id_check,v_entity_check
FROM entity WHERE `entity`=v_entity and `exc`=v_exc;
-- SELECT v_entity_id_check,v_entity_check,v_entity,v_exc;
IF v_entity_check IS NULL THEN
-- SELECT 'Create New Entity and Add';
SET v_entity_id = COALESCE((SELECT MAX(`entity_id`) FROM entity),0) + 1;
SET new_entity = 1;
ELSE
-- SELECT 'Entity Already Present';
SET v_entity_id = v_entity_id_check;
SET new_entity = 0;
END IF;
-- SELECT v_entity_id, v_entity, new_entity;
SET v_name = UPPER(COALESCE(v_name,v_entity));
-- UPDATE entity and underlying/derivatives table
IF new_entity = 1 THEN
-- Insert in respective tables
IF ... THEN
-- SELECT 'Entity Added',v_entity; -- Underlying Entity
INSERT INTO `entity_underlying` (`entity_id`,`entity`,`name`,`exc`,`seg`,`ins`,`isin`,`fo_yn`,`tick`,`lot_size`,`active_yn`)
VALUES (v_entity_id,v_entity,v_name,v_exc,v_seg,v_ins,COALESCE(v_isin,''),COALESCE(v_fo_yn,0),COALESCE(v_tick,0.05),COALESCE(v_lot_size,1),1);
ELSEIF ... THEN
-- SELECT 'Entity Added',v_entity; -- Derivative Entity
INSERT INTO `entity_derivatives` (`entity_id`,`entity`,`name`,`underlying`,`exc`,`seg`,`ins`,`ser`,`isin`,`strike`,`tick`,`lot_size`,`expiry`,`ex_ty`,`active_yn`)
VALUES (v_entity_id,v_entity,v_name,COALESCE(v_underlying_entity_id,-1),v_exc,v_seg,v_ins,COALESCE(v_ser,''),v_isin,COALESCE(v_strike),COALESCE(v_tick,0.05),COALESCE(v_lot_size,-1),v_expiry,v_ex_ty,1);
END IF;
-- Insert in final table
INSERT INTO entity(`entity_id`,`entity`,`exc`,`active_yn`)
VALUES (v_entity_id,v_entity,v_exc,1);
SET v_status='Entity Added';
ELSE
SET v_status='Not a New Entity';
END IF;
END;
UPDATE `ent_st` SET `entity_id`=v_entity_id,`processed`=1,`status`=v_status WHERE id=v_id;
DELETE FROM `temp_ent` WHERE id=v_id;
SET counter = counter +1;
COMMIT;
END WHILE;
END
Any help is highly appreciated. Thanks in advance.
This is really baffling me. I am converting a simple procedure from informix into mysql. What it basically does is tell me what the next event is from an event table and a calendar table. In informix the procedure is simple.
FOREACH
SELECT date,weekno,event
INTO l_date,l_week,l_event
FROM event,calendar
WHERE dayno = dayno
AND date = l_today
AND start >= l_now
UNION
SELECT date,weekno,event
FROM event,calendar
WHERE dayno = dayno
AND date > l_today
UNION
SELECT TODAY,9999,9999
FROM event,calendar
WHERE dayno = dayno
AND event = (SELECT MAX(event) FROM event)
ORDER BY 3
if l_event = 9999 then <error> end if;
EXIT FOREACH
END FOREACH
So basically the query finds the next event and returns it. l_today and l_event are parameters that are passed. So on to the mysql version.
looper: BEGIN
DECLARE curs1 CURSOR FOR
SELECT CONCAT("SELECT date, weekno, event FROM event INNER JOIN calendar ON dayno = dayno",
" WHERE date = '", lv_today ,"' AND start >= '", lv_time ,"'",
" UNION SELECT date, weekno, event FROM event INNER JOIN calendar ON dayno = dayno WHERE date > '", lv_today ,"'",
" UNION SELECT DATE(NOW()) AS date, 9999 AS weekno, 9999 AS event FROM event INNER JOIN calendar ON dayno = dayno",
" WHERE (SELECT MAX(event) FROM event) ORDER BY event ");
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done := TRUE;
OPEN curs1;
loop1: LOOP
FETCH curs1 INTO ldate, lweek, levent;
SELECT ldate, lweek, levent;
LEAVE looper;
END LOOP loop1;
END;
I haven't checked that the rest of the methodology works because I get this error:
Incorrect number of FETCH variables.
Does this mean that I have declare a different variable for each of the query returns? I am new to mysql. If this is the case what would be the best way to solve this conundrum? I have also changed to column and table names.
Many thanks
looper: BEGIN
DECLARE curs1 CURSOR FOR
SELECT eve_date, dia_weekno, eve_event
FROM game_event
INNER JOIN stan_calendar ON eve_abs_dayno = dia_abs_dayno
WHERE eve_date = lv_today
AND eve_start >= lv_time
UNION
SELECT eve_date, dia_weekno, eve_event
FROM game_event
INNER JOIN stan_calendar ON eve_abs_dayno = dia_abs_dayno
WHERE eve_date > lv_today
UNION SELECT DATE(NOW()) AS eve_date, 9999 AS dia_weekno, 9999 AS eve_event
FROM game_event
INNER JOIN stan_calendar ON eve_abs_dayno = dia_abs_dayno
WHERE (SELECT MAX(eve_event) FROM game_event) ORDER BY eve_event;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done := TRUE;
OPEN curs1;
curs_loop: LOOP
FETCH curs1 INTO lv_date, lv_week, lv_event;
SELECT lv_date, lv_week, lv_event;
LEAVE looper;
CLOSE curs1;
END LOOP curs_loop;
Thank you for answering my question, I have put it back to how I thought it was... and it now works. Here is the loop in full.
https://dev.mysql.com/doc/refman/5.7/en/fetch.html The number of columns retrieved by the SELECT statement must match the number of output variables specified in the FETCH statement so yes but in fact you are only selecting 1 concatenated string - I think your first problem is with the select syntax which doesn't need the concat, the brackets or the quotes (unless you are trying to create a prepared statement for some reason - and even if you were I doubt if the code would be correct)
Simple Cursor
DROP PROCEDURE IF EXISTS EC;
DELIMITER $$
CREATE PROCEDURE `EC`(
IN `inemp_no` varchar(255)
)
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT ''
LOOPER:begin
DECLARE done INT DEFAULT FALSE;
declare ename varchar(20);
declare esalary int default 0;
declare emp_cursor CURSOR FOR
SELECT last_name,salary FROM employees where emp_no= inemp_no ;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
open emp_cursor;
read_loop: loop
fetch emp_cursor into ename,esalary;
if done then leave read_loop; end if;
insert into debug_table (msg) values(concat('employee:',ename,' earns:',esalary));
end loop;
close emp_cursor;
end $$
DELIMITER ;
MariaDB [sandbox]> truncate table debug_table;
Query OK, 0 rows affected (0.22 sec)
MariaDB [sandbox]> call ec(2);
Query OK, 0 rows affected (0.03 sec)
MariaDB [sandbox]> select * from debug_table;
+----+--------------------------+------+
| id | msg | MSG2 |
+----+--------------------------+------+
| 1 | employee:BBB earns:39500 | NULL |
+----+--------------------------+------+
1 row in set (0.00 sec)
DELIMITER $$
CREATE PROCEDURE Load_Fact_List()
BEGIN
DECLARE Project_Number_Temp INT;
DECLARE Panel_Id_Temp INT;
DECLARE Employee_Id_Temp INT;
DECLARE Zip_Temp VARCHAR(255);
DECLARE Created_Date_Temp DATE;
DECLARE Country_Temp VARCHAR(255);
DECLARE no_more_rows BOOLEAN;
DECLARE loop_cntr INT DEFAULT 0;
DECLARE num_rows INT DEFAULT 0;
DECLARE load_cur CURSOR FOR
SELECT Project_Id, Panel_Id, Employee_Id, Zip, Created_Date
FROM Fact_List;
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
OPEN load_cur;
select FOUND_ROWS() into num_rows;
the_loop: LOOP
FETCH load_cur
INTO Project_Number_Temp, Panel_Id_Temp, Employee_Id_Temp, Zip_Temp, Created_Date_Temp;
IF no_more_rows THEN
CLOSE load_cur;
LEAVE the_loop;
END IF;
SET Country_Temp= (select Country from Zip where Zip= Zip_Temp);
INSERT INTO Test_Fact
(
Project_Key,
Campaign_Key,
Respondents_Key,
Event_Key,
Employee_Key,
Geography_Key,
Date_Key
)
SELECT (SELECT Project_Key from Project_Dim where Project_Id= Project_Number_Temp AND Quota_Country= Country_Temp),0,(SELECT MAX(Respondents_Key) from Respondents_Dim WHERE Panel_Id= Panel_Id_Temp),1,(select MAX(Employee_Key) from Employee_Dim WHERE Employee_Id= Employee_Id_Temp),(Select Geography_Key from Geography_Dim where Zip= Zip_Temp), (Select Date_Key from Date_Dim where Full_Date= Created_Date_Temp);
SET loop_cntr = loop_cntr + 1;
END LOOP the_loop;
select num_rows, loop_cntr;
END $$
The above code is properly working but it is damn slow. For every 1 hour it is loading 1000 records. I got lacks of records to load into fact table. can anyone suggest me any optimization?
Requirement is to load fact table by looping through other table and gathering required key values from dimension tables.
The usual procedure is actually like this.
You have your dimensions built and you just gathered the data you want to insert into your fact table in a temporary table. Then you insert this data in another temporary table like this:
INSERT INTO tmp_fact_table
(
fact_key,
dim1_key,
dim2_key,
...
fact1,
fact2
...
)
SELECT
ISNULL (f.fact_key, 0),
ISNULL (d1.sid, 0) as whatever,
ISNULL (d2.sid, 0) as whatever2,
...
ISNULL (tt.fact1, 0),
ISNULL (tt.fact2, 0)
FROM
yourTempTable tt
LEFT JOIN Dim1 d1 ON tt.identifying_column = d1.identifying_column
...
LEFT JOIN fact_table f ON
f.dim1_key = d1.sid
AND f.dim2_key = d2.sid
where
fact_key is the identifying column in your fact table
dim1_key is the foreign key in your fact table to the dimensions
fact1 and so on are the facts you want in your fact table, clear
the ISNULL() function returns 0 when no entry is found. 0 is the id of your dummy row in each dimension for unknown data
Then you will have a table where you have the IDs of your dimensions linked to the data you want to import into your fact table with 0 as fact key when the entry in the fact table does not already exist and the ID of the fact table entry otherwise.
Then you update the fact table where tmp_fact_table.fact_key != 0
Then you insert into the fact table where tmp_fact_table.fact_key = 0
That's it.
I'm doing this with millions of rows and it takes about half an hour. 300,000 rows is peanuts.
I'm trying to get munus & submenus on the basis of roles (specified in other table).
On the basis of role, ex. if I chose MenuIDs: 1,2,5 I should get all submenus of M1, M2 & M3.
MenuParentID specifies MenuId of the parent.
MenuID MenuParentID MenuName MenuNavigateUrl HasSubMenus
1 -1 M1 1.aspx 0
2 -1 M2 # 1
3 2 M2.1 2.aspx 0
4 2 M2.2 3.aspx 0
5 -1 M3 # 1
6 5 M3.1 # 1
7 5 M3.2 # 1
8 6 M3.1.1 4.aspx 0
9 6 M3.1.2 5.aspx 0
10 7 M3.2.1 6.aspx 0
11 7 M3.2.2 7.aspx 0
12 -1 M4 # 1
13 12 M4.1 8.aspx 0
14 12 M4.2 9.aspx 0
Here's what I did:
DELIMITER $$
DROP FUNCTION IF EXISTS `myDB`.`GetPermissions`$$
CREATE DEFINER=`root`#`%` FUNCTION `GetPermissions`(
rootMenuID int(11)
) RETURNS varchar(200) CHARSET latin1
BEGIN
DECLARE menuIdList VARCHAR(100);
DECLARE menu_id INT(11);
DECLARE record_not_found INT DEFAULT 0;
DECLARE getMenuCursor CURSOR FOR SELECT DISTINCT(MenuId) FROM MenuIdListTable;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET record_not_found = 1;
CREATE TEMPORARY TABLE MenuIdListTable(MenuId INT(11) NULL);
SET menuIdList = ',';
IF((SELECT COUNT(*) FROM menus WHERE MenuParentID = rootMenuID) > 0) THEN
INSERT INTO MenuIdListTable(MenuID) SELECT MenuID FROM menus WHERE
MenuParentID = rootMenuID;
OPEN getMenuCursor;
read_loop: LOOP
FETCH getMenuCursor INTO menu_id;
IF record_not_found THEN
LEAVE read_loop;
END IF;
SET menuIdList = CONCAT(menuIdList,menu_id,',');
END LOOP read_loop;
END IF;
DROP TEMPORARY TABLE MenuIdListTable;
SET menuIdList = SUBSTR(menuIdList,1,LENGTH(menuIdList)-1);
RETURN menuIdList;
END$$
DELIMITER ;
But I'm not able to apply recursive logic to get all sub menus.
Ex. For 'M3' (MenuID = 5), I'm getting submenus 'M3.1' & 'M3.2'; but not their submenus. i.e For 'M3.1': 'M3.1.1', 'M3.1.2' and for 'M3.2': 'M3.2.1', 'M3.2.2'.
Also problem will persist if one of them have submenus! Please help.
Aww the joy of "connect by prior" from plsql or CTE from other dbs missing mysql.
Since my brain currently does not have the capacity to fully parse your given mysql function, it just throws a reference to: http://explainextended.com/2009/03/17/hierarchical-queries-in-mysql/
It points you to some good explainations (sometimes evil... like the tree strucutre in only one select) regarding hierarchicals in mysql
Try this code:
DROP TABLE IF EXISTS temp;
CREATE TABLE temp(id int,MenuID int,MenuName varchar(50),MenuParentID int,HasSubMenus int) SELECT (#id:=#id+1) as id,MenuID,MenuName,MenuParentID,HasSubMenus from menus,
(SELECT #id:=0) id where MenuParentID = 5;
select * from temp;
DROP TABLE IF EXISTS output;
CREATE TABLE output(MenuID int,MenuName varchar(50),MenuParentID int,HasSubMenus int) select MenuID,MenuName,MenuParentID,HasSubMenus from temp;
SET #idmin = (SELECT min(id) from temp);
SET #idmax = (SELECT max(id) from temp);
WHILE #idmin <= #idmax DO
INSERT INTO output(MenuID,MenuName,MenuParentID,HasSubMenus)
select MenuID,MenuName,MenuParentID,HasSubMenus from menus where MenuParentID=(select MenuID from temp where id = #idmin);
SET #idmin=#idmin+1;
END WHILE;
select (#id:=#id) as id1,GROUP_CONCAT(MenuName,',') from output, (SELECT #id:=1) id1 group by id1;
I want to limit my SELECT results in mySQL by sum.
For Example, this is my table:
(id, val)
Data Entries:
(1,100),
(2,300),
(3,50),
(4,3000)
I want to select first k entries such that the sum of val in those entries is just enough to make it to M.
For example, I want to find entries such that M = 425.
The result should be (1,100),(2,300),(3,50).
How can I do that in a mysql select query?
Try this variant -
SET #sum = 0;
SELECT id, val FROM (
SELECT *, #sum:=#sum + val mysum FROM mytable2 ORDER BY id
) t
WHERE mysum <= 450;
+------+------+
| id | val |
+------+------+
| 1 | 100 |
| 2 | 300 |
| 3 | 50 |
+------+------+
This stored procedure might help:
DELIMITER ;;
CREATE PROCEDURE selectLimitBySum (IN m INT)
BEGIN
DECLARE mTmp INT DEFAULT 0;
DECLARE idTmp INT DEFAULT 0;
DECLARE valTmp INT DEFAULT 0;
DECLARE doneLoop SMALLINT DEFAULT 0;
DECLARE crsSelect CURSOR FOR SELECT id, val FROM test3;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET doneLoop = 1;
OPEN crsSelect;
aloop: LOOP
SET idTmp = 0;
SET valTmp = 0;
FETCH crsSelect INTO idTmp, valTmp;
if doneLoop THEN
LEAVE aloop;
END IF;
SELECT idTmp, valTmp;
SET mTmp = mTmp + valTmp;
if mTmp > m THEN
LEAVE aloop;
END IF;
END LOOP;
CLOSE crsSelect;
END ;;
DELIMITER ;
Please feel free to change the table names or variable names as per your needs.
from mysql reference manual:
The LIMIT clause can be used to constrain the number of rows returned by the SELECT statement. LIMIT takes one or two numeric arguments, which must both be nonnegative integer constants (except when using prepared statements).
So you cannot use limit the way you proposed. To achieve what you want you need to use your application (java, c, php or whatever else), read the result set row by row, and stop when your condition is reached.
or you can use a prepared statement, but anyway you cant have conditional limit (it must be a constant value) and it is not exactly what you asked for.
create table #limit(
id int,
val int
)
declare #sum int, #id int, #val int, #m int;
set #sum=0;
set #m=250; --Value of an entry
declare limit_cursor cursor for
select id, val from your_table order by id
open limit_cursor
fetch next from limit_cursor into #id, #val
while(##fetch_status=0)
begin
if(#sum<#m)
begin
set #sum = #sum+#val;
INSERT INTO #limit values (#id, #val);
fetch next from limit_cursor into #id, #val
end
else
begin
goto case1;
end
end
case1:
close limit_cursor
deallocate limit_cursor
select * from #limit
truncate table #limit