MySQL Stored Procedure Unexpected Behaviour in certain conditions - mysql

I am relatively new to MySQL stored Procedures. I have a stored procedure that works fine in certain conditions and not in other. I'm a bit confused that what causes the error. It is a entity processing SP, based on certain values and conditions it either creates or update entity from data in a staging table.
In conditions when it works fine:
When I only process a single entity.
Works for bulk entries when there is nothing in entity table.
Don't works when:
There are entries in entity table and bulk processing is done. - Error in checking up the already existing entity_id in entity and older entity_id is assigned instead a new one should be created.
(I actually need above scenario to work more frequently than others)
I have tried to keep the code to minimum in order to understand the flow of SP. Please consider all variables as declared. SP might not compile.
CREATE DEFINER=`admin`#`%` PROCEDURE `sp_ent`(test_id int)
BEGIN
-- Move code tables to temp tables
-- Declare all the required variables here
DECLARE counter, len INT;
DECLARE var2 INT;
DECLARE var3 TINYINT(1);
DECLARE var4 DECIMAL(18,4);
DECLARE var5 DATE;
DECLARE var6 VARCHAR(1000);
DECLARE var7 VARCHAR(5000);
DROP TEMPORARY TABLE IF EXISTS `temp_ent`;
IF test_id IS NOT NULL THEN
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_ent` AS (SELECT * FROM `ent_st` WHERE processed = 0 and id=test_id);
ELSE
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_ent` AS (SELECT * FROM `ent_st` WHERE processed = 0);
END IF;
ALTER TABLE `temp_ent` ADD PRIMARY KEY(id);
-- SELECT * FROM `temp_ent`;
DROP TEMPORARY TABLE IF EXISTS `temp_exc`; CREATE TEMPORARY TABLE IF NOT EXISTS `temp_exc` AS (SELECT * FROM `code_exc`);
-- A few more like above
SET counter=1, len=(SELECT COUNT(*) FROM `temp_ent`);
WHILE counter <= len DO
SELECT `id`,var1, var2, var3
INTO v_id,var1, var2, var3
FROM `temp_ent` LIMIT 1;-- WHERE `id` = v_id;
BEGIN
DECLARE insufficient_information CONDITION FOR SQLSTATE '45000';
DECLARE CONTINUE HANDLER FOR insufficient_information SET v_proccessed=1;
SET v_status = CASE
WHEN ... THEN ...
ELSE 'valid'
END;
IF v_status <> 'valid' THEN SIGNAL insufficient_information; END IF;
SELECT `entity_id`,`entity`
INTO v_underlying_entity_id,v_underlying_entity_symbol
FROM `entity` WHERE `entity_id` = v_underlying OR `entity` = v_underlying_entity;
SET v_underlying_entity_symbol = COALESCE(v_underlying_entity_symbol,v_underlying_entity);
SET v_entity = (
CASE
WHEN ... THEN ...
WHEN ... THEN ...
.
.
WHEN ... THEN ...
END
);
SELECT `entity_id`,`entity`
INTO v_entity_id_check,v_entity_check
FROM entity WHERE `entity`=v_entity and `exc`=v_exc;
-- SELECT v_entity_id_check,v_entity_check,v_entity,v_exc;
IF v_entity_check IS NULL THEN
-- SELECT 'Create New Entity and Add';
SET v_entity_id = COALESCE((SELECT MAX(`entity_id`) FROM entity),0) + 1;
SET new_entity = 1;
ELSE
-- SELECT 'Entity Already Present';
SET v_entity_id = v_entity_id_check;
SET new_entity = 0;
END IF;
-- SELECT v_entity_id, v_entity, new_entity;
SET v_name = UPPER(COALESCE(v_name,v_entity));
-- UPDATE entity and underlying/derivatives table
IF new_entity = 1 THEN
-- Insert in respective tables
IF ... THEN
-- SELECT 'Entity Added',v_entity; -- Underlying Entity
INSERT INTO `entity_underlying` (`entity_id`,`entity`,`name`,`exc`,`seg`,`ins`,`isin`,`fo_yn`,`tick`,`lot_size`,`active_yn`)
VALUES (v_entity_id,v_entity,v_name,v_exc,v_seg,v_ins,COALESCE(v_isin,''),COALESCE(v_fo_yn,0),COALESCE(v_tick,0.05),COALESCE(v_lot_size,1),1);
ELSEIF ... THEN
-- SELECT 'Entity Added',v_entity; -- Derivative Entity
INSERT INTO `entity_derivatives` (`entity_id`,`entity`,`name`,`underlying`,`exc`,`seg`,`ins`,`ser`,`isin`,`strike`,`tick`,`lot_size`,`expiry`,`ex_ty`,`active_yn`)
VALUES (v_entity_id,v_entity,v_name,COALESCE(v_underlying_entity_id,-1),v_exc,v_seg,v_ins,COALESCE(v_ser,''),v_isin,COALESCE(v_strike),COALESCE(v_tick,0.05),COALESCE(v_lot_size,-1),v_expiry,v_ex_ty,1);
END IF;
-- Insert in final table
INSERT INTO entity(`entity_id`,`entity`,`exc`,`active_yn`)
VALUES (v_entity_id,v_entity,v_exc,1);
SET v_status='Entity Added';
ELSE
SET v_status='Not a New Entity';
END IF;
END;
UPDATE `ent_st` SET `entity_id`=v_entity_id,`processed`=1,`status`=v_status WHERE id=v_id;
DELETE FROM `temp_ent` WHERE id=v_id;
SET counter = counter +1;
COMMIT;
END WHILE;
END
Any help is highly appreciated. Thanks in advance.

Related

error in stored procedure ...into keyword

Two tables Borrower(rollno,name,bookissue_date) and Fine(rollno,name,amount)
delimiter //
create procedure student( in roll_no int,in Nameofbook varchar(40))
begin
declare Dateofiss1 date;
Declare cur cursor for
select Dateofiss from Borrower where Roll_no = roll into Dateofiss1;
OPEN cur;
fetch cur into Dateofiss1
if(datediff(sysdate(),Dateofiss1)<15) then varchar(20))
update Borrower set status='R'where Roll_no=roll_no
elseif(datediff(sysdate(),Dateofiss1)>=15)and datediff (sysdate(),Dateofiss1<30)
SET FINEAMOUNT=5*(datediff(sysdate(),Dateofiss1)-15)
insert into Fine(Roll_no,Date,amount)values(rollno,sysdate,fineamount);
update.borrower set status='R' where Roll_no='rollno';
elseif (datediff(sysdate(),Dateofiss1)>30)
SET FINEAMOUNT=50*(datediff(sysdate(),Dateofiss1)-15)
insert into Fine(Roll_no,Date,amount)values(rollno,sysdate,fineamount);
update.borrower set status='R' where Roll_no='rollno';
close cur;
end if
select * from Borrower;
elect * from Fine;
end
You have a number of syntax errors.
You have an extraneous varchar(20)) in the first if statement.
You're missing THEN in the ELSEIF statements.
You wrote update.borrower instead of update borrower.
You have roll_no in quotes in some of your update statements.
The roll_no parameter is the same as a table column, since column names are case-insensitive. The condition where Roll_no = roll_no will match every row because of this. Give the parameter a different name.
In a SELECT, the INTO clause goes after FROM, not at the end.
There's no need to use a cursor if you're using SELECT INTO. Just execute the query and it will set the variable.
You can also simplify the code by putting the date difference in a variable, so you don't have to repeatedly calculate it. And in the ELSEIF you don't need to test >= 15, since you'll only get there if the < 15 test failed.
The UPDATE statement is the same in all conditions, so it doesn't need to be in the IF at all.
delimiter //
create procedure student( in p_roll_no int,in Nameofbook varchar(40))
begin
declare Dateofiss1 date;
declare diff INT;
select Dateofiss from Borrower into Dateofiss1 where Roll_no = p_roll_no;
OPEN cur;
SET diff = datediff(sysdate(),Dateofiss1)
IF diff BETWEEN 15 AND 29 THEN
SET FINEAMOUNT= 5 * (diff - 15)
insert into Fine(Roll_no,Date,amount)values(rollno,sysdate,fineamount);
else
SET FINEAMOUNT= 50 * (diff - 15)
insert into Fine(Roll_no,Date,amount)values(rollno,sysdate,fineamount);
end if
update Borrower set status='R'where Roll_no=p_roll_no
select * from Borrower;
select * from Fine;
end

Foreach Data in Field Insert Selected Field from One Database to Another in MySQL

I have two (2) databases of dissimilar Schematics,
db1 migrated from MSSQL to MYSQL
and
db2 created from Laravel Migration.
Here's the challenge:
The tables of db1 do not have id columns (Primary Key) like is easily found on db2 tables. So I kept getting the warning message:
Current selection does not contain a unique column. Grid edit, checkbox, Edit, Copy and Delete features are not available.
So I had to inject the id columns on the tables in the db1
I need to extract fields [level_name, class_name] from stdlist in db1,
Create levels (id,level_name,X,Y) on db2
classes (id,class_name,level_id) on db2
To throw more light: The level_id should come from the already created levels table
I have already succeeded in extracting the first instance using the following snippet:
First Query to Create Levels
INSERT INTO db2.levels(level_name,X,Y)
SELECT class_name as level_name,1 as X,ClassAdmitted as Y
FROM db1.stdlist
GROUP BY ClassAdmitted;
This was successful.
Now, I need to use the newly created ids in levels table to fill up level_id column in the classes table.
For that to be possible, must I re-run the above selection schematics? Is there no better way to maybe join the table column from db1.levels to db2.stdlist and extract the required fields for the new insert schematics.
I'll appreciate any help. Thanks in advance.
Try adding a column for Processed and then do a while exists loop
INSERT INTO db2.levels(level_name,X,Y)
SELECT class_name as level_name,1 as X,ClassAdmitted as Y, 0 as Processed
FROM db1.stdlist
GROUP BY ClassAdmitted;
WHILE EXISTS(SELECT * FROM db2.levels WHERE Processed = 0)
BEGIN
DECLARE #level_name AS VARCHAR(MAX)
SELECT TOP 1 #level_name=level_name FROM db2.levels WHERE Processed = 0
--YOUR CODE
UPDATE db2.levels SET Processed=1 WHERE level_name=#level_name
END
You may need to dump into a temp table first and then insert into your real table (db2.levels) when you're done processing. Then you wouldn't need the Unnecessary column of processed on the final table.
This is what worked for me eventually:
First, I picked up the levels from the initial database thus:
INSERT INTO db2.levels(`name`,`school_id`,`short_code`)
SELECT name ,school_id,short_code
FROM db1.levels
GROUP BY name
ORDER BY CAST(IF(REPLACE(name,' ','')='','0',REPLACE(name,' ','')) AS UNSIGNED
INTEGER) ASC;
Then I created a PROCEDURE for the classes insertion
CREATE PROCEDURE dowhileClasses()
BEGIN
SET #Level = 1;
SET #Max = SELECT count(`id`) FROM db2.levels;
START TRANSACTION;
WHILE #Level <= #Max DO
BEGIN
DECLARE val1 VARCHAR(255) DEFAULT NULL;
DECLARE val2 VARCHAR(255) DEFAULT NULL;
DECLARE bDone TINYINT DEFAULT 0;
DECLARE curs CURSOR FOR
SELECT trim(`Class1`)
FROM db1.dbo_tblstudent
WHERE CAST(IF(REPLACE(name,' ','')='','0',REPLACE(name,' ','')) AS UNSIGNED INTEGER) =#Level
GROUP BY `Class1`;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET bDone = 1;
OPEN curs;
SET bDone = 0;
REPEAT
FETCH curs INTO val1;
IF bDone = 0 THEN
SET #classname = val1;
SET #levelID = (SELECT id FROM db2.levels WHERE short_code=#Level limit 1);
SET #schoolId = 1;
SET #classId = (SELECT `id` FROM db2.classes where class_name = #classname and level_id= #levelID limit 1);
IF #classId is null and #classname is not null THEN
INSERT INTO db2.classes(class_name,school_id,level_id)
VALUES(#classname,#schoolId,#levelID);
END IF;
END IF;
UNTIL bDone END REPEAT;
CLOSE curs;
END;
SELECT CONCAT('lEVEL: ',#Level,' Done');
SET #Level = #Level + 1;
END WHILE;
END;
//
delimiter ;
CALL dowhileClasses();
With this, I was able to dump The classes profile matching the previously created level_ids.
The whole magic relies on the CURSOR protocol.
For further details here is one of the documentations I used.

declaring variable inside mysql stored procedure

we are trying to declare a variable inside mysql stored procedure that has transaction implemented in it. but it seems to be giving a syntax error :
following is the syntax of the stored procedure:
CREATE PROCEDURE `sp_MarkAppointmentRefferal`(
p_AppId bigint,
p_NewLocation bigint,
p_userId bigint,
p_ReferralReason varchar(500),
p_NewLocationName varchar(100)
)
begin
declare v_OldLocation int default 0;
set v_OldLocation = (select LocationId FROM appointments where iAppID = p_AppId limit 1 );
DECLARE EXIT HANDLER FOR SQLEXCEPTION
BEGIN
ROLLBACK;
select -1;
END;
START TRANSACTION;
update table
set is_referred = 1,
referred_timestamp = now(),
referral_reason = p_ReferralReason
where iAppID = p_AppId
limit 1;
-- create a new appointment for the new referred location..
insert into appointments
(vAppName, vAppType, dAppDate, vCell, iPatID, iAppStatus, iuserid, iActive,
dInsertDate, iHSID, daily_ticket_no, LocationId, visit_id, encounter_id, ReferredFrom,ReferredOPDName, opd_name )
select vAppName, vAppType, now(), vCell, iPatID, iAppStatus, p_userId,
1, now(), iHSID, fn_GenerateNextAppointmentTicket(now(),p_NewLocation) , p_NewLocation, visit_id, encounter_id+1,
(select LocationId FROM appointments where iAppID = p_AppId limit 1),
(select OPD_Name FROM appointments where iAppID = p_AppId limit 1), p_NewLocationName
FROM appointments
where iAppID = p_AppId limit 1;
select LAST_INSERT_ID();
COMMIT;
end;
the syntax checker is saying that declare command is not valid here.
have also tried to place this inside the transaction clause and similar error shows up ..
any help is appreciated..
All declare statements should be at the top of the stored procedure body. Moving DECLARE EXIT HANDLER before the SET statement should fix the problem.

Stored Procedure taking ages to execute?

DELIMITER $$
CREATE PROCEDURE Load_Fact_List()
BEGIN
DECLARE Project_Number_Temp INT;
DECLARE Panel_Id_Temp INT;
DECLARE Employee_Id_Temp INT;
DECLARE Zip_Temp VARCHAR(255);
DECLARE Created_Date_Temp DATE;
DECLARE Country_Temp VARCHAR(255);
DECLARE no_more_rows BOOLEAN;
DECLARE loop_cntr INT DEFAULT 0;
DECLARE num_rows INT DEFAULT 0;
DECLARE load_cur CURSOR FOR
SELECT Project_Id, Panel_Id, Employee_Id, Zip, Created_Date
FROM Fact_List;
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
OPEN load_cur;
select FOUND_ROWS() into num_rows;
the_loop: LOOP
FETCH load_cur
INTO Project_Number_Temp, Panel_Id_Temp, Employee_Id_Temp, Zip_Temp, Created_Date_Temp;
IF no_more_rows THEN
CLOSE load_cur;
LEAVE the_loop;
END IF;
SET Country_Temp= (select Country from Zip where Zip= Zip_Temp);
INSERT INTO Test_Fact
(
Project_Key,
Campaign_Key,
Respondents_Key,
Event_Key,
Employee_Key,
Geography_Key,
Date_Key
)
SELECT (SELECT Project_Key from Project_Dim where Project_Id= Project_Number_Temp AND Quota_Country= Country_Temp),0,(SELECT MAX(Respondents_Key) from Respondents_Dim WHERE Panel_Id= Panel_Id_Temp),1,(select MAX(Employee_Key) from Employee_Dim WHERE Employee_Id= Employee_Id_Temp),(Select Geography_Key from Geography_Dim where Zip= Zip_Temp), (Select Date_Key from Date_Dim where Full_Date= Created_Date_Temp);
SET loop_cntr = loop_cntr + 1;
END LOOP the_loop;
select num_rows, loop_cntr;
END $$
The above code is properly working but it is damn slow. For every 1 hour it is loading 1000 records. I got lacks of records to load into fact table. can anyone suggest me any optimization?
Requirement is to load fact table by looping through other table and gathering required key values from dimension tables.
The usual procedure is actually like this.
You have your dimensions built and you just gathered the data you want to insert into your fact table in a temporary table. Then you insert this data in another temporary table like this:
INSERT INTO tmp_fact_table
(
fact_key,
dim1_key,
dim2_key,
...
fact1,
fact2
...
)
SELECT
ISNULL (f.fact_key, 0),
ISNULL (d1.sid, 0) as whatever,
ISNULL (d2.sid, 0) as whatever2,
...
ISNULL (tt.fact1, 0),
ISNULL (tt.fact2, 0)
FROM
yourTempTable tt
LEFT JOIN Dim1 d1 ON tt.identifying_column = d1.identifying_column
...
LEFT JOIN fact_table f ON
f.dim1_key = d1.sid
AND f.dim2_key = d2.sid
where
fact_key is the identifying column in your fact table
dim1_key is the foreign key in your fact table to the dimensions
fact1 and so on are the facts you want in your fact table, clear
the ISNULL() function returns 0 when no entry is found. 0 is the id of your dummy row in each dimension for unknown data
Then you will have a table where you have the IDs of your dimensions linked to the data you want to import into your fact table with 0 as fact key when the entry in the fact table does not already exist and the ID of the fact table entry otherwise.
Then you update the fact table where tmp_fact_table.fact_key != 0
Then you insert into the fact table where tmp_fact_table.fact_key = 0
That's it.
I'm doing this with millions of rows and it takes about half an hour. 300,000 rows is peanuts.

How to insert multiple rows based on a quantity value in one row?

In MySQL, I am converting a table from a single row per item type (a quantity of items) to a single row per item, so that additional detail can be stored about individual items.
Here is an example source table:
id parent_id qty item_type
-- --------- --- ---------
1 10291 2 widget
2 10292 4 thinger
I want to create a new table with a new column containing info that cannot be applied to more than one item. Thus, the above table would end up as follows:
id parent_id item_type info
-- --------- --------- ----
1 10291 widget [NULL]
2 10291 widget [NULL]
3 10292 thinger [NULL]
4 10292 thinger [NULL]
5 10292 thinger [NULL]
6 10292 thinger [NULL]
Is there a way I can iterate or loop each row of the source table, inserting a number of records equal to the source qty column? I would prefer to do this in sql instead of code to keep all of the conversion steps together (there are many others).
You can do with stored procedure. That will be like below. Below is stored procedure I am using for inserting products into log based on their quantity.
Seem you have to do similar task. You can get how to use database cursor in stored procedure to loop over a result set in MySQL from below example.
DELIMITER $$
DROP PROCEDURE IF EXISTS CursorProc$$
CREATE PROCEDURE CursorProc()
BEGIN
DECLARE no_more_products, quantity_in_stock INT DEFAULT 0;
DECLARE prd_code VARCHAR(255);
DECLARE cur_product CURSOR FOR
SELECT productCode FROM products;
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_products = 1;
/* for loggging information */
CREATE TABLE infologs (
Id int(11) NOT NULL AUTO_INCREMENT,
Msg varchar(255) NOT NULL,
PRIMARY KEY (Id)
);
OPEN cur_product;
FETCH cur_product INTO prd_code;
REPEAT
SELECT quantityInStock INTO quantity_in_stock
FROM products
WHERE productCode = prd_code;
IF quantity_in_stock < 100 THEN
INSERT INTO infologs(msg)
VALUES (prd_code);
END IF;
FETCH cur_product INTO prd_code;
UNTIL no_more_products = 1
END REPEAT;
CLOSE cur_product;
SELECT * FROM infologs;
DROP TABLE infologs;
END$$
DELIMITER;
Seems your task is 90% same as above procedure. Just do needful changes. It will work.
I think you can create stored procedure, declare a cursor that reads source table and for each row inserts qty rows into destination table.
Based on other answers which provided some insight, I was able to find additional information (by Kevin Bedell) to create a stored procedure and use a cursor in a loop. I have simplified my solution so that it matches the example in my question:
DROP PROCEDURE IF EXISTS proc_item_import;
DELIMITER $$
CREATE PROCEDURE proc_item_import()
BEGIN
# Declare variables to read records from the cursor
DECLARE parent_id_val INT(10) UNSIGNED;
DECLARE item_type_val INT(10) UNSIGNED;
DECLARE quantity_val INT(3);
# Declare variables for cursor and loop control
DECLARE no_more_rows BOOLEAN;
DECLARE item_qty INT DEFAULT 0;
# Declare the cursor
DECLARE item_cur CURSOR FOR
SELECT
i.parent_id, i.qty, i.item_type
FROM items i;
# Declare handlers for exceptions
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
# Open the cursor and loop through results
OPEN item_cur;
input_loop: LOOP
FETCH item_cur
INTO parent_id_val, item_type_val, quantity_val;
# Break out of the loop if there were no records or all have been processed
IF no_more_rows THEN
CLOSE item_cur;
LEAVE input_loop;
END IF;
SET item_qty = 0;
qty_loop: LOOP
INSERT INTO items_new
(parent_id, item_type)
SELECT
parent_id_val, item_type_val;
SET item_qty = item_qty + 1;
IF item_qty >= quantity_val THEN
LEAVE qty_loop;
END IF;
END LOOP qty_loop;
END LOOP input_loop;
END$$
DELIMITER ;
Before asking this question, I had not used a stored procedures, cursors, or loops. That said, I have read and encountered them frequently on SE and elsewhere, and this was a good opportunity to learn
It may be worth noting that the example on Kevin's page (linked above) does not use END%% (just END) which caused some headache in trying to get the script to work. When creating a procedure, it is necessary to change the delimiter temporarily so that semicolons terminate statements inside the procedure, but not the creation process of the procedure itself.
That is just an example of code that I have here, it is not adapted to your needs, but it does exactly what you need, and it is simple than a procedure, or temporary table.
SELECT event, id, order_ref, storeitem_barcode_create(8), NOW()
FROM (
SELECT mss.id, mss.event, mss.order_ref, mss.quantity, mss.product_id,
#rowID := IF(#lastProductID = mss.product_id AND #lastID = mss.id, #rowID + 1, 0) AS rowID,
#lastProductID := mss.product_id,
#lastID := mss.id
FROM module_barcode_generator mbg,
(SELECT #rowID := 0, #lastProductID := 0, #lastID := 0) t
INNER JOIN module_events_store_sold mss ON mss.order_ref = "L18T2P"
) tbl
WHERE rowId < quantity;
Typo in JYelton's solution for his/her own question:
FETCH item_cur
INTO parent_id_val, item_type_val, quantity_val;
Should be:
FETCH item_cur
INTO parent_id_val, quantity_val, item_type_val;
Otherwise very good.