I've been Googleing around for a while and I am sure that the problem is that I don't understand clearly how CURSORs in MySQL work.
A short explanation of the problem: I'm writing such function (simplified):
CREATE DEFINER=`me`#`localhost` FUNCTION `product_move`(prID INT, tr_type VARCHAR(2), clID INT, am INT, dnID INT, usrID INT, price FLOAT(10,2), ti DATETIME, barc TINYTEXT, cmt TINYTEXT, lnID INT)
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE cur_id INT;
DECLARE net_pr FLOAT(10,2);
DECLARE cur_r INT;
DECLARE remaind INT DEFAULT 0;
DECLARE avg_price FLOAT(10,2) DEFAULT 0;
DECLARE curs CURSOR FOR SELECT `products_transactionsID`,
`price`,
`remains`
FROM `products_transactions`
WHERE `productID`=prID AND `remains`>0 AND `type`='V'
ORDER BY `products_transactionsID` ASC;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN curs;
SET remaind=am;
read_loop:LOOP
FETCH curs INTO cur_id,net_pr,cur_r;
IF done THEN
LEAVE read_loop;
END IF;
IF (cur_r>=remaind) THEN
SET avg_price = avg_price + (net_pr * remaind);
UPDATE `products_transactions` SET `remains`=`remains`-remaind WHERE products_transactionsID=cur_id;
LEAVE read_loop;
ELSE
SET avg_price = avg_price + (net_pr * cur_r);
SET remaind=remaind-cur_r;
UPDATE `products_transactions` SET `remains`=0 WHERE products_transactionsID=cur_id;
END IF;
END LOOP;
CLOSE curs;
SET avg_price=avg_price/am;
INSERT INTO products_transactions
(`products_transactionsID`,`clientID`,`date_created`,`delivery_notesID`,`type`,`productID`,`amountIN`,`amountOUT`,`barcodes`,`in_stock`,`out_stock`,`out_repair`,`out_loss`,`booked`,`ordered`,`userID`,`price`,`comments`,`fifo_buy_price`)
SELECT NULL, clID, ti, dnID , tr_type, prID, 0, am, barc, products_transactions.in_stock-am, products_transactions.out_stock,
products_transactions.out_repair, products_transactions.out_loss, products_transactions.booked, products_transactions.ordered,usrID,price,cmt,avg_price
FROM
products_transactions WHERE productID=prID ORDER BY products_transactionsID DESC LIMIT 1;
So, we insert a new row in this table, based upon some calculations from the previously selected rows and updating these rows meanwhile.
The problem is with the avg_price variable, which should be calculated based on the net_pr variable which is FETCH'ed from the cursor. But somehow, instead of being FETCH'ed from the SELECT, the net_pr variable takes the value of the price input parameter of my function! How is that possible?
My guesses have been so far:
a variable name conflict? Searched through the code but I can't find any.
updating the table within the LOOP could make the CURSOR loose its position? It would make sense, but that wouldn't result in this, either...
I'd apreciate any ideas.
Two things that I can see:
1) Don't update the table that you're using in the cursor. MySQL says the cursor is read only but I wouldn't trust this. Set your value, exit the cursor, and then update the table.
2) Using the same name for a variable in the proc definition and a column in a select gives a conflict: http://dev.mysql.com/doc/refman/5.0/en/local-variable-scope.html
"A local variable should not have the same name as a table column. If an SQL statement, such as a SELECT ... INTO statement, contains a reference to a column and a declared local variable with the same name, MySQL currently interprets the reference as the name of a variable. "
Related
I want to add the salary from two tables in stored procedure on the basis of id column:
DDl:
create table salary1 (id varchar(20), salary varchar(20));
create table salary2 (id varchar(20), salary varchar(20));
DML:
insert into salary1 values('1', '100');
insert into salary1 values('2', '200');
insert into salary2 values('1', '10');
insert into salary2 values('2', '10');
Database: mysql
Output should like this:
id total_sal
1 110
2 210
My stored procedure look like:
CREATE PROCEDURE totalSal()
BEGIN
DECLARE tbl1_id varchar(30);
DECLARE tbl1_sal varchar(30);
DECLARE tbl2_id varchar(30);
DECLARE tbl2_sal varchar(30);
DECLARE total_sal varchar(30);
DECLARE c1 CURSOR FOR SELECT * FROM salary1;
DECLARE c2 CURSOR FOR SELECT * FROM salary2;
-- Open first cursor
OPEN c1;
LOOP
FETCH c1 INTO tbl1_id, tbl1_sal;
-- Open second cursor
OPEN c2;
LOOP
FETCH c2 INTO tbl2_id, tbl2_sal;
IF tbl1_id = tbl2_id THEN
set total_sal := tbl1_sal + tbl2_sal;
ELSE
set total_sal := tbl_sal;
END IF;
END LOOP;
CLOSE c2;
END LOOP;
CLOSE c1;
end $$
It got's successfully compiled, but when i am running the procedure i am getting the below error:
ERROR 1329 (02000): No data - zero rows fetched, selected, or processed
I have also used the DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1; in my procedure. but still my problem is unresolved.
If someone can solve this problem in oracle, that would also help me.
Note : I cannot perform join operation on these tables. Because of a few performance issues.
Thanks in advance !!!
Solution 1:
Using collection and only one iteration of 2 loop
You should consider to fix your performance issue on join. Performing loop is slower than a set base approach in most case.
If I follow your logic, what you realy want is to loop trough all the salary2 table for each salary1 row in order to found the right ID => millions of loop.
You can consider doing 2 separated loop and store data inside and indexed array. ( the key will be the tlb1_id).
If the key exist : sum the salary values, if not exist insert it inside the array.
At the end of the procedure, just select the array as table.
Solution 2:
Using a join on integer indexed columns
you can add a new integer column on each table
Populate this column with the casted value of the ID column
Add an index on these columns on each tables
After that you will be able to perform a join
Have a look at this fiddle http://sqlfiddle.com/#!9/c445de/1 , it can be time consuming to perform theses step and disk space consumuming to add a new columns and indexes but the join operation may be faster than before.
You can do something like this... I have moved the second cursor inside the loop so that it only goes over the id's from table 1. This should help the logic for the procedure but still I would recommend trying to figure out how to fix the join to get the results as that seems like an easier way and should be much faster if done correctly.
CREATE PROCEDURE totalSal()
BEGIN
DECLARE tbl1_id varchar(30);
DECLARE tbl1_sal varchar(30);
DECLARE tbl2_id varchar(30);
DECLARE tbl2_sal varchar(30);
DECLARE total_sal varchar(30);
DECLARE c1 CURSOR FOR SELECT * FROM salary1;
-- Open first cursor
OPEN c1;
LOOP
FETCH c1 INTO tbl1_id, tbl1_sal;
SELECT COUNT(*) INTO v_rowcount FROM salary2 WHERE id = tbl1_id;
IF v_rowcount > 0 THEN
Begin
DECLARE c2 CURSOR FOR SELECT * FROM salary2 WHERE id = tbl1_id;
-- Open second cursor
OPEN c2;
LOOP
FETCH c2 INTO tbl2_id, tbl2_sal;
IF tbl1_id = tbl2_id THEN
set total_sal := tbl1_sal + tbl2_sal;
ELSE
set total_sal := tbl_sal;
END IF;
END LOOP;
CLOSE c2;
END IF;
END
END LOOP;
CLOSE c1;
end $$
Well you asked for an answer without JOIN, but that seemed arbitrary, so here's an answer with JOIN.
SELECT
sums1.id
, S1Sum + S2Sum AS SalarySum
FROM (SELECT id, SUM(CAST(salary AS int)) AS S1Sum
FROM salary1
GROUP BY id) sums1
JOIN (SELECT id, SUM(CAST(salary AS int)) AS S2Sum
FROM salary2
GROUP BY id) sums2
ON sums1.id = sums2.id
I am guessing your performance is bad because all of your columns are varchar when they should be int or numeric. But we don't have much to go on so hopefully this helps you come to a solid solution.
Also the post was edited to add both MySQL and Oracle tags so it's difficult to determine what the syntax should be...
I am writing a MySQL Stored Procedure for the first time, and I am running into an issue - I think with the Handler Code. Basically, I want this code to update all rows in the pps_users table, but for some reason I am hitting the 'finished condition' for the handler after only two rows are fetched.
I tried the same thing with the REPEAT syntax and got the same result. If I just run the cursor query I correctly get the 10,000 records I expect, but when I run the whole thing as is, I hit the finished code after only 1 or 2 records.
DELIMITER $$
CREATE DEFINER=`root`#`localhost` PROCEDURE `changeNFLFavTeams`()
BEGIN
DECLARE favNFLTeam varchar(100) DEFAULT "";
DECLARE favNCAATeam varchar(100) DEFAULT "";
DECLARE v_finished INTEGER DEFAULT 0;
DECLARE user_id bigint(20);
DECLARE fullNameOfTeam varchar(100) DEFAULT "";
DECLARE update_favs CURSOR FOR select id, favorite_nfl_team from pps_users WHERE favorite_nfl_team is not null;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET v_finished = 1;
OPEN update_favs;
updaterecord: LOOP
FETCH update_favs INTO user_id, favNFLTeam;
select user_id, favNFLTeam as "Test";
if v_finished = 1
then
select "finished" as "finished";
LEAVE updaterecord;
end if;
select full_name into fullNameOfTeam
from teams t
inner join display_names dt on dt.entity_id = t.id
and dt.entity_type = 'teams'
and dt.first_name = favNFLTeam
and team_key like 'l.nfl.com%' LIMIT 1;
select user_id, fullNameOfTeam AS "BeforeUpdate";
IF fullNameOfTeam != ''
THEN
-- here for whatever_transformation_may_be_desired
-- Find the Full name for the record they chose
UPDATE pps_users p
SET favorite_nfl_team = fullNameOfTeam
WHERE user_id = p.id;
ELSE
SELECT 'A' AS 'A'; -- no op
END IF;
end loop updaterecord;
CLOSE update_favs;
END
This is because if your SELECT full_name into fullNameOfTeam... query returns no rows, then it will set v_finished to 1. That, apparently, happens early on, and forces an exit from the main loop.
The key is to realize that the CONTINUE HANDLER for NOT FOUND does not apply to the cursor alone.
You should either put the secondary query into its own BEGIN..END block with its own CONTINUE handler, or (easier) just set v_finished = 0 after the SELECT full_name into fullNameOfTeam... statement.
DELIMITER $$
CREATE PROCEDURE Load_Fact_List()
BEGIN
DECLARE Project_Number_Temp INT;
DECLARE Panel_Id_Temp INT;
DECLARE Employee_Id_Temp INT;
DECLARE Zip_Temp VARCHAR(255);
DECLARE Created_Date_Temp DATE;
DECLARE Country_Temp VARCHAR(255);
DECLARE no_more_rows BOOLEAN;
DECLARE loop_cntr INT DEFAULT 0;
DECLARE num_rows INT DEFAULT 0;
DECLARE load_cur CURSOR FOR
SELECT Project_Id, Panel_Id, Employee_Id, Zip, Created_Date
FROM Fact_List;
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
OPEN load_cur;
select FOUND_ROWS() into num_rows;
the_loop: LOOP
FETCH load_cur
INTO Project_Number_Temp, Panel_Id_Temp, Employee_Id_Temp, Zip_Temp, Created_Date_Temp;
IF no_more_rows THEN
CLOSE load_cur;
LEAVE the_loop;
END IF;
SET Country_Temp= (select Country from Zip where Zip= Zip_Temp);
INSERT INTO Test_Fact
(
Project_Key,
Campaign_Key,
Respondents_Key,
Event_Key,
Employee_Key,
Geography_Key,
Date_Key
)
SELECT (SELECT Project_Key from Project_Dim where Project_Id= Project_Number_Temp AND Quota_Country= Country_Temp),0,(SELECT MAX(Respondents_Key) from Respondents_Dim WHERE Panel_Id= Panel_Id_Temp),1,(select MAX(Employee_Key) from Employee_Dim WHERE Employee_Id= Employee_Id_Temp),(Select Geography_Key from Geography_Dim where Zip= Zip_Temp), (Select Date_Key from Date_Dim where Full_Date= Created_Date_Temp);
SET loop_cntr = loop_cntr + 1;
END LOOP the_loop;
select num_rows, loop_cntr;
END $$
The above code is properly working but it is damn slow. For every 1 hour it is loading 1000 records. I got lacks of records to load into fact table. can anyone suggest me any optimization?
Requirement is to load fact table by looping through other table and gathering required key values from dimension tables.
The usual procedure is actually like this.
You have your dimensions built and you just gathered the data you want to insert into your fact table in a temporary table. Then you insert this data in another temporary table like this:
INSERT INTO tmp_fact_table
(
fact_key,
dim1_key,
dim2_key,
...
fact1,
fact2
...
)
SELECT
ISNULL (f.fact_key, 0),
ISNULL (d1.sid, 0) as whatever,
ISNULL (d2.sid, 0) as whatever2,
...
ISNULL (tt.fact1, 0),
ISNULL (tt.fact2, 0)
FROM
yourTempTable tt
LEFT JOIN Dim1 d1 ON tt.identifying_column = d1.identifying_column
...
LEFT JOIN fact_table f ON
f.dim1_key = d1.sid
AND f.dim2_key = d2.sid
where
fact_key is the identifying column in your fact table
dim1_key is the foreign key in your fact table to the dimensions
fact1 and so on are the facts you want in your fact table, clear
the ISNULL() function returns 0 when no entry is found. 0 is the id of your dummy row in each dimension for unknown data
Then you will have a table where you have the IDs of your dimensions linked to the data you want to import into your fact table with 0 as fact key when the entry in the fact table does not already exist and the ID of the fact table entry otherwise.
Then you update the fact table where tmp_fact_table.fact_key != 0
Then you insert into the fact table where tmp_fact_table.fact_key = 0
That's it.
I'm doing this with millions of rows and it takes about half an hour. 300,000 rows is peanuts.
Am new to Stored procedures.I wrote the stored procedure to copy table from one dtabase to another database.On executing my stored procedures everytime My datas are added in the destination table .My row counts was increasing on every execution.
Please help to resolve the issue.Hope the problem In the loops.
My SP is:
--exec mall
alter procedure mall
as
begin
declare #mallid int
declare #mallname nvarchar(40)
declare #mallstatus nvarchar(40)
declare #malludsuomid nchar(2)
declare #malludsassetcode nvarchar(6)
declare #malludsassettype nvarchar(15)
declare #malludsremarks nvarchar(max)
declare #malludsdwdb int
declare #mallsecterr int
declare #mallassetid int
declare #secterr int
declare #Maxmallid int
declare #mallentityid int
Select #mallentityid = customtable.Bord_TableId From CRM.dbo.Custom_Tables as customtable With (NoLock) Where Upper(Bord_Caption) = Upper('Mall') And Bord_Deleted Is Null
DECLARE cur_address CURSOR FOR
SELECT
udsasset.Asset_ID,udsasset.Asset_Name,udsasset.Asset_Status,udsasset.UOM_ID, udsasset.Asset_Code,udsasset.Asset_Type,udsasset.Remarks,udsasset.DW_Key_Source_DB --,crmterr.TPro_SecTerr
from
CMA_UDS.dbo.Dim_Asset as udsasset
OPEN cur_address
FETCH NEXT FROM cur_address INTO #mallid,#mallname,#mallstatus,#malludsuomid,#malludsassetcode,#malludsassettype,#malludsremarks,#malludsdwdb --,#mallsecterr
WHILE ##FETCH_STATUS = 0
BEGIN
if not exists (select crmmall.mall_MallID from CRM.dbo.Mall as crmmall where crmmall.mall_MallID = #mallid)
begin
exec #Maxmallid = CRM.dbo.crm_next_id #Table_Id=#mallentityid
insert into
CRM.dbo.Mall
(mall_MallID,mall_Name,mall_Status,mall_uds_UOMID,mall_uds_asset_code,mall_uds_asset_type,
mall_uds_remarks,mall_uds_dw_db,mall_CreatedBy,mall_CreatedDate,mall_Secterr,mall_AMOSUploaded,mall_asset_id)
values(#Maxmallid,#mallname,#mallstatus,#malludsuomid,#malludsassetcode,#malludsassettype,#malludsremarks,#malludsdwdb,1,GETDATE(),
#mallsecterr,GETDATE(),#mallid)
end
else
begin
update
CRM.dbo.Mall
set
mall_asset_id=#mallid,mall_Name = #mallname,mall_Status=#mallstatus,mall_uds_UOMID =#malludsuomid,mall_uds_asset_code=#malludsassetcode,
mall_uds_asset_type=#malludsassettype,mall_uds_remarks=#malludsremarks,mall_uds_dw_db=#malludsdwdb,mall_UpdatedBy=1,
mall_UpdatedDate=GETDATE(),mall_Secterr=#mallsecterr,mall_AMOSUploaded=GETDATE()
where
mall_MallID=#mallid
end
FETCH NEXT FROM cur_address INTO #mallid,#mallname,#mallstatus,#malludsuomid,#malludsassetcode,#malludsassettype,#malludsremarks,#malludsdwdb--,#mallsecterr
end
CLOSE cur_address
DEALLOCATE cur_address
End
Why are you inserting crm_next_id as the value in mall_MallID, but using that same id to compare with #mallid to see if the record is already inserted? For example, if you have id 5, and you insert a new record with id 150, it's not going to see that the record is already inserted when you run the SP again. Next run, it will add record with id 151, then 152, and so forth forever. You shouldn't use the same field as both an auto-increment identity and a foreign key reference at the same time...
You either need to use the same #mallid when you insert the new records so they match, or after you generate a new id and insert into the table, update the original record CMA_UDS.dbo.Dim_Asset to have Asset_ID = #mallid so they are linked up properly. Which method you use depends on the meanings of those id's and what constraints you have in your particular application.
In MySQL, I am converting a table from a single row per item type (a quantity of items) to a single row per item, so that additional detail can be stored about individual items.
Here is an example source table:
id parent_id qty item_type
-- --------- --- ---------
1 10291 2 widget
2 10292 4 thinger
I want to create a new table with a new column containing info that cannot be applied to more than one item. Thus, the above table would end up as follows:
id parent_id item_type info
-- --------- --------- ----
1 10291 widget [NULL]
2 10291 widget [NULL]
3 10292 thinger [NULL]
4 10292 thinger [NULL]
5 10292 thinger [NULL]
6 10292 thinger [NULL]
Is there a way I can iterate or loop each row of the source table, inserting a number of records equal to the source qty column? I would prefer to do this in sql instead of code to keep all of the conversion steps together (there are many others).
You can do with stored procedure. That will be like below. Below is stored procedure I am using for inserting products into log based on their quantity.
Seem you have to do similar task. You can get how to use database cursor in stored procedure to loop over a result set in MySQL from below example.
DELIMITER $$
DROP PROCEDURE IF EXISTS CursorProc$$
CREATE PROCEDURE CursorProc()
BEGIN
DECLARE no_more_products, quantity_in_stock INT DEFAULT 0;
DECLARE prd_code VARCHAR(255);
DECLARE cur_product CURSOR FOR
SELECT productCode FROM products;
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_products = 1;
/* for loggging information */
CREATE TABLE infologs (
Id int(11) NOT NULL AUTO_INCREMENT,
Msg varchar(255) NOT NULL,
PRIMARY KEY (Id)
);
OPEN cur_product;
FETCH cur_product INTO prd_code;
REPEAT
SELECT quantityInStock INTO quantity_in_stock
FROM products
WHERE productCode = prd_code;
IF quantity_in_stock < 100 THEN
INSERT INTO infologs(msg)
VALUES (prd_code);
END IF;
FETCH cur_product INTO prd_code;
UNTIL no_more_products = 1
END REPEAT;
CLOSE cur_product;
SELECT * FROM infologs;
DROP TABLE infologs;
END$$
DELIMITER;
Seems your task is 90% same as above procedure. Just do needful changes. It will work.
I think you can create stored procedure, declare a cursor that reads source table and for each row inserts qty rows into destination table.
Based on other answers which provided some insight, I was able to find additional information (by Kevin Bedell) to create a stored procedure and use a cursor in a loop. I have simplified my solution so that it matches the example in my question:
DROP PROCEDURE IF EXISTS proc_item_import;
DELIMITER $$
CREATE PROCEDURE proc_item_import()
BEGIN
# Declare variables to read records from the cursor
DECLARE parent_id_val INT(10) UNSIGNED;
DECLARE item_type_val INT(10) UNSIGNED;
DECLARE quantity_val INT(3);
# Declare variables for cursor and loop control
DECLARE no_more_rows BOOLEAN;
DECLARE item_qty INT DEFAULT 0;
# Declare the cursor
DECLARE item_cur CURSOR FOR
SELECT
i.parent_id, i.qty, i.item_type
FROM items i;
# Declare handlers for exceptions
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_rows = TRUE;
# Open the cursor and loop through results
OPEN item_cur;
input_loop: LOOP
FETCH item_cur
INTO parent_id_val, item_type_val, quantity_val;
# Break out of the loop if there were no records or all have been processed
IF no_more_rows THEN
CLOSE item_cur;
LEAVE input_loop;
END IF;
SET item_qty = 0;
qty_loop: LOOP
INSERT INTO items_new
(parent_id, item_type)
SELECT
parent_id_val, item_type_val;
SET item_qty = item_qty + 1;
IF item_qty >= quantity_val THEN
LEAVE qty_loop;
END IF;
END LOOP qty_loop;
END LOOP input_loop;
END$$
DELIMITER ;
Before asking this question, I had not used a stored procedures, cursors, or loops. That said, I have read and encountered them frequently on SE and elsewhere, and this was a good opportunity to learn
It may be worth noting that the example on Kevin's page (linked above) does not use END%% (just END) which caused some headache in trying to get the script to work. When creating a procedure, it is necessary to change the delimiter temporarily so that semicolons terminate statements inside the procedure, but not the creation process of the procedure itself.
That is just an example of code that I have here, it is not adapted to your needs, but it does exactly what you need, and it is simple than a procedure, or temporary table.
SELECT event, id, order_ref, storeitem_barcode_create(8), NOW()
FROM (
SELECT mss.id, mss.event, mss.order_ref, mss.quantity, mss.product_id,
#rowID := IF(#lastProductID = mss.product_id AND #lastID = mss.id, #rowID + 1, 0) AS rowID,
#lastProductID := mss.product_id,
#lastID := mss.id
FROM module_barcode_generator mbg,
(SELECT #rowID := 0, #lastProductID := 0, #lastID := 0) t
INNER JOIN module_events_store_sold mss ON mss.order_ref = "L18T2P"
) tbl
WHERE rowId < quantity;
Typo in JYelton's solution for his/her own question:
FETCH item_cur
INTO parent_id_val, item_type_val, quantity_val;
Should be:
FETCH item_cur
INTO parent_id_val, quantity_val, item_type_val;
Otherwise very good.