Very slow execution when calling function - mysql

I have two functions. The second function uses the output from the first function.
One is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitfactor;
CREATE FUNCTION fp_splitfactor_price (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
SELECT IFNULL(EXP(SUM(LOG(f.p_split_factor))),1) INTO splitfactor
FROM fp_v2_fp_basic_splits AS f
WHERE f.fsym_id = id AND f.p_split_date > startdate AND f.p_split_date < NOW();
RETURN splitfactor;
END$$
DELIMiTER ;
Second one is:
DELIMITER $$
DROP FUNCTION IF EXISTS fp_splitadjprice;
CREATE FUNCTION fp_splitadjprice (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
DECLARE splitadjprice FLOAT;
DECLARE spinofffactor FLOAT;
SET splitfactor = 1.0;
SELECT fp_splitfactor(id, startdate) INTO splitfactor;
SELECT (p_price * splitfactor) INTO splitadjprice
FROM fp_v2_fp_basic_prices
WHERE fsym_id = id AND p_date = startdate;
RETURN splitadjprice;
END$$
DELIMITER ;
I then try to exectute a query as the following:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
b.region AS Region,
p.p_date,
p.p_price AS Unadjusted_Price,
fp_splitadjprice(p.fsym_id,p_date) AS Adjusted_Price
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id
So basically my query calls the second function, which then calls the first function in order to return a value to the query. The execution is extremely slow though, but I do not understand why that is the case?

I found out that the slow execution was entire due to MySQL workbench not handling large datasets well. Once I migrated everything to BigQuery on Google Cloud everything worked perfectly.
STAY AWAY FROM CALLING FUNCTIONS ON LARGE DATASETS IN MySQL Workbench!

Related

Slow nested function

Please let me know what information you seek to improve the question, rather than just downvoting.
I have a function that looks like this:
DELIMITER $$
DROP FUNCTION IF EXISTS f_splitadjprice;
CREATE FUNCTION f_splitadjprice (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
DECLARE splitadjprice FLOAT;
SELECT f_splitfactor(id, startdate) INTO splitfactor;
SELECT (f.p_price FROM fp_v2_fp_basic_prices as f WHERE f.fsym_id = id AND
f.p_date = startdate) * splitfactor INTO splitadjprice;
RETURN splitadjprice;
END$$
DELIMITER ;
The function for splitfactor is:
DELIMITER $$
DROP FUNCTION IF EXISTS f_splitfactor;
CREATE FUNCTION f_splitfactor (id CHAR(8), startdate DATE)
RETURNS FLOAT
BEGIN
DECLARE splitfactor FLOAT;
SELECT IFNULL(EXP(SUM(LOG(f.p_split_factor))),1) INTO splitfactor
FROM fp_v2_fp_basic_splits AS f
WHERE f.fsym_id = id AND f.p_split_date > startdate AND f.p_split_date <
NOW();
RETURN splitfactor;
END$$
DELIMiTER ;
The function f_splitadjprice runs extremely slow. About 14 seconds PR row. I have tried to run the individual pieces of the function by themselves. That is, the function call f_splitfactor and SELECT (f.p_price FROM fp_v2_fp_basic_prices as f WHERE f.fsym_id = id AND
f.p_date = startdate). When running these two by themselves outside of the function they take 0,001 seconds to run. So the whole problem is that as soon as I want to do in combination through the nested function it takes 100.000 times longer?
The solution is to not call tables within functions. In general that seems to be bad practice and it is nevertheless extremely slow. One should instead try to get rid of the function and perform the function directly in the query.

Getting syntax errors when creating function

I have been spending over 2 hours now trying to google my way to the answer.
I am completely new to SQL and MySQL and I have tried to write the following function:
CREATE FUNCTION fp_spinofffactor (id char(8), startdate date)
RETURNS float
BEGIN
DECLARE spinoffFactor float; (ERROR- EXPECTED A ";")
select spinoffFactor = ISNULL(EXP(SUM(LOG(spinoffFactor))),1)
from(
select case when (prev_price- divs) <= 0 THEN 1
else (prev_price- divs)/prev_price end as spinoffFactor
from (select
divs,
fp_v2.fp_prevUnadjPrice(id, ex_date) as prev_price
from (
select sum(fbd.p_divs_pd) as divs,fbd.p_divs_exdate as ex_date
from fp_v2.fp_basic_dividends fbd
where fbd.fsym_id = id
and fbd.p_divs_s_pd=1
and fbd.p_divs_exdate > startdate
group by fbd.p_divs_exdate ) a ) b ) c;
return spinofffactor; ERROR (Return is not valid at this position)
END ERROR (END IS NOT VALID AT THIS position)
But I get multiple syntax errors. I have written the errors where I get them.
I have a hard time finding information about the syntax rules of MySQL and the workbench.
Can anyone help ?
You need to provide delimiter in MySql workbench to tell where your code begins and ends.
Assuming your syntax is correct, you can write as below.
DELIMITER $$
CREATE FUNCTION fp_spinofffactor (id char(8), startdate date)
RETURNS float
BEGIN
DECLARE spinoffFactor float;
select spinoffFactor = ISNULL(EXP(SUM(LOG(spinoffFactor))),1)
from(
select case when (prev_price- divs) <= 0 THEN 1
else (prev_price- divs)/prev_price end as spinoffFactor
from (select
divs,
fp_v2.fp_prevUnadjPrice(id, ex_date) as prev_price
from (
select sum(fbd.p_divs_pd) as divs,fbd.p_divs_exdate as ex_date
from fp_v2.fp_basic_dividends fbd
where fbd.fsym_id = id
and fbd.p_divs_s_pd=1
and fbd.p_divs_exdate > startdate
group by fbd.p_divs_exdate ) a ) b ) c;
return spinofffactor;
END$$
DELIMITER ;
You can also, run this from MySQL command prompt and it should work.

User defined function only returns NULL

I have this following MySQL code:
DELIMITER $$
CREATE FUNCTION durationInMinutes(id INT)
RETURNS INT DETERMINISTIC
BEGIN
DECLARE Minutes INT;
SET Minutes =
(SELECT TIME_TO_SEC(TIMEDIFF(timeDeparture, timeArrival)) FROM AirRoute
WHERE pk_id = id) / 60;
RETURN Minutes;
END$$
DELIMITER;
Basically, this function calculates the duration of a flight in minutes. The parameter is the id of the flight. For some reason though, this function always returns NULL. I even checked this:
SELECT (SELECT TIME_TO_SEC(TIMEDIFF(timeDeparture, timeArrival)) FROM AirRoute
WHERE pk_id = 925) / 60;
This does return the correct answer if I put id = 925, so there could be something wrong with the RETURN statement.
I suspect there is a column called id in the table. I always name parameters and local variables in a way to distinguish them from column names:
CREATE FUNCTION durationInMinutes (
in_id INT
)
RETURNS INT DETERMINISTIC
BEGIN
DECLARE out_Minutes INT;
SELECT out_Minutes := TIME_TO_SEC(TIMEDIFF(timeDeparture, timeArrival))
FROM AirRoute ar
WHERE ar.pk_id = in_id) / 60;
RETURN out_Minutes;
END$$
Ok, I solved it. This is my corrected code:
DELIMITER $$
CREATE FUNCTION durationInMinutes(id INT)
RETURNS INT DETERMINISTIC
BEGIN
RETURN (SELECT TIME_TO_SEC(TIMEDIFF(timeDeparture, timeArrival))
FROM AirRoute
WHERE pk_id = id / 60);
END$$
DELIMITER ;
Still, I really don't understand why it wasn't possible using a temp variable.

How to reduce time to process data in procedure of mysql

Thank you forum...please help me..
I have table which contain TagName and other table contains taglog .im passing tag names to procedure called GetAvg which will return avg of all tags.it works well, but it takes about 35 sec to show 100 tag values.how to reduce time .please help me im new in database.
DELIMITER $$
CREATE DEFINER=`root`#`localhost` PROCEDURE `GetAvg`(IN FromTime datetime, IN ToTime datetime)
BEGIN
DECLARE no_more_alarms INT DEFAULT 0;
DECLARE TempTagName VARCHAR(45);
DECLARE val FLOAT;
DECLARE cur_tag CURSOR FOR
select Tag_AVG from Report
where(Tag_AVG IS NOT NULL );
DECLARE CONTINUE HANDLER FOR NOT FOUND
SET no_more_alarms = 1;
DROP TABLE IF EXISTS `tempAVG`;
CREATE TABLE tempAVG (
val FLOAT
);
OPEN cur_tag;
FETCH cur_tag INTO TempTagName;
REPEAT
SELECT AVG(value) INTO val
FROM jas_taglog
WHERE ((TagId = (select TagId from jas_tags where jas_tags.Name = TempTagName)) AND jas_taglog.LogTime between FromTime and ToTime) ;
INSERT INTO tempAvg(Val)
VALUES (val);
FETCH cur_tag INTO TempTagName;
UNTIL no_more_alarms = 1
END REPEAT;
CLOSE cur_tag;
SELECT * FROM tempAVG;
END
You are manually implementing loops that could be written in a single query:
...
BEGIN
INSERT INTO tempAvg
SELECT AVG(jas_taglog.value)
FROM jas_taglog
JOIN jas_tags USING (TagId)
JOIN Report ON jas_tags.Name = Tag_AVG
WHERE jas_taglog.LogTime BETWEEN FromTime AND ToTime
GROUP BY Tag_AVG;
END

Having trouble doing multiple cursors in a MySQL stored procedure

I'm writing a store procedure to create two temporary tables do an union select of the two.
When using either first or second cursor alone with the other commented the procedure works,
but when I run the query to create the procedure with the 2 cursors, it fails.i've changed the code to reflect Ike Walker's suggestion.
Here is the script:
DELIMITER //
DROP PROCEDURE IF EXISTS joinemailsmsdailygraph//
CREATE PROCEDURE joinemailsmsdailygraph(IN previousDay VARCHAR(20), IN today VARCHAR(20))
READS SQL DATA
BEGIN
DECLARE hours INT;
DECLARE sms INT;
DECLARE email INT;
DECLARE smsdone INT DEFAULT 0;
DECLARE emaildone INT DEFAULT 0;
DECLARE cursorsms CURSOR FOR SELECT HOUR(sm.date_created) AS `HOUR OF DAY`, COUNT(*) AS smscount
FROM sms_message_delivery smd
JOIN sms_message sm ON sm.sms_message_id = smd.sms_message_id
WHERE DATE(sm.date_created) >= DATE(previousDay) AND DATE(sm.date_created) < DATE(today)
GROUP BY HOUR(sm.date_created);
DECLARE CONTINUE HANDLER FOR NOT FOUND SET smsdone =1;
DECLARE cursoremail CURSOR FOR SELECT HOUR(em.date_created) AS `HOUR OF DAY`, COUNT(*) AS emailcount
FROM email_message_delivery emd
LEFT JOIN email_message em ON emd.email_message_id=em.email_message_id
WHERE DATE(em.date_created) >= DATE(previousDay) AND DATE(em.date_created) < DATE(today)
GROUP BY HOUR(em.date_created);
DECLARE CONTINUE HANDLER FOR NOT FOUND SET emaildone =1;
DROP TEMPORARY TABLE IF EXISTS tempsms;
CREATE TEMPORARY TABLE tempsms (hours_day INT, sms_count INT, email_count INT);
OPEN cursorsms;
sms_loop: LOOP
FETCH cursorsms INTO hours , sms;
IF smsdone = 1 THEN
LEAVE sms_loop;
END IF;
INSERT INTO tempsms (hours_day, sms_count) VALUES (hours, sms);
END LOOP sms_loop;
CLOSE cursorsms;
DROP TEMPORARY TABLE IF EXISTS tempemail;
CREATE TEMPORARY TABLE tempemail (hours_day INT , sms_count INT , email_count INT);
OPEN cursoremail;
email_loop: LOOP
FETCH cursoremail INTO hours, email;
IF emaildone=1 THEN
LEAVE email_loop;
END IF;
INSERT INTO tempemail(hours_day, email_count) VALUES(hours, email);
END LOOP email_loop;
CLOSE cursoremail;
SELECT hours_day, sms_count , email_count FROM tempsms
UNION
SELECT hours_day, sms_count, email_count FROM tempemail;
END//
DELIMITER;
it gives this as error
Query : CREATE PROCEDURE joinemailsmsdailygraph(IN previousDay VARCHAR(20), IN today VARCHAR(20)) READS SQL DATA BEGIN DECLARE hours INT...
Error Code : 1338
Cursor declaration after handler declaration
Execution Time : 00:00:00:000
Transfer Time : 00:00:00:000
Total Time : 00:00:00:000
ive tried putting both continue handlers at the end of all declare section but it complains about declare block overlapping or so.
Can you please tell me what I'm doing wrong? Thanks for reading.
Why are you using cursors ? You could easily do this without a tmp table also by just using a union.
drop procedure if exists join_email_sms_daily_graph;
delimiter #
create procedure join_email_sms_daily_graph
(
in previousDay varchar(20),
in today varchar(20)
)
begin
create temporary table tmp
(
hours_day int unsigned,
sms_count int unsigned default 0,
email_count int unsigned default 0
)engine=memory;
insert into tmp (hours_day, sms_count)
select
hour(sm.date_created) as hours_day,
count(*) AS sms_count
from
sms_message_delivery smd
join sms_message sm ON sm.sms_message_id = smd.sms_message_id
where
date(sm.date_created) >= date(previousDay) and date(sm.date_created) < date(today)
group by
hour(sm.date_created);
insert into tmp (hours_day, email_count)
select
hour(em.date_created) as hours_day,
count(*) AS email_count
from
email_message_delivery emd
left join email_message em ON emd.email_message_id=em.email_message_id
where
date(em.date_created) >= date(previousDay) and date(em.date_created) < date(today)
group by
hour(em.date_created);
select * from tmp;
drop temporary table if exists tmp;
end#
delimiter;
You need to declare all of the cursors up front, before the logic of your procedure begins.
If you do things in this order it should work:
DECLARE variables
DECLARE cursors
DECLARE handlers
...
logic