How can I add control flow into a mysql function? - mysql

I want to optimise a mysql 5.7 function that reads settings from a table. the function returns a 1 or 2 if the date parsed in is in 'semester 1' or 'semester 2'. the dates for semester 1 and 2 change each year.
we have confirmed that dateIn is a valid date.
the function is:
DELIMITER //
CREATE function getSemester (dateIN date)
RETURNS INT DETERMINISTIC
BEGIN
DECLARE sem int;
select if( dateIN < a.mindate,1,2) into sem
from (SELECT min(date(value)) mindate FROM `settings` WHERE name = CONCAT(‘sem2_‘,year(dateIN),‘_start’) ) a;
return sem;
END//
DELIMITER ;
settings is defined as:
CREATE TABLE `settings` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`value` varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
the settings data is:
INSERT INTO `mdl_sap_settings` (`id`, `name`, `value`)
VALUES
(4, 'sem2_2012_start', '2012/7/16'),
(15, 'sem2_2013_start', '2013/7/1'),
(25, 'sem2_2014_start', '2014/6/30'),
(29, 'sem2_2015_start', '2015/6/29'),
(37, 'sem2_2016_start', '2016/6/27'),
(42, 'sem2_2011_start', '2011/7/16'),
(50, 'sem2_2017_start', '2017/6/26'),
(56, 'sem2_2018_start', '2018/6/25'),
(63, 'sem2_2019_start', '2019/6/24');
the issue is the function is slow quite slow when called on 20,000 rows. I thought to optimise it by having some sort of flow control in the function something like:
if (year(dateIN) = 2012)
{
return dateIN < '2012-07-16' ? 1 : 2;
}
if (year(dateIN) = 2013)
{
return dateIN < '2013-07-01' ? 1 : 2;
}
... etc.
We need to keep the select as is query because if the code is not maintained we want it to return the correct values.
i was just wondering if this sort of control flow is possible in a mysql function, or is there an alternative way to optimise the function?

I don't have 20k records to test with but if I were you, I'd rewrite your function like this and see if that helps.
DELIMITER //
CREATE function getSemester (dateIN date)
RETURNS INT DETERMINISTIC
BEGIN
DECLARE sem int;
DECLARE nameByDate varchar(255);
SET nameByDate = (SELECT CONCAT('sem2_',year(dateIN),'_start') );
select if( dateIN < a.mindate,1,2) into sem
from (SELECT min(date(value)) mindate FROM `test`.`settings` WHERE name = nameByDate ) a;
return sem;
END//
DELIMITER ;
I'd avoid adding a concat in the where clause as there is a performance hit on large dataset. So I moved it out of where clause and assigned it to a variable once. If that doesn't help you can also try the query with explain and see if that provides any hints.
EXPLAIN select getSemester('2019/6/24');
Good luck.

With 20,000 rows in the results table, the my query run without the getSemester function in 157 msec.
with the version of getSemster in the question it was taking 1.1 seconds.
with this optimisation the query was running in 0.9 seconds.
DELIMITER //
CREATE function getSemester (dateIN date)
RETURNS tinyint DETERMINISTIC
BEGIN
DECLARE sem tinyint;
DECLARE sem2_start_label char(16);
set sem2_start_label = CONCAT('sem2_',year(dateIN),'_start');
select case year(dateIN)
when 2012 then
case when dateIN < date('2012-07-16') then 1 else 2 end
when 2013 then
case when dateIN < date('2013-07-01') then 1 else 2 end
when 2014 then
case when dateIN < date('2014-06-30') then 1 else 2 end
when 2015 then
case when dateIN < date('2015-06-29') then 1 else 2 end
when 2016 then
case when dateIN < date('2016-06-27') then 1 else 2 end
when 2017 then
case when dateIN < date('2017-06-26') then 1 else 2 end
when 2018 then
case when dateIN < date('2018-06-25') then 1 else 2 end
when 2019 then
case when dateIN < date('2019-06-24') then 1 else 2 end
else
case when dateIN < (SELECT min(date(value)) mindate FROM `settings` WHERE name = sem2_start_label ) then 1 else 2 end
end into sem;
return sem;
END//
DELIMITER ;
what i did was only select the date from the settings table when it was unknown -- seems to give a small optimisation.

Related

MySQL - copy or update rows with a change within one table

I have a database table like this one:
group | detailsID | price
EK | 1 | 1.40
EK | 2 | 1.50
EK | 3 | 1.60
H | 1 | 2.40
H | 2 | 2.50
Now I want to copy the data from group "EK" to the group "H", so the prices for the detailsID's must be adjusted for the detailsIDs 1 and 2, and the entry for detailsID 3 must be inserted for group "H".
How can I do that with one or two MySQL query's?
Thanks!
We can try doing an INSERT INTO ... SELECT with ON DUPLICATE KEY UPDATE:
INSERT INTO yourTable (`group`, detailsID, price)
SELECT 'H', detailsID, price
FROM yourTable t
WHERE `group` = 'EK'
ON DUPLICATE KEY UPDATE price = t.price;
But this assumes that there exists a unique key on (group, detailsID). If this would not be possible, then this approach would not work.
As an alternative, I might do this in two steps. First, remove the H group records, then insert the updated H records you expect.
DELETE
FROM yourTable
WHERE `group` = 'H';
INSERT INTO yourTable (`group`, detailsID, price)
SELECT 'H', detailsID, price
FROM yourTable
WHERE `group` = 'EK';
I use the above approach because a single update can't handle your requirement, since new records also need to be inserted.
Note that you should avoid naming your columns and tables using reserved MySQL keywords such as GROUP.
You can try this as well, Following code implemented using stored procedures. Very simple not that difficult to understand. You may need to modify data type and optimize the code as per the requirement.
DELIMITER $$;
DROP PROCEDURE IF EXISTS update_H $$;
CREATE PROCEDURE update_H()
BEGIN
DECLARE finished INTEGER DEFAULT 0;
DECLARE `group_col` varchar(255) DEFAULT "";
DECLARE `detaildid_col` varchar(255) DEFAULT "";
DECLARE `price_col` varchar(255) DEFAULT "";
DECLARE H_FOUND INTEGER DEFAULT 0;
DECLARE pull_data CURSOR FOR select `group`, `detaildid`, `price` from test.newtab WHERE `group` = 'EK';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN pull_data;
traverse_data: LOOP
FETCH pull_data INTO group_col, detaildid_col, price_col;
IF finished = 1 THEN
LEAVE traverse_data;
END IF;
SET H_FOUND = (SELECT count(*) from test.newtab where `group` = 'H' AND `detaildid` = detaildid_col);
IF ( H_FOUND = 1 ) THEN
UPDATE test.newtab SET `price` = price_col where `group` = 'H' AND `detaildid` = detaildid_col;
ELSE
INSERT INTO test.newtab (`group`, `detaildid`, `price`) VALUES ('H', detaildid_col, price_col);
END IF;
END LOOP traverse_data;
CLOSE pull_data;
END $$;
DELIMITER ;
You can call this procedure by executing, call update_H();

Column cannot be null - procedure

I am trying to create a procedure in MySQL that insert weeks (for current year) to my week table. But there is a problem because after first row is added for the next one I get an error: number column cannot be null. I am new to MySQL so I will appreciate any help.
CREATE PROCEDURE generateWeeks()
BEGIN
SET #currentYear = YEAR(CURDATE());
SET #nextYear = #currentYear + 1;
SET #startOfCurrentWeek = CURDATE();
WHILE(#currentYear < #nextYear) DO
SET #endOfCurrentWeek = DATE_ADD(#startOfCurrentWeek , INTERVAL 7 DAY);
SET #weekNumber = WEEK(#startOfCurrentWeek, 3) -
WEEK(#startOfCurrentWeek - INTERVAL DAY(#startOfCurrentWeek)-1 DAY, 3) + 1;
INSERT INTO `week` (`number`, `start_date`, `end_date`)
VALUES (#weekNumber, #startOfCurrentWeek, #endOfCurrentWeek);
SET #startOfCurrentWeek = #endOfCurrentWeek + 1;
SET #currentYear = YEAR(#endOfCurrentWeek);
END WHILE;
END //
DELIMITER ;
EDITED:
Table Creation:
CREATE TABLE `week` (
`id` INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
`number` INT(11) NOT NULL,
`start_date` DATE NOT NULL,
`end_date` DATE NOT NULL
)
Why for first while iteration everything is ok (rows is added), but in the next one I get null value in #weekNumber variable ?
The line:
SET #startOfCurrentWeek = #endOfCurrentWeek + 1;
will convert the variable into a integer. Use date_add instead.
Also, instead of using user-defined variables (#endOfCurrentWeek) you better use local variabled (declare v_endOfCurrentWeek date).

Stored procedure inserting the same value over and over again

I'm facing an unlikely event handled by the Stored Procedure for MySQL.
I've managed to search for multiple rows in the stored procedure but I am testing it before I can go live.
My current issue is that the result from the below query returns 100 different result set.
But when the query to insert is where the problem begins. It's inserting the same #UID over and over again until it reaches 100.
Is there a way I can increment to the next row before I even insert it?
DECLARE pSpot INT(11) DEFAULT 0;
DECLARE con INT(11) DEFAULT 97;
DECLARE tempString VARCHAR(255) DEFAULT NULL;
DECLARE x INT(11) DEFAULT 0;
IF(pSpot<=97) -- condition 1
THEN
SELECT #uid:=uid,
nickname,
lastsync,
dob,
gender,
gender_preference,
Latest_LAT,
Latest_LON,
country,
imagetoken
FROM search_optimized_table
where country = #csid
and TIMESTAMPDIFF(YEAR,DOB,CURDATE()) BETWEEN minAge and maxAge
and gender=#tempString
and lastsync BETWEEN UNIX_TIMESTAMP(DATE_ADD(NOW(),INTERVAL -7 DAY))
and UNIX_TIMESTAMP(DATE_ADD(NOW(),INTERVAL 0 DAY))
LIMIT con;
SET con = con - pSpot;
SET pSpot = pSpot + (Select found_rows());
WHILE x <= pSpot DO
INSERT INTO temp_local_history (pUid) values (#uid);
END WHILE;
Well, i did a solution since the cursor method returns me 0 .
IF(pSpot<=97) -- condition 1
THEN
SET con = con - pSpot;
INSERT INTO temp_local_search(
SELECT #uid:=uid FROM search_optimized_table where country = #csid and TIMESTAMPDIFF(YEAR,DOB,CURDATE()) BETWEEN minAge and maxAge and gender=#tempString and lastsync BETWEEN UNIX_TIMESTAMP(DATE_ADD(NOW(),INTERVAL -7 DAY)) and UNIX_TIMESTAMP(DATE_ADD(NOW(),INTERVAL 0 DAY)) LIMIT con
);

Function that generate Code returns the same things

There is a MySQL function in our web system to generate Code. The structure of the code is
district_cd(length:2) + date(length:8) + sequence no(length:5,start at 1).<like : ab2016090800001>
The sequence no was saved in table and will be updated (+1) when generate a new code.
But sometimes it returned two same codes and makes us fall in trouble. Here are the captures to replicate this problem, I will attach the DDL after this.
Step 1.Client1->change to manual commit then generate a code, but do not commit.
SET autocommit = 0;
select * from applies;
select * from sequence where apply_date = "2016-09-08";
select nextval("ab");
insert into applies (apply_id,apply_no,created,district_cd) values (2,"ab2016090800002","ab",now());
select * from sequence where apply_date = "2016-09-08";
Step2.Client2->change to manual commit then generate a code, stuck as Client1 locked
SET autocommit = 0;
select * from applies;
select * from sequence where apply_date = "2016-09-08";
insert into applies (apply_id,apply_no,created,district_cd) values (3,"ab20160908123456780","ab",now());
Step3.Client1->commit;
commit;
select * from sequence where apply_date = "2016-09-08";
Step4.Client2->code was generated and two records appeared in sequence table
select * from sequence where apply_date = "2016-09-08";
capture of Step4
Step5.Client2->commit;one of the two records that appeared in sequence table was deleted.The codes generated are duplicated.
commit;
select * from sequence where apply_date = "2016-09-08";
select * from applies;
capture of Step5
※DDL
Table:applies (apply_no:save the code)
CREATE TABLE `applies` (
`apply_id` varchar(100) NOT NULL DEFAULT '',
`apply_no` varchar(100) NOT NULL DEFAULT '',
`district_cd` varchar(100) DEFAULT NULL,
`created` datetime DEFAULT NULL,
PRIMARY KEY (`apply_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Table:sequence (current_value:save current sequnce value)
CREATE TABLE `sequence` (
`district_cd` varchar(3) NOT NULL DEFAULT '',
`current_value` int(11) NOT NULL DEFAULT '0',
`apply_date` date NOT NULL DEFAULT '0000-00-00',
PRIMARY KEY (`district_cd`,`current_value`,`apply_date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Function:currval->get current sequence value by district_cd
DELIMITER ;;
CREATE DEFINER=`usr`#`%` FUNCTION `currval`(d VARCHAR(3)) RETURNS int(11)
DETERMINISTIC
BEGIN
DECLARE value INTEGER;
DECLARE needInitSequence INTEGER;
DECLARE today DATE;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET needInitSequence = 1;
SET value = 0;
SET today = current_date();
SELECT `current_value` INTO value
FROM `sequence`
WHERE `district_cd` = d AND `apply_date` = today limit 1;
IF needInitSequence = 1 THEN
INSERT INTO `sequence` (`district_cd`, `current_value`, `apply_date`) VALUES (d, value, today);
END IF;
RETURN value;
END
;;
DELIMITER ;
Function:nextval->generate code by district_cd
DELIMITER ;;
CREATE DEFINER=`usr`#`%` FUNCTION `nextval`(d VARCHAR(3)) RETURNS varchar(16) CHARSET utf8
DETERMINISTIC
BEGIN
DECLARE value INTEGER;
SET value = currval(d);
UPDATE `sequence`
SET `current_value` = `current_value` + 1
WHERE `district_cd` = d AND `apply_date` = current_date();
RETURN concat(d, date_format(now(), '%Y%m%d'), LPAD(currval(d), 5, '0'));
END
;;
DELIMITER ;
Triggers of applies->a business logic,if the length of apply_no is greater than 18,it will call the function:nextval to generate a new code
DELIMITER ;;
CREATE TRIGGER `convert_long_no` BEFORE INSERT ON `applies` FOR EACH ROW BEGIN
IF ((SELECT LENGTH(NEW.apply_no)) >= 18) THEN
SET NEW.apply_no = (SELECT nextval(NEW.district_cd));
END IF;
END
;;
DELIMITER ;
My Questions:
Why did the function:nextval returns two same codes?
Why did two records appear in sequnce when update the record.

Obtaining the amount of time spent above a threshold value from time series data

Given a table like this:
`sensor` int(11)
`reading` decimal(5,2)
`timestamp` datetime
that is representing temperature data and logging an entry whenever a value changes, how would I go about finding the amount of time recorded above a given value?
So there may be a bunch of readings from, say, 16 up to 30, the requirement would be to find the amount of time spent above 16.
Two solutions
I propose two solutions:
one query, but I don't know if it is very efficient with a lot of data, because of the subquery;
one function.
Schema
I tested the query and the function with this table:
CREATE TABLE `temperature` (
`sensor` int(11) NOT NULL,
`timestamp` datetime NOT NULL,
`reading` decimal(5, 2) NOT NULL,
PRIMARY KEY (`sensor`,`timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
And some data:
1;2014-09-18 17:00:00;15.0
1;2014-09-18 18:00:00;16.0
1;2014-09-18 19:00:00;15.0
Solution 1: a query
SELECT SUM(elapsed_time)
FROM (
SELECT
(UNIX_TIMESTAMP((
SELECT MIN(t2.timestamp)
FROM temperature t2
WHERE t1.sensor = t2.sensor AND t1.timestamp < t2.timestamp
)) - UNIX_TIMESTAMP(t1.timestamp))
AS elapsed_time
FROM temperature t1
WHERE t1.sensor = 1 AND t1.reading >= 16.0
) a;
Solution 2: a function
The function returns the amount of time above a value in seconds for a sensor.
It initializes the amount of time to 0. It then read all readings of the desired sensor. If the temperature is above your requirement, it adds the amount of time to go until the next reading to the sum. At last, it returns the amount of time above your requirement for the sensor.
DROP FUNCTION IF EXISTS getTotalTimeAbove;
DELIMITER $$
CREATE FUNCTION getTotalTimeAbove(sensor_id INTEGER, above DECIMAL(5, 2))
RETURNS INTEGER
BEGIN
DECLARE sum INTEGER DEFAULT 0;
DECLARE curr_time DATETIME;
DECLARE next_time DATETIME;
DECLARE curr_reading INTEGER;
DECLARE next_reading INTEGER;
DECLARE done INT DEFAULT FALSE;
DECLARE cur CURSOR FOR
SELECT `timestamp`, `reading`
FROM `temperature`
WHERE `sensor` = sensor_id;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur;
read_loop: LOOP
FETCH cur INTO next_time, next_reading;
IF done THEN
LEAVE read_loop;
END IF;
IF (curr_reading >= above) THEN
SET sum = sum + (UNIX_TIMESTAMP(next_time) - UNIX_TIMESTAMP(curr_time));
END IF;
SET curr_time = next_time;
SET curr_reading = next_reading;
END LOOP;
CLOSE cur;
RETURN sum;
END
$$
DELIMITER ;
The query and its result:
> SELECT getTotalTimeAbove(1, 16.0);
3600
Bonus
You can have the total amount of recorded time for a sensor with this query:
SELECT UNIX_TIMESTAMP(MAX(`timestamp`)) - UNIX_TIMESTAMP(MIN(`timestamp`))
FROM `temperature`
WHERE `sensor` = 1