I am trying to create a procedure in MySQL that insert weeks (for current year) to my week table. But there is a problem because after first row is added for the next one I get an error: number column cannot be null. I am new to MySQL so I will appreciate any help.
CREATE PROCEDURE generateWeeks()
BEGIN
SET #currentYear = YEAR(CURDATE());
SET #nextYear = #currentYear + 1;
SET #startOfCurrentWeek = CURDATE();
WHILE(#currentYear < #nextYear) DO
SET #endOfCurrentWeek = DATE_ADD(#startOfCurrentWeek , INTERVAL 7 DAY);
SET #weekNumber = WEEK(#startOfCurrentWeek, 3) -
WEEK(#startOfCurrentWeek - INTERVAL DAY(#startOfCurrentWeek)-1 DAY, 3) + 1;
INSERT INTO `week` (`number`, `start_date`, `end_date`)
VALUES (#weekNumber, #startOfCurrentWeek, #endOfCurrentWeek);
SET #startOfCurrentWeek = #endOfCurrentWeek + 1;
SET #currentYear = YEAR(#endOfCurrentWeek);
END WHILE;
END //
DELIMITER ;
EDITED:
Table Creation:
CREATE TABLE `week` (
`id` INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,
`number` INT(11) NOT NULL,
`start_date` DATE NOT NULL,
`end_date` DATE NOT NULL
)
Why for first while iteration everything is ok (rows is added), but in the next one I get null value in #weekNumber variable ?
The line:
SET #startOfCurrentWeek = #endOfCurrentWeek + 1;
will convert the variable into a integer. Use date_add instead.
Also, instead of using user-defined variables (#endOfCurrentWeek) you better use local variabled (declare v_endOfCurrentWeek date).
Related
I want to optimise a mysql 5.7 function that reads settings from a table. the function returns a 1 or 2 if the date parsed in is in 'semester 1' or 'semester 2'. the dates for semester 1 and 2 change each year.
we have confirmed that dateIn is a valid date.
the function is:
DELIMITER //
CREATE function getSemester (dateIN date)
RETURNS INT DETERMINISTIC
BEGIN
DECLARE sem int;
select if( dateIN < a.mindate,1,2) into sem
from (SELECT min(date(value)) mindate FROM `settings` WHERE name = CONCAT(‘sem2_‘,year(dateIN),‘_start’) ) a;
return sem;
END//
DELIMITER ;
settings is defined as:
CREATE TABLE `settings` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`value` varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
the settings data is:
INSERT INTO `mdl_sap_settings` (`id`, `name`, `value`)
VALUES
(4, 'sem2_2012_start', '2012/7/16'),
(15, 'sem2_2013_start', '2013/7/1'),
(25, 'sem2_2014_start', '2014/6/30'),
(29, 'sem2_2015_start', '2015/6/29'),
(37, 'sem2_2016_start', '2016/6/27'),
(42, 'sem2_2011_start', '2011/7/16'),
(50, 'sem2_2017_start', '2017/6/26'),
(56, 'sem2_2018_start', '2018/6/25'),
(63, 'sem2_2019_start', '2019/6/24');
the issue is the function is slow quite slow when called on 20,000 rows. I thought to optimise it by having some sort of flow control in the function something like:
if (year(dateIN) = 2012)
{
return dateIN < '2012-07-16' ? 1 : 2;
}
if (year(dateIN) = 2013)
{
return dateIN < '2013-07-01' ? 1 : 2;
}
... etc.
We need to keep the select as is query because if the code is not maintained we want it to return the correct values.
i was just wondering if this sort of control flow is possible in a mysql function, or is there an alternative way to optimise the function?
I don't have 20k records to test with but if I were you, I'd rewrite your function like this and see if that helps.
DELIMITER //
CREATE function getSemester (dateIN date)
RETURNS INT DETERMINISTIC
BEGIN
DECLARE sem int;
DECLARE nameByDate varchar(255);
SET nameByDate = (SELECT CONCAT('sem2_',year(dateIN),'_start') );
select if( dateIN < a.mindate,1,2) into sem
from (SELECT min(date(value)) mindate FROM `test`.`settings` WHERE name = nameByDate ) a;
return sem;
END//
DELIMITER ;
I'd avoid adding a concat in the where clause as there is a performance hit on large dataset. So I moved it out of where clause and assigned it to a variable once. If that doesn't help you can also try the query with explain and see if that provides any hints.
EXPLAIN select getSemester('2019/6/24');
Good luck.
With 20,000 rows in the results table, the my query run without the getSemester function in 157 msec.
with the version of getSemster in the question it was taking 1.1 seconds.
with this optimisation the query was running in 0.9 seconds.
DELIMITER //
CREATE function getSemester (dateIN date)
RETURNS tinyint DETERMINISTIC
BEGIN
DECLARE sem tinyint;
DECLARE sem2_start_label char(16);
set sem2_start_label = CONCAT('sem2_',year(dateIN),'_start');
select case year(dateIN)
when 2012 then
case when dateIN < date('2012-07-16') then 1 else 2 end
when 2013 then
case when dateIN < date('2013-07-01') then 1 else 2 end
when 2014 then
case when dateIN < date('2014-06-30') then 1 else 2 end
when 2015 then
case when dateIN < date('2015-06-29') then 1 else 2 end
when 2016 then
case when dateIN < date('2016-06-27') then 1 else 2 end
when 2017 then
case when dateIN < date('2017-06-26') then 1 else 2 end
when 2018 then
case when dateIN < date('2018-06-25') then 1 else 2 end
when 2019 then
case when dateIN < date('2019-06-24') then 1 else 2 end
else
case when dateIN < (SELECT min(date(value)) mindate FROM `settings` WHERE name = sem2_start_label ) then 1 else 2 end
end into sem;
return sem;
END//
DELIMITER ;
what i did was only select the date from the settings table when it was unknown -- seems to give a small optimisation.
There is a MySQL function in our web system to generate Code. The structure of the code is
district_cd(length:2) + date(length:8) + sequence no(length:5,start at 1).<like : ab2016090800001>
The sequence no was saved in table and will be updated (+1) when generate a new code.
But sometimes it returned two same codes and makes us fall in trouble. Here are the captures to replicate this problem, I will attach the DDL after this.
Step 1.Client1->change to manual commit then generate a code, but do not commit.
SET autocommit = 0;
select * from applies;
select * from sequence where apply_date = "2016-09-08";
select nextval("ab");
insert into applies (apply_id,apply_no,created,district_cd) values (2,"ab2016090800002","ab",now());
select * from sequence where apply_date = "2016-09-08";
Step2.Client2->change to manual commit then generate a code, stuck as Client1 locked
SET autocommit = 0;
select * from applies;
select * from sequence where apply_date = "2016-09-08";
insert into applies (apply_id,apply_no,created,district_cd) values (3,"ab20160908123456780","ab",now());
Step3.Client1->commit;
commit;
select * from sequence where apply_date = "2016-09-08";
Step4.Client2->code was generated and two records appeared in sequence table
select * from sequence where apply_date = "2016-09-08";
capture of Step4
Step5.Client2->commit;one of the two records that appeared in sequence table was deleted.The codes generated are duplicated.
commit;
select * from sequence where apply_date = "2016-09-08";
select * from applies;
capture of Step5
※DDL
Table:applies (apply_no:save the code)
CREATE TABLE `applies` (
`apply_id` varchar(100) NOT NULL DEFAULT '',
`apply_no` varchar(100) NOT NULL DEFAULT '',
`district_cd` varchar(100) DEFAULT NULL,
`created` datetime DEFAULT NULL,
PRIMARY KEY (`apply_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Table:sequence (current_value:save current sequnce value)
CREATE TABLE `sequence` (
`district_cd` varchar(3) NOT NULL DEFAULT '',
`current_value` int(11) NOT NULL DEFAULT '0',
`apply_date` date NOT NULL DEFAULT '0000-00-00',
PRIMARY KEY (`district_cd`,`current_value`,`apply_date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Function:currval->get current sequence value by district_cd
DELIMITER ;;
CREATE DEFINER=`usr`#`%` FUNCTION `currval`(d VARCHAR(3)) RETURNS int(11)
DETERMINISTIC
BEGIN
DECLARE value INTEGER;
DECLARE needInitSequence INTEGER;
DECLARE today DATE;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET needInitSequence = 1;
SET value = 0;
SET today = current_date();
SELECT `current_value` INTO value
FROM `sequence`
WHERE `district_cd` = d AND `apply_date` = today limit 1;
IF needInitSequence = 1 THEN
INSERT INTO `sequence` (`district_cd`, `current_value`, `apply_date`) VALUES (d, value, today);
END IF;
RETURN value;
END
;;
DELIMITER ;
Function:nextval->generate code by district_cd
DELIMITER ;;
CREATE DEFINER=`usr`#`%` FUNCTION `nextval`(d VARCHAR(3)) RETURNS varchar(16) CHARSET utf8
DETERMINISTIC
BEGIN
DECLARE value INTEGER;
SET value = currval(d);
UPDATE `sequence`
SET `current_value` = `current_value` + 1
WHERE `district_cd` = d AND `apply_date` = current_date();
RETURN concat(d, date_format(now(), '%Y%m%d'), LPAD(currval(d), 5, '0'));
END
;;
DELIMITER ;
Triggers of applies->a business logic,if the length of apply_no is greater than 18,it will call the function:nextval to generate a new code
DELIMITER ;;
CREATE TRIGGER `convert_long_no` BEFORE INSERT ON `applies` FOR EACH ROW BEGIN
IF ((SELECT LENGTH(NEW.apply_no)) >= 18) THEN
SET NEW.apply_no = (SELECT nextval(NEW.district_cd));
END IF;
END
;;
DELIMITER ;
My Questions:
Why did the function:nextval returns two same codes?
Why did two records appear in sequnce when update the record.
Given a table like this:
`sensor` int(11)
`reading` decimal(5,2)
`timestamp` datetime
that is representing temperature data and logging an entry whenever a value changes, how would I go about finding the amount of time recorded above a given value?
So there may be a bunch of readings from, say, 16 up to 30, the requirement would be to find the amount of time spent above 16.
Two solutions
I propose two solutions:
one query, but I don't know if it is very efficient with a lot of data, because of the subquery;
one function.
Schema
I tested the query and the function with this table:
CREATE TABLE `temperature` (
`sensor` int(11) NOT NULL,
`timestamp` datetime NOT NULL,
`reading` decimal(5, 2) NOT NULL,
PRIMARY KEY (`sensor`,`timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
And some data:
1;2014-09-18 17:00:00;15.0
1;2014-09-18 18:00:00;16.0
1;2014-09-18 19:00:00;15.0
Solution 1: a query
SELECT SUM(elapsed_time)
FROM (
SELECT
(UNIX_TIMESTAMP((
SELECT MIN(t2.timestamp)
FROM temperature t2
WHERE t1.sensor = t2.sensor AND t1.timestamp < t2.timestamp
)) - UNIX_TIMESTAMP(t1.timestamp))
AS elapsed_time
FROM temperature t1
WHERE t1.sensor = 1 AND t1.reading >= 16.0
) a;
Solution 2: a function
The function returns the amount of time above a value in seconds for a sensor.
It initializes the amount of time to 0. It then read all readings of the desired sensor. If the temperature is above your requirement, it adds the amount of time to go until the next reading to the sum. At last, it returns the amount of time above your requirement for the sensor.
DROP FUNCTION IF EXISTS getTotalTimeAbove;
DELIMITER $$
CREATE FUNCTION getTotalTimeAbove(sensor_id INTEGER, above DECIMAL(5, 2))
RETURNS INTEGER
BEGIN
DECLARE sum INTEGER DEFAULT 0;
DECLARE curr_time DATETIME;
DECLARE next_time DATETIME;
DECLARE curr_reading INTEGER;
DECLARE next_reading INTEGER;
DECLARE done INT DEFAULT FALSE;
DECLARE cur CURSOR FOR
SELECT `timestamp`, `reading`
FROM `temperature`
WHERE `sensor` = sensor_id;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur;
read_loop: LOOP
FETCH cur INTO next_time, next_reading;
IF done THEN
LEAVE read_loop;
END IF;
IF (curr_reading >= above) THEN
SET sum = sum + (UNIX_TIMESTAMP(next_time) - UNIX_TIMESTAMP(curr_time));
END IF;
SET curr_time = next_time;
SET curr_reading = next_reading;
END LOOP;
CLOSE cur;
RETURN sum;
END
$$
DELIMITER ;
The query and its result:
> SELECT getTotalTimeAbove(1, 16.0);
3600
Bonus
You can have the total amount of recorded time for a sensor with this query:
SELECT UNIX_TIMESTAMP(MAX(`timestamp`)) - UNIX_TIMESTAMP(MIN(`timestamp`))
FROM `temperature`
WHERE `sensor` = 1
I'm using MySql and I have a Policy table with StartDate and EndDate columns.
How can I write a SELECT query to give me a new row for each month in the date range between these 2 columns.
For example if I have a policy with:
Id StartDate EndDate
123456 2011-05-25 2011-07-26
I would want to see:
Id PolicyId StartDate EndDate
1 123456 2011-05-25 2011-06-24
2 123456 2011-06-25 2011-07-24
3 123456 2011-07-25 2011-07-26
I'm not sure about performance because I'm not much experienced with stored procedures so there might be a better approach. Also, you might want to change the structure of the temporary table (aka. PolicyList). Anyway…
This can also be converted into before/after triggers instead of executing it each time.
DROP PROCEDURE IF EXISTS CreatePolicyList;
DELIMITER //
CREATE PROCEDURE CreatePolicyList()
BEGIN
DECLARE origId, done INT DEFAULT 0;
DECLARE startD, endD DATE;
DECLARE cur CURSOR FOR
SELECT id, StartDate, EndDate FROM Policy;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
DROP TEMPORARY TABLE IF EXISTS PolicyList;
CREATE TEMPORARY TABLE PolicyList (
id INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
PolicyId INT(11) NOT NULL,
StartDate DATE NOT NULL,
EndDate DATE NOT NULL,
PRIMARY KEY (id)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
OPEN cur;
recLoop: LOOP
FETCH cur INTO origId, startD, endD;
IF (done)
THEN LEAVE recLoop;
END IF;
-- following is an alternative to keep records like
-- "2011-05-25, 2011-06-25" in a single record
-- WHILE startD < DATE_SUB(endD, INTERVAL 1 MONTH) DO
WHILE startD < DATE_ADD(DATE_SUB(endD, INTERVAL 1 MONTH), INTERVAL 1 DAY) DO
INSERT INTO PolicyList (PolicyId, StartDate, EndDate)
VALUES (origId, startD,DATE_SUB(
DATE_ADD(startD, INTERVAL 1 MONTH),
INTERVAL 1 DAY
));
SET startD = DATE_ADD(startD, INTERVAL 1 MONTH);
END WHILE;
IF startD >= DATE_SUB(endD, INTERVAL 1 MONTH) THEN
INSERT INTO PolicyList (PolicyId, StartDate, EndDate)
VALUES (origId, startD, endD);
END IF;
END LOOP;
CLOSE cur;
END //
CALL CreatePolicyList;
and then query:
SELECT * FROM PolicyList
ORDER BY PolicyId, StartDate;
I'm trying to generate a primary key for my table, something like this
(simplified version) - the purpose is to have a daily incremented key:
DELIMITER ^
CREATE TABLE `ADDRESS` (
ID INTEGER NOT NULL DEFAULT -1,
NAME VARCHAR(25),
PRIMARY KEY(`ID`))^
CREATE FUNCTION `GETID`()
RETURNS INTEGER
deterministic
BEGIN
declare CURR_DATE DATE;
declare maxid, _year, _month, _day, newid INTEGER;
set CURR_DATE = CURRENT_DATE;
set _year = EXTRACT(YEAR FROM CURR_DATE);
set _mon = EXTRACT(MONTH FROM CURR_DATE);
set _day = EXTRACT(DAY FROM CURR_DATE);
set newid = (_year - (_year/100) * 100) * 10000 + _mon * 100 + _day;
select max(ID) into maxid From `ADDRESS`;
if (maxid is null) then
set maxid = 0;
end if;
if (MAXID / 1000 != newid) then
set MAXID = newid * 1000;
end if;
set MAXID = MAXID + 1;
return MAXID;
END^
CREATE TRIGGER `ADDRESS_ID_TRIGGER` BEFORE INSERT ON `ADDRESS`
FOR EACH ROW
BEGIN
if new.id=-1 then
set new.id = getid();
end if ;
END^
COMMIT^
DELIMITER ;
Generally it works fine, but when I test it with multiple inserts at the same time
it obviously fails (e.g. no dirty reads, the select max will fail for the 2nd insert,
thus it will generate the same id as fro the 1st insert).
Workaround:
Make primary key AUTO_INCREMENT.
Add TIMESTAMP field and use BEFORE INSERT/UPDATE trigget to set CURRENT_TIMESTAMP().
Also you can use ON UPDATE CURRENT_TIMESTAMP option for TIMESTAMP field, value will be updated automatically.
So, ID is ID, and TIMESTAMP field contains date and time.