Timestamp handling function breaks replication - mysql

We need an increasing microseconds timestamp that is NEVER ever allowed to decrease, especially if the time is altered (eg ntp) or on restarts etc. I'm currently forced to use MariaDB 5.5.68 (on CentOS).
Basically it is an implementation of https://en.wikipedia.org/wiki/Lamport_timestamp
Currently it works like this:
Table creation:
CREATE TABLE `tblboxmicro` (
`microTime` BIGINT UNSIGNED NOT NULL -- the highest microtime used
) ENGINE=MYISAM DEFAULT CHARSET=utf8;
INSERT INTO `tblboxmicro` (`microTime`) VALUES (0);
We create this function in order to use that easily:
delimiter //
create function getLamportMicros()
returns bigint
reads sql data
begin
declare ret bigint;
UPDATE tblboxmicro SET microTime = GREATEST(round(##SESSION.timestamp * 1000000, 0), microTime+1);
SELECT microTime into ret FROM tblboxmicro;
return ret;
end
//
delimiter ;
It is used this way (a real life example query from PHP with PDO):
INSERT INTO tblboxusers
(microTime, roleNodeId,
userNodeId, boxId, roleId, email, name, notes, meta, cipher, accesscode)
VALUES (getLamportMicros(),0,?,?,?,?,?,?,?,?,?)
The problem
Now we want to establish a cross-master replication and this function is always breaking the replication. It says
Slave SQL: Could not execute Update_rows event on table box.tblboxmicro; Can't find record in 'tblboxmicro', Error_code: 1032; handler error HA_ERR_END_OF_FILE;
Is there a more clever way to reach our goal in a way that does not break replication? It has to be fast, of course...
Edit: We use MIXED binlog format.

We finally found the issue. Mariadb was not able to correctly replicate the #SESSION timestamp. We changed the getLamportMicros() function like this for making it work:
delimiter //
create function getLamportMicros()
returns bigint
reads sql data
begin
declare ret bigint;
select GREATEST(CAST(1000000*UNIX_TIMESTAMP(current_timestamp(6)) AS UNSIGNED), microTime+1) into ret from tblboxmicro;
update tblboxmicro set microTime = GREATEST(ret, microTime);
return ret;
end
//
delimiter ;
This is what I found: https://dev.mysql.com/doc/refman/5.7/en/replication-features-variables.html
In statement-based replication, session variables are not replicated
properly when used in statements that update tables. For example, the
following sequence of statements do not insert the same data on the
source and the replica:
SET max_join_size=1000; INSERT INTO mytable VALUES(##max_join_size);
Thus, by replacing ##SESSION.timestamp with the UNIX_TIMESTAMP(current_timestamp(6)) function, it seems to work now. The casting was simply to make it usable for our specific case.

Related

SQL Trigger fallback

My bosses nor any of the DBAs know how to make triggers and I don't neither(new-ish programmer), they just copied/pasted some old triggers from the DB for examples for me. Anyway, my triggers copy data from one table to another after an update/insert. The insert/update that goes to the original table is vital, so if anything fails, I just want the trigger to fail and the original insert/update to still run just fine.
I am using MySQL to test, but we use DB2, but they won't give me access to test triggers in their DB2 environemt, so this the closest solution I can use to test the trigger logic.
I noticed that BEGIN ATOMIC is in the example triggers, does that do what I want? And what would be equivalent in MySQL, so I can test?
I have subselects in my triggers are these safe? Should I declare variables to help avoid issues?
--#SET TERMINATOR #
create table test_trigger (i int) in userspace1#
create table test_trigger_copy (i int) in userspace1#
create or replace trigger test_trigger_air
after insert on test_trigger
referencing new as n
for each row
begin
declare continue handler for sqlexception begin end;
insert into test_trigger_copy(i) values (case when mod (n.i, 2)=0 then cast(RAISE_ERROR('70001', 'No even numbers!') as int) else n.i end);
end#
insert into test_trigger(i) values 1, 2, 3#
select * from test_trigger_copy#
select * from test_trigger#
For DB2 (for LUW at least) databases.
The INSERT statement inside the trigger generates an exception when you try to insert an even number into the base table. The CONTINUE handler consumes exceptions, so you get all inserted numbers in the base table, but only odd numbers in the copy table.
A close MySQL equivalent to DB2's BEGIN ATOMIC, would be a combination of START TRANSACTION, COMMIT, EXIT HANDLER, and ROLLBACK. Instead of:
BEGIN ATOMIC
<statement(s)>
END
one could do the following in MySQL:
BEGIN
DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN
ROLLBACK;
RESIGNAL;
END;
START TRANSACTION;
<statement(s)>
COMMIT;
END
As to subselects (you mean subqueries?) in the trigger, I do not see why they would not work, as long as you observe the restrictions.

Syntax error in a compound sql statement around Start Transaction, i guess

I am struggling with this compound mysql. I am using the Start Transaction for the first time. So anything will be really helpful.
START TRANSACTION
INSERT
INTO
p_ucourse(
course_name,
course_goal,
course_time,
course_creator_id,
course_status
)
VALUES(
'This Course',
'Goal of this course',
480,
1,
1
);
SET
ucourse_id = LAST_INSERT_ID();
INSERT
INTO
r_ucourse_module(course_id,
module_id,
rank)
VALUES(ucourse_id, 1, 1);
INSERT
INTO
r_ucourse_eu(
course_id,
lu_id,
rank,
afterclass
)
VALUES(ucourse_id, 1, 1, 0);
COMMIT
And it throws up the following error:
#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INSERT
INTO
p_ucourse(
course_name,
course_goal,
course_time,' at line 2
I am not being able to get the syntax of the thing.
Missing semi-colon at the end of the first line. Ditto with Commit Try:
START TRANSACTION;
..
..
COMMIT;
See START TRANSACTION, COMMIT, and ROLLBACK Syntax for a few examples.
...
...
as for your other issue, consider the below:
create table MyThings
( id int auto_increment primary key,
thing varchar(100) not null
);
insert MyThings(thing) values ('Fred, the pet Anchovy');
SET ucourse_id = LAST_INSERT_ID(); -- Error 1193: unknown sys var ...
SET #ucourse_id = LAST_INSERT_ID(); -- YIPPIE, not a problem (user variable)
So, the first one above (SET ucourse_id) choked, because it was assumed to be a LOCAL Variable (as it did not have an # sign). The whole thing was not running in a stored proc/function (I assumed). Local Variables need to have life breathed into them with a DECLARE.
but...
declare k int; -- error, can't do this outside of a store proc/func etc
So, one should read up on User Variables vs Local Variables, when and how one can use them.
Local Variables: DECLARE can be used in Stored Procedures, Functions, Events, and Triggers. But they need to occur grouped together at the top only, before any commands and typcially right after BEGIN. Otherwise, other errors will occur.
User Variables: (such as #myBirthday) No DECLARE is used with them. They are used free-wheeling with less restrictions, such as when you are just hacking around outside of Stored procs, functions, events, and triggers (but can certainly be used inside of them). Note, these are the only type of variables that will succeed with PREPARE, such as the PREPARE stmt001 FROM #theSql; part of it. This last fact is not typically figured out until one wastes a lot of time with it.

set range for column

I'm using in my database, many fields of a certain range, like:
CREATE TABLE figures (
deg FLOAT,-- between 0 and pi
prc FLOAT,-- between 0 and 1
.......
);
CREATE TRIGGER filter1 BEFORE UPDATE ON figures FOR EACH ROW SET
NEW.deg=IF(NEW.deg>3.1415926 OR NEW.deg<0, OLD.deg,NEW.deg),
NEW.prc=IF(NEW.prc>1 OR NEW.prc<0, OLD.prc,NEW.prc),
..........;
CREATE TRIGGER filter2 BEFORE INSERT ON figures FOR EACH ROW SET
NEW.deg=IF(NEW.deg>3.1415926 OR NEW.deg<0, NULL,NEW.deg),
NEW.prc=IF(NEW.prc>1 OR NEW.prc<0, NULL,NEW.prc),
.........;
Is there any way to write it more clearly ?
Something like:
--CREATE PROCEDURE/FUNCTION between()..................
CREATE TABLE figures (
deg FLOAT between(0,3.1415),
prc FLOAT between(0,1),
.......
At least, I don't want to write every filter twice. (ON INSERT,ON UPDATE)
prior to MySQL 8.0.16
Triggers are the best solution
Re:check constraints...
'The CHECK clause is parsed but ignored by all storage engines.'.....
'The reason for accepting but ignoring syntax clauses is for compatibility, to
make it easier to port code from other SQL servers, and to run applications
that create tables with references. '
lifted directly from: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html
From MySQL 8.0.16 though they now work as you would expect
CREATE TABLE figures (
deg FLOAT,
prc FLOAT,
CONSTRAINT `deg_min` CHECK ((`deg` > 0)),
CONSTRAINT `deg_max` CHECK ((`deg` < 3.1415)),
CONSTRAINT `prc_min` CHECK ((`prc` > 0)),
CONSTRAINT `prc_max` CHECK ((`prc` < 1))
)
At least, I don't want to write every filter twice. (ON INSERT,ON UPDATE)
You can write a stored function and call that in your trigger.
DELIMITER $$
CREATE FUNCTION check_deg (degree FLOAT, olddegree FLOAT) RETURNS FLOAT
BEGIN
DECLARE result FLOAT;
result = IF(degree>3.1415926 OR degree <0, olddegree,degree);
RETURN result;
END$$
DELIMITER ;
That way you have one point where the limits are defined and if anything changes you only have to change the boundaries in one place.
The best solution is to use a CHECK() constraint, but MySQL doesn't support CHECK() constraints. (MySQL parses them, then ignores them.)
In some cases, you can replace a CHECK() constraint with a foreign key reference to a table that contains all the valid values. Floating-point numbers are not a good candidate for that kind of solution, though.
That leaves triggers. In your case, your best bet is to use a trigger that calls a stored function.
I have a unfinished idea :
CREATE PROCEDURE check_constraint (table VARCHAR,field VARCHAR,condition VARCHAR)
BEGIN
SET #update_trigger = CONCAT ("
IF EXIST TRIGGER check_constraint_",table,"_",field,"
BEGIN /*I don't know yet what to do*/ END
ELSE /*IF TRIGGER DONT EXIST*/
BEGIN
CREATE TRIGGER check_constraint_",table,"_",field,"
BEFORE UPDATE ON ",table," FOR EACH ROW SET
NEW.",field,"=IF("condition, ", OLD.",field,",NEW.",field,");
END
");
PREPARE update_trigger FROM #update_trigger;
EXECUTE update_trigger;
DEALLOCATE PREPARE update_trigger;
SET #insert_trigger = ..............................
END
After we have a completed function, we can just call it during creation of the database:
CALL check_constraint("table","field","NEW.field<34567");

Filtering A MySQL Result Set Based On The Return Value Of A Function

I'm a SQL noob, and I need a little bit of help understanding the big picture of if it is possible, and if so how to go about filtering a result set based on the return value of a function which is passed one of the fields of the record.
Let's say I have a table called "Numbers" with just one field: "Value".
How could I correctly specify the following "pseudo-sql"?:
SELECT Value FROM numbers WHERE IsPrime(Value)=true
Can I accomplish such a thing, and if so, where/how do I put/store "IsPrime"?
I'm using MySQL.
I agree with extraneon that it's usually better to store this value in the database rather than compute it in the where clause. This way you can compute it once per row and index the data for faster performance.
As long as you are on MySQL 5.x, I would recommend a stored function, as opposed to a UDF. You can add an IS_PRIME column of type TINYINT to your DB, add an index on that column, then use the stored function to calculate the value at insert time. You can even calculate the IS_PRIME value using a before insert trigger if you don't want to change the insert code.
The stored function would look something like this:
DELIMITER $$
DROP FUNCTION IF EXISTS IS_PRIME $$
CREATE FUNCTION IS_PRIME(P_INT BIGINT) RETURNS TINYINT
BEGIN
DECLARE V_FACTOR BIGINT;
DECLARE V_MAX_FACTOR BIGINT;
SET V_FACTOR := 2;
SET V_MAX_FACTOR := round(sqrt(P_INT),0);
WHILE (V_FACTOR <= V_MAX_FACTOR)
DO
IF (P_INT % V_FACTOR) = 0
THEN
RETURN FALSE;
END IF;
SET V_FACTOR := V_FACTOR + 1;
END WHILE;
RETURN TRUE;
END $$
DELIMITER ;
I think you may find some help with the doc about the user-defined functions: http://dev.mysql.com/doc/refman/5.1/en/adding-functions.html
I don't know anything about user defined functions, but I could imagine that for computation-intensive functions it might be best to have that value precomputed and stored in the database somewhere.
Depending on how the data gets in the database you could require the client to compute isPrime (that can be done if the client is a web service or app on your own server), or perhaps a scheduled job which processes every record with isPrime is null or something like that. That is not instantaneous, but for some scenario's it may be good enough.
If isPrime is only used sometimes you could also post-process/filter the data on the client when retrieving data.

MySQL query browser procedure error code -1

I'm having a rather strange problem with MySQL. Trying to create a procedure to update some fields in the database (the code is below).
The problem is with the line that is currently commented. It seems that if no SELECT statements get executed during the procedure MySQL query browser will return an error code of "-1, error executing SQL query".
I tried the same thing in HeidiSQL and the error was "cannot return result set". So I suppose the question is do I always have to select something in the procedure, or is there some other thing I missed.
The query works fine when the comment is removed.
DELIMITER /
DROP PROCEDURE IF EXISTS updateFavourites /
CREATE PROCEDURE updateFavourites(quota INT)
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE artist_id,releases INT;
DECLARE c_artist Cursor FOR
SELECT Artist.id_number,COUNT(Artist.id_number) FROM Artist
JOIN CD ON CD.is_fronted_by = Artist.id_number
GROUP BY Artist.id_number;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000'
SET done=1;
IF quota > 0 THEN
OPEN c_artist;
REPEAT
FETCH c_artist INTO artist_id,releases;
IF NOT done THEN
IF releases >= quota THEN
UPDATE CD SET CD.rating='favourite' WHERE CD.is_fronted_by = artist_id;
END IF;
END IF;
UNTIL done END REPEAT;
CLOSE c_artist;
-- SELECT 'Great success';
ELSE
SELECT CONCAT('\'quota\' must be greater than 0.',' Got (',quota,')');
END IF;
END /
DELIMITER ;
Here's the sql to create the tables and some data:
DROP TABLE IF EXISTS CD;
DROP TABLE IF EXISTS Artist;
CREATE TABLE Artist (
id_number INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(50),
);
CREATE TABLE CD (
catalog_no INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,
is_fronted_by INT UNSIGNED,
rating ENUM ('favourite','top draw','good','so-so','poor','rubbish'),
CONSTRAINT fk_CD_Artist FOREIGN KEY (is_fronted_by) REFERENCES Artist(id_number) ON UPDATE CASCADE
);
INSERT INTO Artist VALUES(11,'Artist 1');
INSERT INTO Artist VALUES(10,'Artist 2');
INSERT INTO CD VALUES (7,11, 'top draw');
INSERT INTO CD VALUES (650,11,'good');
INSERT INTO CD VALUES (651,11,'good');
INSERT INTO CD VALUES (11,10,'favourite');
Query Browser is not for running scripts, just single query.
I tried your code by moving cursor into each query (except DELIMITER) and pressing Ctrl+Enter.
It created that stored procedure without problem. (just refresh schema on the left).
If you wish creating procedure, use menu "Script"->"Create stored procedure/function".
But better forget about QueryBrowser it is not supported at all (and actunally not useful).
If you have decent hardware and plenty resources, try Workbench 5.2 otherwise use SQLyog
Googling around, there are several reports of the same error, but little information to solve the problem. There's even a bug logged at mysql.com but it appears to have been abandoned without being resolved.
There's another StackOverflow question on the same error, but it's also unresolved.
All it means is that there is no result set from the query. Looking at the source code, it appears that sometimes an error status of MYX_SQL_ERROR is set when the query has no result set. Perhaps this is not an appropriate consequence?
I notice that when I use the mysql command-line client, it yields no error for calling a proc that returns no result set.
update: I tried to revive that MySQL bug report, and provide a good test case for them. They changed the bug from "no feedback" to "verified" -- so at least they acknowledge it's a bug in Query Browser:
[11 Dec 9:18] Sveta Smirnova
Bill,
thank you for the feedback. Verified
as described.
Although most likely this only be
fixed when MySQL Query Browser
functionality is part of MySQL
workbench.
I guess the workaround is to ignore the -1 error, or to test your stored procedures in the command-line mysql client, where the error does not occur.
The comment supposes the issue will disappear as the Query Browser functionality becomes part of MySQL Workbench. This is supposed to happen in MySQL Workbench 5.2. I'll download this beta and give it a try.
MySQL Workbench 5.2 is in Beta, but I would assume MySQL engineering can't predict when the Beta will become GA. Those kinds of predictions are hard enough under standard conditions, but there's a lot of extra uncertainty of MySQL's fate due to the unresolved Oracle acquisition.
update: Okay, I have tried MySQL Workbench 5.2.10 beta. I executed a stored procedure like this:
CREATE PROCEDURE FooProc(doquery SMALLINT)
BEGIN
IF doquery THEN
SELECT * FROM Foo;
END IF;
END
When I CALL FooProc(0) the response is no result set, and the status is simply "OK".
When I CALL FooProc(1) the response is the result of SELECT * FROM Foo as expected.
However, there's another bug related to calling procedures. Procedures may have multiple result sets, so it's hard to know when to close the statement when you execute a CALL query. The consequence is that MySQL Workbench 5.2 doesn't close the statement, and if you try to do another query (either CALL or SELECT) it gives you an error:
Commands out of sync; you can't run this command now.
MySQL doesn't support multiple concurrent open queries. So the last one must be closed before you can start a new one. But it isn't closing the CALL query. This bug is also logged at the MySQL site.
The bug about commands out of sync has been resolved. They say it's fixed in MySQL Workbench 5.2.11.
Try putting BEGIN and END blocks around the multiple statements in the IF block as such:
IF quota > 0 THEN
BEGIN
OPEN c_artist;
REPEAT
FETCH c_artist INTO artist_id,releases;
IF NOT done THEN
IF releases >= quota THEN
UPDATE CD SET CD.rating='favourite' WHERE CD.is_fronted_by = artist_id;
END IF;
END IF;
UNTIL done END REPEAT;
CLOSE c_artist;
END;
ELSE
SELECT CONCAT('\'quota\' must be greater than 0.',' Got (',quota,')');
END IF;