My program require to generate UNIQUE transaction Id through MySQL due to multiple machine environment.
I using the following MySQL function, I google review it is not atomic as I think it is.
DELIMITER $$
CREATE DEFINER=`` FUNCTION `getNextTXID`(dummy INT) RETURNS int(11)
DETERMINISTIC
BEGIN
DECLARE txid bigint(20);
SET txid = (SELECT next_txid FROM txid_seq LIMIT 1 FOR UPDATE);
IF txid > 15000000 THEN
SET txid = 0;
END IF;
UPDATE txid_seq SET next_txid = txid + 500 LIMIT 1;
RETURN txid;
END
I previously using last_insert_id, but a new requirement to reset after 15M number was imposed. I cannot reproduce a race condition any 2 of 100 Processes actually get the same transaction number (in a batch of 500, if application used up all 500, get again).
Question:
Does this function atomic
Any other way of doing it correctly?
Table : MyISAM
Storage Engine : MyISAM
Auto commit : TRUE
Edit:
I am using MySQL C API.
Thanks in advance for any apply.
It's not atomic, because you're reading and writing separately. But this should do the same operation atomically, while still returning the value from LAST_INSERT_ID():
UPDATE txid_seq SET next_txid = LAST_INSERT_ID((next_txid * (next_txid <= 15000000)) + 500) LIMIT 1;
Related
I'm having an issue with one very specific procedure I am working on, either it takes roughly about 45 seconds to run and or it takes over minutes to complete. I know what the issue is, but at this time I am not sure how to get around it to be honest. In the SP, you will see a section (while block) commented out that basically create's a token to be inserted into the table, currently I am inserting just TEST to get around the blocking issue.
Here's the current working version that runs fine by just inserting a dummy value.
BEGIN
DECLARE userexist INT(1);
DECLARE tokenID VARCHAR(12) default '';
DECLARE rnd_str text;
DECLARE ready int default 0;
DECLARE userID INT(20) default 0;
DECLARE lastID int default 0;
SELECT
su.User_ID,
1 AS user_exist
INTO userID, userexist
FROM System_User su
WHERE AES_DECRYPT(su.User_Email, 'AAAA') = email
AND AES_DECRYPT(su.User_Password, 'AAAA') = userpassword
AND su.User_Is_Active = 1;
IF userexist = 1 THEN
UPDATE System_User_Login
SET System_User_Login.System_Logout_Time = NOW(),
System_User_Login.System_Logout_Type = 1
WHERE System_User_Login.System_Login_User_ID = userID;
SET tokenID = 'TEST'; -- dummy value for now
INSERT INTO System_User_Login(System_Login_Time, System_Login_Token, System_Login_User_ID, System_Token_Valid_Period)
VALUES(NOW(), tokenID, userID, (NOW() + INTERVAL 30 DAY));
SET lastID = LAST_INSERT_ID();
-- WHILE ready = 0 DO
-- SET rnd_str = lpad(conv(floor(rand()*pow(36,16)), 12, 36), 12, 0);
-- IF NOT EXISTS (SELECT 1 FROM System_User_Login sul WHERE sul.System_Login_Token = rnd_str) THEN
-- update System_User_Login
-- SET System_Login_Token = rnd_str
-- WHERE System_Login_ID = lastID;
-- SET ready = 1;
-- END IF;
-- END WHILE;
SELECT
sul.System_Login_Token AS Token
FROM System_User_Login sul
WHERE sul.System_Login_ID = lastID;
ELSE
SELECT '0' AS Token;
END IF;
END
The while loop inside this procedure is to create a new token to be assigned; I've tried before and after the insert to no avail; even triggers. I must not be seeing something and or need to rethink my approach.
If your user list is high then the probability of this collision will increase. This will cause multiple iterations. You can avoid that by following
1: You can use Universal unique identifiers if working on MySQL 8 but make sure length constraints matches as per your requirement.
UUID
UUID Short
2: Try the following
Unique Identifier = CONCAT(<some_random_string>, unix_timestamp, <primary_key_id>)
So if you are setting token for user = 4 then
SELECT LPAD(CONCAT(CONV(FLOOR(RAND()* POW(36,16)), 12, 36), UNIX_TIMESTAMP(), '4'), 25, 0);
If you are not generating more than one token in one sec then UNIX_TIMESTAMP will make it unique.
Further, if we are generating more than one token per sec then we reduce collision by generating a random number using CONV(FLOOR(RAND()* POW(36,16)), 12, 36)
If by chance, it generates the same random string for multiple, we have a unique id for each user, which will definitely make it unique.
Assumption: You are not creating multiple tokens for the same user in one second if so there might be chances of getting a collision.
Note: If you use the second approach, you can use two SQL statements for your complete procedure. Here is a way
I'd like to have a tricky SQL statement as an Event that runs every couple of minutes.
Currently, I'm doing so with Java, using 3 separate statements that executing sequentiality in a transaction connection.
Q: I don't know how to construct such an SQL statement without Java. If impossible to have a single SQL statement, I'd like to use transaction (as I'm using in Java) and rollback in case of failure in any of those separate statements.
My Case:
I have 3 tables: "Factory", "Plan", "Machine".
I want to do something as below:
1.
WHERE Machines.annualCheck == "TRUE"
SET Machine.status = "IN_ANNUAL_CHECK"
For machines that got updated I need to do the following:
2.1 Update the related factory
WHERE Factory.id == Machine.linkFactoryID
UPDATE Factory.totalActiveMachines = --1
2.2 Delete the upcoming plans that planned to be handled by the related machine
DELETE rows WHERE Plan.willHandleByMachineID = Machine.ID
p.s. I'm using MySQL
Thank you!
Update:
In following to Simonare suggestion, I tired to do the following:
DELIMITER $
CREATE PROCEDURE annualCheck(IN Machine_ID int, IN Factory_ID int)
BEGIN
UPDATE machine_table
SET machine_table.annualCheck = 'IN_ANNUAL_CHECK'
WHERE machine_table.machine_id = Machine_ID;
UPDATE factory_table
SET factory_table.totalActiveMachines = factory_table.totalActiveMachines - 1
WHERE factory_table.factory_id = Factory_ID;
DELETE FROM plan_table WHERE plan_table.assign_to_machine = Machine_ID
END$
DELIMITER $$
BEGIN
SELECT #m_id = machine_id, #f_id = link_factory_id
FROM machine_table
WHERE machine_table.annualCheck = 'TRUE';
END$$
CALL annualCheck(#m_id,#f_id)
I don't know why, but I'm running into syntax errors - one after the other.
It's my first time to use PROCEDURE and DELIMITER. Am I doing it right?
you can use stored procedure
delimiter //
CREATE PROCEDURE myProc (IN Machine_ID int)
BEGIN
UPDATE myTable
SET Machine.status = "IN_ANNUAL_CHECK"
WHERE Machines.annualCheck == "TRUE";
Update the related factory
WHERE Factory.id == Machine.linkFactoryID
UPDATE Factory.totalActiveMachines = totalActiveMachines -1;
DELETE FROM Plan WHERE Plan.willHandleByMachineID = Machine_ID;
END//
then you can execute it either from mysql
CALL simpleproc(#a);
or from Java
It is also possible to create trigger on the Machine table, something like this:
CREATE TRIGGER `TRG_Machines_AfterUpdate` AFTER UPDATE ON `Machine` FOR EACH ROW BEGIN
IF OLD.annualCheck = 'TRUE' AND NEW.annualCheck = 'IN_ANNUAL_CHECK' THEN
UPDATE
Factory
SET
totalActiveMachines = totalActiveMachines - 1
WHERE
id = NEW.linkFactoryID
;
DELETE FROM
Plan
WHERE
willHandleByMachineID = NEW.ID
;
END;
END
So you can just issue normal update:
UPDATE Machine SET annualCheck = 'IN_ANNUAL_CHECK' WHERE annualCheck = 'TRUE'
I searched a lot of doing a task but found no appropriate solution.
Basically the scenario is. I have a user_comment table in which there are 5 column(id,parent_id,user_comments,is_deleted,modified_datetime). There is a parent child relationship like 1->2,1->3,2->4,2->5,5->7 etc. Now i am sending the id from the front end and i want to update the column is_deleted to 1 and modified_datetime on all the records on
this id as well as the all the children and children's of children.
I am trying to doing this by using a recursive procedure. Below is the code of my procedure
CREATE DEFINER=`root`#`localhost` PROCEDURE `user_comments`(
IN mode varchar(45),
IN comment_id int,
)
BEGIN
DECLARE p_id INT DEFAULT NULL ;
if(mode = 'delete')
then
update user_comment set is_deleted = 1, modified_datetime = now()
where id = comment_id ;
select id from user_comment where parent_id = comment_id into p_id ;
if p_id is not null
then
SET ##GLOBAL.max_sp_recursion_depth = 255;
SET ##session.max_sp_recursion_depth = 255;
call user_comments('delete', p_id);
end if;
end if;
END
By using this procedure it give me an error of more than one row.
If i return the select query without giving it to variable then shows me the the appropriate results on the select query but i have to call this procedure recursively based on getting the ids of the select query.
I need help i have already passed 2 days into this.
I used cursor also. Below is the code of cursor
CREATE DEFINER=`root`#`localhost` PROCEDURE `user_comments`(
IN mode varchar(45),
IN comment_id int,
)
BEGIN
DECLARE p_emp int;
DECLARE noMoreRow INT;
DECLARE cur_emp CURSOR FOR select id from user_comment where parent_id = comment_id ;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET noMoreRow = 0;
if(mode = 'delete')
then
OPEN cur_emp;
LOOPROWS: LOOP
IF noMoreRow = 0 THEN
update user_comment set is_deleted = 1, modified_datetime = now() where id = comment_id
CLOSE cur_emp;
LEAVE LOOPROWS;
END IF;
FETCH cur_emp INTO p_emp;
update user_comment set is_deleted = 1, modified_datetime = now() where id = p_emp ;
SET ##GLOBAL.max_sp_recursion_depth = 255;
SET ##session.max_sp_recursion_depth = 255;
call user_comments('delete', p_emp);
END LOOP;
end if;
END
After using cursor i am getting a thread error.i don't know how can overcome this problem!!!
Mysql's documentation on select ... into varlist clearly says:
The selected values are assigned to the variables. The number of
variables must match the number of columns. The query should return a
single row. If the query returns no rows, a warning with error code
1329 occurs (No data), and the variable values remain unchanged. If
the query returns multiple rows, error 1172 occurs (Result consisted
of more than one row). If it is possible that the statement may
retrieve multiple rows, you can use LIMIT 1 to limit the result set to
a single row.
Since you wrote in the OP that a comment can be parent of many comments, using simple variables cannot be a solution. You should use a CURSOR instead, that can store an entire resultset.
You loop through the records within the cursos as shown in the sample code in the above link and call user_comments() in a recursive way.
UPDATE
If your receive
Error Code: 1436. Thread stack overrun
error, then you can do 2 things:
Increase the thread_stack setting in the config file and restart mysql server.
You can try to simplify your code to use less recursions and therefore less stack space. For example, when you fetch all children into the cursor, then rather calling the user_comments() recursively for each, you can set all direct children's status within the code and call the function recirsively on grand-childrens only (if any). You can also change your data structure and use nested set model to approach hierarchical structures.
Nested set model is more complex to understand, it is less resource intensive to traverse, but more resource intensive to maintain.
I have written a procedure which uses cursor to loop through the rows. It is taking too long to execute.
CREATE PROCEDURE test_port()
BEGIN
declare done BOOL default FALSE;
declare I,J,C,P,NOB int default 0;
declare n,k,t int default 0;
declare Lid int default 0;
declare inTS timestamp;
select max(id) into n from MAIN_TBL;
select ctrValue into k from ID_CNT;
set k=k+1;
WHILE k<=n
do
select SourcePort,DestPort,LinkID,NoOfBytes,insertTime into I,J,Lid,NOB,inTS from MAIN_TBL where id=k;
select count(*) into t from APP_PORTMAP_MSTR where Port in (I,J);
IF(t=1) THEN
select Port into P from APP_PORTMAP_MSTR where Port in (I,J);
SET C=0;
select count(*) INTO C from LINK_APP_TBL where LinkID=Lid and Port=P;
insert into TRAFFIC_HIST_TBL(LinkID,Port,NoOfBytes,Time_1) values(Lid,P,NOB,inTS);
IF(C=0) THEN
insert into LINK_APP_TBL(Port,LinkID) values(P,Lid);
END IF;
ELSE
if(I>J && J<>0) THEN
SET C=0;
select count(*) INTO C from LINK_APP_TBL where LinkID=Lid and Port=J;
IF(C=0) THEN
insert into LINK_APP_TBL(Port,LinkID) values(J,Lid);
insert into TRAFFIC_HIST_TBL(LinkID,Port,NoOfBytes,Time_1) values(Lid,J,NOB,inTS);
END IF;
ELSE
SET C=0;
select count(*) INTO C from LINK_APP_TBL where LinkID=Lid and Port=I;
IF(C=0) THEN
insert into LINK_APP_TBL(Port,LinkID) values(I,Lid);
insert into TRAFFIC_HIST_TBL(LinkID,Port,NoOfBytes,Time_1) values(Lid,I,NOB,inTS);
END IF;
END IF;
END IF;
SET k=k+1;
END WHILE;
END$$
\d ;
The possible reason for being slow is the "Insert" statements but can we improve the performance in any ways ? It is processing around 10K records at a time.
10k per second is a big number... you can try delete some indexes that you are not using or you can move your data files to faster disk (like SSD) that makes a huge difference...
Well, after carefully analysing the processlist, I found that "query end" state is taking very long time.
Hence, after searching some threads I found that setting innodb_flush_log_at_trx_commit to either 0 or 2 will speed up the operations but they prohibit innodb to be ACID complaint.
Changing the values may result into 1 instruction loss in case of OS crash or power failure.
Ref:
UPDATE statements are in "query end state"
I am currently running the following stored procedure. While it is a lot more efficient than my original procedure it is still taking an excessive amount of time. I'm not actually sure what the slow down is as the first 10k-30k records happened fast, but it has grown slower and slower as it gets further in. I'm expecting to update about 250k rows of about 1.7 million. Once this is complete I will then be doing something similar to insert records into each "Solar System".
To give you an example of the time this is taking. It has now been running for a little over 24 hours and it is only on iteration 786 of the 1716 it has to do. The reason for the changing limit on the selects is that there are 1000 possible rows per a sector in my table. I don't personally see any slow downs, but then I don't understand the inner workings of MySQL that well.
This work is being done on my local computer, no it is not slow, but there is always the possibility that there are changes that need to be done at the server level that would make these queries more efficient. If need be I can change the server settings so that is a possibility also. FYI I'm using the stock configuration from MySQL on a Windows 7.
DECLARE CurrentOffset int; -- Current offset limit to only deal with one
DECLARE CurrentOffsetMultiplier int;
DECLARE RandRow int; -- Random Row to make a Solar System with
DECLARE CheckSystemExists int; -- Used to insure RandRow is not already a Solar System Row
DECLARE TotalSystemLoops int; -- Total number of loops so each Galaxy gets it's systems.
DECLARE RandomSolarSystemCount int; -- This is the number of Solar Systems that will be in each Galaxy;
DECLARE UpdateSolarCount int;
DECLARE NumberOfOffsets int;
SET CurrentOffsetMultiplier = 0;
SET NumberOfOffsets = 1716;
SET CurrentOffset = 0;
OffsetLoop: LOOP
SET UpdateSolarCount = 0;
/*Sets the amount of Solary Systems going in a Galaxy*/
CheckRandomSolarSystemCount: LOOP
SET RandomSolarSystemCount = FLOOR(125 + RAND() * (175 - 125) + 1);
IF RandomSolarSystemCount >= 125 THEN
IF RandomSolarSystemCount <= 175 THEN
LEAVE CheckRandomSolarSystemCount;
END IF;
END IF;
END LOOP;
UpdateGalaxyWithSolarSystems: LOOP
SET UpdateSolarCount = UpdateSolarCount + 1;
IF UpdateSolarCount > RandomSolarSystemCount THEN
LEAVE UpdateGalaxyWithSolarSystems;
END IF;
/*Sets RandRow and CheckSystemExists*/
CheckExistsLoop: Loop
SET RandRow = FLOOR(0 + RAND() * (1000)+ 1);
SET CheckSystemExists = (SELECT COUNT(*)
FROM
(SELECT * FROM
(SELECT * FROM galaxies2 LIMIT CurrentOffset, 1000) AS LimitedTable
LIMIT RandRow ,1) AS RandTable
WHERE SolarSystemName IS NULL);
IF CheckSystemExists THEN
LEAVE CheckExistsLoop;
END IF;
END LOOP;
/*Updates the tables SolarSystemName column with a default system name*/
UPDATE galaxies2
SET SolarSystemName = CONCAT("Solar System ", RandRow)
WHERE galaxies2.idGalaxy =
(SELECT LimitedTable.idGalaxy AS GalaxyID FROM
(SELECT galaxies2.idGalaxy FROM galaxies2 LIMIT CurrentOffset, 1000) AS LimitedTable
LIMIT RandRow ,1)
;
END LOOP;
SET CurrentOffsetMultiplier = CurrentOffsetMultiplier + 1;
SET CurrentOffset = CurrentOffsetMultiplier * 1000;
IF CurrentOffsetMultiplier = 1717 THEN
LEAVE OffsetLoop;
END IF;
END LOOP;
It's getting slower and slower because you are "walking" through the galaxies2 table.
SELECT * FROM galaxies2 LIMIT CurrentOffset, 1000
As the CurrentOffset value increases, MySQL has to "walk" through more and more records to get to the starting point. You may actually be able to get a speed boost by specifying an ORDER BY on the primary key. You would want to have an ORDER BY anyway since MySQL just reads records randomly if no order is specified. It does not read the records in any particular order so you could (though unlikely) get the same set of records in different offsets.
It would be better to specify a range on the auto increment field. Assuming you have one. Then the first and last queries should perform about the same. It's not as ideal since there could be gaps from deleted records.
SELECT * FROM galaxies2 WHERE auto_incr_field BETWEEN CurrentOffset AND CurrentOffset+1000