I'm having an issue with one very specific procedure I am working on, either it takes roughly about 45 seconds to run and or it takes over minutes to complete. I know what the issue is, but at this time I am not sure how to get around it to be honest. In the SP, you will see a section (while block) commented out that basically create's a token to be inserted into the table, currently I am inserting just TEST to get around the blocking issue.
Here's the current working version that runs fine by just inserting a dummy value.
BEGIN
DECLARE userexist INT(1);
DECLARE tokenID VARCHAR(12) default '';
DECLARE rnd_str text;
DECLARE ready int default 0;
DECLARE userID INT(20) default 0;
DECLARE lastID int default 0;
SELECT
su.User_ID,
1 AS user_exist
INTO userID, userexist
FROM System_User su
WHERE AES_DECRYPT(su.User_Email, 'AAAA') = email
AND AES_DECRYPT(su.User_Password, 'AAAA') = userpassword
AND su.User_Is_Active = 1;
IF userexist = 1 THEN
UPDATE System_User_Login
SET System_User_Login.System_Logout_Time = NOW(),
System_User_Login.System_Logout_Type = 1
WHERE System_User_Login.System_Login_User_ID = userID;
SET tokenID = 'TEST'; -- dummy value for now
INSERT INTO System_User_Login(System_Login_Time, System_Login_Token, System_Login_User_ID, System_Token_Valid_Period)
VALUES(NOW(), tokenID, userID, (NOW() + INTERVAL 30 DAY));
SET lastID = LAST_INSERT_ID();
-- WHILE ready = 0 DO
-- SET rnd_str = lpad(conv(floor(rand()*pow(36,16)), 12, 36), 12, 0);
-- IF NOT EXISTS (SELECT 1 FROM System_User_Login sul WHERE sul.System_Login_Token = rnd_str) THEN
-- update System_User_Login
-- SET System_Login_Token = rnd_str
-- WHERE System_Login_ID = lastID;
-- SET ready = 1;
-- END IF;
-- END WHILE;
SELECT
sul.System_Login_Token AS Token
FROM System_User_Login sul
WHERE sul.System_Login_ID = lastID;
ELSE
SELECT '0' AS Token;
END IF;
END
The while loop inside this procedure is to create a new token to be assigned; I've tried before and after the insert to no avail; even triggers. I must not be seeing something and or need to rethink my approach.
If your user list is high then the probability of this collision will increase. This will cause multiple iterations. You can avoid that by following
1: You can use Universal unique identifiers if working on MySQL 8 but make sure length constraints matches as per your requirement.
UUID
UUID Short
2: Try the following
Unique Identifier = CONCAT(<some_random_string>, unix_timestamp, <primary_key_id>)
So if you are setting token for user = 4 then
SELECT LPAD(CONCAT(CONV(FLOOR(RAND()* POW(36,16)), 12, 36), UNIX_TIMESTAMP(), '4'), 25, 0);
If you are not generating more than one token in one sec then UNIX_TIMESTAMP will make it unique.
Further, if we are generating more than one token per sec then we reduce collision by generating a random number using CONV(FLOOR(RAND()* POW(36,16)), 12, 36)
If by chance, it generates the same random string for multiple, we have a unique id for each user, which will definitely make it unique.
Assumption: You are not creating multiple tokens for the same user in one second if so there might be chances of getting a collision.
Note: If you use the second approach, you can use two SQL statements for your complete procedure. Here is a way
Related
I tried to write a SQL-function that generates an unused unique ID in a range between 1000000 and 4294967295. I need numeric values, so UUID() alike is not a solution. It doesn't sound that difficult, but for some reason, the code below does not work when called within an INSERT-statement on a table as value for the primary key (not auto_increment, of course). The statement is like INSERT INTO table (id, content) VALUES ((SELECT getRandomID(0,0)), 'blabla bla');
(Since default values are not allowed in such functions, I shortly submit 0 for each argument and set it in the function to the desired value.)
Called once and separated from INSERT or Python-code, everything is fine. Called several times, something weird happens and not only the whole process but also the server might hang within REPEAT. The process is then not even possible to kill/restart; I have to reboot the machine -.-
It also seems to only have some random values ready for me, since the same values appear again and again after some calls, allthough I actually thought that the internal rand() would be a sufficient start/seed for the outer rand().
Called from Python, the loop starts to hang after some rounds although the very first one in my tests always produces a useful, new ID and therefore should quit after the first round. Wyh? Well, the table is empty...so SELECT COUNT(*)... returns 0 which actually is the signal for leaving the loop...but it doesn't.
Any ideas?
I'm running MariaDB 10.something on SLES 12.2. Here is the exported source code:
DELIMITER $$
CREATE DEFINER=`root`#`localhost` FUNCTION `getRandomID`(`rangeStart` BIGINT UNSIGNED, `rangeEnd` BIGINT UNSIGNED) RETURNS bigint(20) unsigned
READS SQL DATA
BEGIN
DECLARE rnd BIGINT unsigned;
DECLARE i BIGINT unsigned;
IF rangeStart is null OR rangeStart < 1 THEN
SET rangeStart = 1000000;
END IF;
IF rangeEnd is null OR rangeEnd < 1 THEN
SET rangeEnd = 4294967295;
END IF;
SET i = 0;
r: REPEAT
SET rnd = FLOOR(rangeStart + RAND(RAND(FLOOR(1 + rand() * 1000000000))*10) * (rangeEnd - rangeStart));
SELECT COUNT(*) INTO i FROM `table` WHERE `id` = rnd;
UNTIL i = 0 END REPEAT r;
RETURN rnd;
END$$
DELIMITER ;
A slight improvement:
SELECT COUNT(*) INTO i FROM `table` WHERE `id` = rnd;
UNTIL i = 0 END REPEAT r;
-->
UNTIL NOT EXISTS( SELECT 1 FROM `table` WHERE id = rnd ) REPEAT r;
Don't pass any argument to RAND -- that is for establishing a repeatable sequence of random numbers.
mysql> SELECT RAND(123), RAND(123), RAND(), RAND()\G
*************************** 1. row ***************************
RAND(123): 0.9277428611440052
RAND(123): 0.9277428611440052
RAND(): 0.5645420109522921
RAND(): 0.12561983719991504
1 row in set (0.00 sec)
So simplify to
SET rnd = FLOOR(rangeStart + RAND() * (rangeEnd - rangeStart));
If you want to include rangeEnd in the possible outputs, add 1:
SET rnd = FLOOR(rangeStart + RAND() * (rangeEnd - rangeStart + 1));
I searched a lot of doing a task but found no appropriate solution.
Basically the scenario is. I have a user_comment table in which there are 5 column(id,parent_id,user_comments,is_deleted,modified_datetime). There is a parent child relationship like 1->2,1->3,2->4,2->5,5->7 etc. Now i am sending the id from the front end and i want to update the column is_deleted to 1 and modified_datetime on all the records on
this id as well as the all the children and children's of children.
I am trying to doing this by using a recursive procedure. Below is the code of my procedure
CREATE DEFINER=`root`#`localhost` PROCEDURE `user_comments`(
IN mode varchar(45),
IN comment_id int,
)
BEGIN
DECLARE p_id INT DEFAULT NULL ;
if(mode = 'delete')
then
update user_comment set is_deleted = 1, modified_datetime = now()
where id = comment_id ;
select id from user_comment where parent_id = comment_id into p_id ;
if p_id is not null
then
SET ##GLOBAL.max_sp_recursion_depth = 255;
SET ##session.max_sp_recursion_depth = 255;
call user_comments('delete', p_id);
end if;
end if;
END
By using this procedure it give me an error of more than one row.
If i return the select query without giving it to variable then shows me the the appropriate results on the select query but i have to call this procedure recursively based on getting the ids of the select query.
I need help i have already passed 2 days into this.
I used cursor also. Below is the code of cursor
CREATE DEFINER=`root`#`localhost` PROCEDURE `user_comments`(
IN mode varchar(45),
IN comment_id int,
)
BEGIN
DECLARE p_emp int;
DECLARE noMoreRow INT;
DECLARE cur_emp CURSOR FOR select id from user_comment where parent_id = comment_id ;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET noMoreRow = 0;
if(mode = 'delete')
then
OPEN cur_emp;
LOOPROWS: LOOP
IF noMoreRow = 0 THEN
update user_comment set is_deleted = 1, modified_datetime = now() where id = comment_id
CLOSE cur_emp;
LEAVE LOOPROWS;
END IF;
FETCH cur_emp INTO p_emp;
update user_comment set is_deleted = 1, modified_datetime = now() where id = p_emp ;
SET ##GLOBAL.max_sp_recursion_depth = 255;
SET ##session.max_sp_recursion_depth = 255;
call user_comments('delete', p_emp);
END LOOP;
end if;
END
After using cursor i am getting a thread error.i don't know how can overcome this problem!!!
Mysql's documentation on select ... into varlist clearly says:
The selected values are assigned to the variables. The number of
variables must match the number of columns. The query should return a
single row. If the query returns no rows, a warning with error code
1329 occurs (No data), and the variable values remain unchanged. If
the query returns multiple rows, error 1172 occurs (Result consisted
of more than one row). If it is possible that the statement may
retrieve multiple rows, you can use LIMIT 1 to limit the result set to
a single row.
Since you wrote in the OP that a comment can be parent of many comments, using simple variables cannot be a solution. You should use a CURSOR instead, that can store an entire resultset.
You loop through the records within the cursos as shown in the sample code in the above link and call user_comments() in a recursive way.
UPDATE
If your receive
Error Code: 1436. Thread stack overrun
error, then you can do 2 things:
Increase the thread_stack setting in the config file and restart mysql server.
You can try to simplify your code to use less recursions and therefore less stack space. For example, when you fetch all children into the cursor, then rather calling the user_comments() recursively for each, you can set all direct children's status within the code and call the function recirsively on grand-childrens only (if any). You can also change your data structure and use nested set model to approach hierarchical structures.
Nested set model is more complex to understand, it is less resource intensive to traverse, but more resource intensive to maintain.
I am currently running the following stored procedure. While it is a lot more efficient than my original procedure it is still taking an excessive amount of time. I'm not actually sure what the slow down is as the first 10k-30k records happened fast, but it has grown slower and slower as it gets further in. I'm expecting to update about 250k rows of about 1.7 million. Once this is complete I will then be doing something similar to insert records into each "Solar System".
To give you an example of the time this is taking. It has now been running for a little over 24 hours and it is only on iteration 786 of the 1716 it has to do. The reason for the changing limit on the selects is that there are 1000 possible rows per a sector in my table. I don't personally see any slow downs, but then I don't understand the inner workings of MySQL that well.
This work is being done on my local computer, no it is not slow, but there is always the possibility that there are changes that need to be done at the server level that would make these queries more efficient. If need be I can change the server settings so that is a possibility also. FYI I'm using the stock configuration from MySQL on a Windows 7.
DECLARE CurrentOffset int; -- Current offset limit to only deal with one
DECLARE CurrentOffsetMultiplier int;
DECLARE RandRow int; -- Random Row to make a Solar System with
DECLARE CheckSystemExists int; -- Used to insure RandRow is not already a Solar System Row
DECLARE TotalSystemLoops int; -- Total number of loops so each Galaxy gets it's systems.
DECLARE RandomSolarSystemCount int; -- This is the number of Solar Systems that will be in each Galaxy;
DECLARE UpdateSolarCount int;
DECLARE NumberOfOffsets int;
SET CurrentOffsetMultiplier = 0;
SET NumberOfOffsets = 1716;
SET CurrentOffset = 0;
OffsetLoop: LOOP
SET UpdateSolarCount = 0;
/*Sets the amount of Solary Systems going in a Galaxy*/
CheckRandomSolarSystemCount: LOOP
SET RandomSolarSystemCount = FLOOR(125 + RAND() * (175 - 125) + 1);
IF RandomSolarSystemCount >= 125 THEN
IF RandomSolarSystemCount <= 175 THEN
LEAVE CheckRandomSolarSystemCount;
END IF;
END IF;
END LOOP;
UpdateGalaxyWithSolarSystems: LOOP
SET UpdateSolarCount = UpdateSolarCount + 1;
IF UpdateSolarCount > RandomSolarSystemCount THEN
LEAVE UpdateGalaxyWithSolarSystems;
END IF;
/*Sets RandRow and CheckSystemExists*/
CheckExistsLoop: Loop
SET RandRow = FLOOR(0 + RAND() * (1000)+ 1);
SET CheckSystemExists = (SELECT COUNT(*)
FROM
(SELECT * FROM
(SELECT * FROM galaxies2 LIMIT CurrentOffset, 1000) AS LimitedTable
LIMIT RandRow ,1) AS RandTable
WHERE SolarSystemName IS NULL);
IF CheckSystemExists THEN
LEAVE CheckExistsLoop;
END IF;
END LOOP;
/*Updates the tables SolarSystemName column with a default system name*/
UPDATE galaxies2
SET SolarSystemName = CONCAT("Solar System ", RandRow)
WHERE galaxies2.idGalaxy =
(SELECT LimitedTable.idGalaxy AS GalaxyID FROM
(SELECT galaxies2.idGalaxy FROM galaxies2 LIMIT CurrentOffset, 1000) AS LimitedTable
LIMIT RandRow ,1)
;
END LOOP;
SET CurrentOffsetMultiplier = CurrentOffsetMultiplier + 1;
SET CurrentOffset = CurrentOffsetMultiplier * 1000;
IF CurrentOffsetMultiplier = 1717 THEN
LEAVE OffsetLoop;
END IF;
END LOOP;
It's getting slower and slower because you are "walking" through the galaxies2 table.
SELECT * FROM galaxies2 LIMIT CurrentOffset, 1000
As the CurrentOffset value increases, MySQL has to "walk" through more and more records to get to the starting point. You may actually be able to get a speed boost by specifying an ORDER BY on the primary key. You would want to have an ORDER BY anyway since MySQL just reads records randomly if no order is specified. It does not read the records in any particular order so you could (though unlikely) get the same set of records in different offsets.
It would be better to specify a range on the auto increment field. Assuming you have one. Then the first and last queries should perform about the same. It's not as ideal since there could be gaps from deleted records.
SELECT * FROM galaxies2 WHERE auto_incr_field BETWEEN CurrentOffset AND CurrentOffset+1000
Is it possible to have a query wait until it has results instead of returning immediately with an empty result set. For example:
SELECT Column1 FROM Table1
If Table1 is empty, the query would return with an empty result set. However, I want it to not return, but wait until at least one row is available, preferably with a timeout of some sort. I would prefer to do this without involving Service Broker into the equation, if possible.
Clarification:
CLR is enabled on the server, but the call is coming from a platform independent C++ program via SQLAPI++/ODBC. So, no C#/.NET tricks are possible. The goal is to make a call into a stored procedure, specify a timeout and not return until data is available (and returned by the stored proc) or the specified timeout expires.
For example:
EXEC GetData #Timeout=2000 -- Wait for upto 5000 milliseconds for a result set to be
-- available
Ugly, but effective. As long as the query being executed is low cost, e.g. waiting for rows to appear in an empty table, this shouldn't be too much of a resource swine.
declare #Timeout as Int = 5000 -- Milliseconds.
-- Start and end times.
declare #Now as DateTime = GetDate()
declare #TimeoutExpiration as DateTime = DateAdd( ms, #Timeout, #Now )
-- Set the delay interval to the smaller of #Timeout / 10 or 1 second.
declare #DelayInterval as Time = DateAdd( ms, #Timeout / 10, 0 )
if #DelayInterval > '00:00:01.000'
set #DelayInterval = '00:00:01.000'
declare #WaitForDelay as VarChar(12) = Cast( #DelayInterval as VarChar(12) ) -- WaitFor insists on a truncated string for the delay.
select #Timeout as 'Timeout (ms)', #Now as 'Now', #TimeoutExpiration as 'TimeoutExpiration', #DelayInterval as 'DelayInterval'
declare #Result as Table ( Foo Int ) -- Modify to match the schema of your expected results.
-- Execute the query in a loop until either a result is returned or the timeout expires.
declare #RowCount as Int = 0
declare #Iterations as Int = 0
while #TimeoutExpiration >= GetDate() and #RowCount = 0
begin
-- Randomly decide to insert a row into the results. (Replace with your query.)
insert into #Result
select 42 as Foo
where Rand() > 0.8
-- Handle the query result.
set #RowCount = ##RowCount
if #RowCount = 0
waitfor delay #WaitForDelay
set #Iterations = #Iterations + 1
end
-- Return the result.
select #Iterations as 'Iterations' -- Just for demonstration purposes.
select *
from #Result
I have searched SO, for similar and I found one other posting similar to this and I followed what I thought was the follow up but I'm still seeing a problem.
I have also been sifting through the MySQL manuals, and what I have here looks like it is correct.
DELIMITER $$
CREATE DEFINER=`perimUser`#`localhost` PROCEDURE `assignLOBId`()
BEGIN
declare id, done INT default 0;
declare name VarChar(45);
declare lobCursor Cursor for Select idLineOfBusiness as id, name from LineOfBusiness;
declare continue handler for not found set done = 1;
OPEN lobCursor;
my_loop: LOOP
FETCH lobCursor INTO id, name;
IF done = 1 THEN
CLOSE lobCursor;
LEAVE my_loop;
END IF;
insert into test values (id, name);
UPDATE medium set idLOB = id where LOB = name;
UPDATE low set idLOB = id where LOB = name;
End LOOP my_loop;
END
I have run the Query that I"m using for the cursor and it does return 13 rows. Tables medium and low are full of data about 600 rows in each. the LOB match values in the LOB column of each. The values that were used to create the ones in lineofbusiness were generated from medium and low.
The goal here is to use this pattern a number of times as I work to normalized the data in medium and low. Otherwise I'd take the short cut and create a bunch of manual update statements.
I'm not too sure why your cursor isn't working as expected (you don't say whether your test table is populated with the results that you expect?), but it appears to me your procedure is simply implementing a multiple-table UPDATE (so can probably be entirely replaced with the following):
UPDATE LineOfBusiness
LEFT JOIN medium ON LineOfBusiness.name = medium.LOB
LEFT JOIN low ON LineOfBusiness.name = low.LOB
SET medium.idLOB = LineOfBusiness.idLineOfBusiness
, low.idLOB = LineOfBusiness.idLineOfBusiness