MySQL: get a random unique integer ID - mysql

I tried to write a SQL-function that generates an unused unique ID in a range between 1000000 and 4294967295. I need numeric values, so UUID() alike is not a solution. It doesn't sound that difficult, but for some reason, the code below does not work when called within an INSERT-statement on a table as value for the primary key (not auto_increment, of course). The statement is like INSERT INTO table (id, content) VALUES ((SELECT getRandomID(0,0)), 'blabla bla');
(Since default values are not allowed in such functions, I shortly submit 0 for each argument and set it in the function to the desired value.)
Called once and separated from INSERT or Python-code, everything is fine. Called several times, something weird happens and not only the whole process but also the server might hang within REPEAT. The process is then not even possible to kill/restart; I have to reboot the machine -.-
It also seems to only have some random values ready for me, since the same values appear again and again after some calls, allthough I actually thought that the internal rand() would be a sufficient start/seed for the outer rand().
Called from Python, the loop starts to hang after some rounds although the very first one in my tests always produces a useful, new ID and therefore should quit after the first round. Wyh? Well, the table is empty...so SELECT COUNT(*)... returns 0 which actually is the signal for leaving the loop...but it doesn't.
Any ideas?
I'm running MariaDB 10.something on SLES 12.2. Here is the exported source code:
DELIMITER $$
CREATE DEFINER=`root`#`localhost` FUNCTION `getRandomID`(`rangeStart` BIGINT UNSIGNED, `rangeEnd` BIGINT UNSIGNED) RETURNS bigint(20) unsigned
READS SQL DATA
BEGIN
DECLARE rnd BIGINT unsigned;
DECLARE i BIGINT unsigned;
IF rangeStart is null OR rangeStart < 1 THEN
SET rangeStart = 1000000;
END IF;
IF rangeEnd is null OR rangeEnd < 1 THEN
SET rangeEnd = 4294967295;
END IF;
SET i = 0;
r: REPEAT
SET rnd = FLOOR(rangeStart + RAND(RAND(FLOOR(1 + rand() * 1000000000))*10) * (rangeEnd - rangeStart));
SELECT COUNT(*) INTO i FROM `table` WHERE `id` = rnd;
UNTIL i = 0 END REPEAT r;
RETURN rnd;
END$$
DELIMITER ;

A slight improvement:
SELECT COUNT(*) INTO i FROM `table` WHERE `id` = rnd;
UNTIL i = 0 END REPEAT r;
-->
UNTIL NOT EXISTS( SELECT 1 FROM `table` WHERE id = rnd ) REPEAT r;
Don't pass any argument to RAND -- that is for establishing a repeatable sequence of random numbers.
mysql> SELECT RAND(123), RAND(123), RAND(), RAND()\G
*************************** 1. row ***************************
RAND(123): 0.9277428611440052
RAND(123): 0.9277428611440052
RAND(): 0.5645420109522921
RAND(): 0.12561983719991504
1 row in set (0.00 sec)
So simplify to
SET rnd = FLOOR(rangeStart + RAND() * (rangeEnd - rangeStart));
If you want to include rangeEnd in the possible outputs, add 1:
SET rnd = FLOOR(rangeStart + RAND() * (rangeEnd - rangeStart + 1));

Related

DB times out or takes a long time to execute

I'm having an issue with one very specific procedure I am working on, either it takes roughly about 45 seconds to run and or it takes over minutes to complete. I know what the issue is, but at this time I am not sure how to get around it to be honest. In the SP, you will see a section (while block) commented out that basically create's a token to be inserted into the table, currently I am inserting just TEST to get around the blocking issue.
Here's the current working version that runs fine by just inserting a dummy value.
BEGIN
DECLARE userexist INT(1);
DECLARE tokenID VARCHAR(12) default '';
DECLARE rnd_str text;
DECLARE ready int default 0;
DECLARE userID INT(20) default 0;
DECLARE lastID int default 0;
SELECT
su.User_ID,
1 AS user_exist
INTO userID, userexist
FROM System_User su
WHERE AES_DECRYPT(su.User_Email, 'AAAA') = email
AND AES_DECRYPT(su.User_Password, 'AAAA') = userpassword
AND su.User_Is_Active = 1;
IF userexist = 1 THEN
UPDATE System_User_Login
SET System_User_Login.System_Logout_Time = NOW(),
System_User_Login.System_Logout_Type = 1
WHERE System_User_Login.System_Login_User_ID = userID;
SET tokenID = 'TEST'; -- dummy value for now
INSERT INTO System_User_Login(System_Login_Time, System_Login_Token, System_Login_User_ID, System_Token_Valid_Period)
VALUES(NOW(), tokenID, userID, (NOW() + INTERVAL 30 DAY));
SET lastID = LAST_INSERT_ID();
-- WHILE ready = 0 DO
-- SET rnd_str = lpad(conv(floor(rand()*pow(36,16)), 12, 36), 12, 0);
-- IF NOT EXISTS (SELECT 1 FROM System_User_Login sul WHERE sul.System_Login_Token = rnd_str) THEN
-- update System_User_Login
-- SET System_Login_Token = rnd_str
-- WHERE System_Login_ID = lastID;
-- SET ready = 1;
-- END IF;
-- END WHILE;
SELECT
sul.System_Login_Token AS Token
FROM System_User_Login sul
WHERE sul.System_Login_ID = lastID;
ELSE
SELECT '0' AS Token;
END IF;
END
The while loop inside this procedure is to create a new token to be assigned; I've tried before and after the insert to no avail; even triggers. I must not be seeing something and or need to rethink my approach.
If your user list is high then the probability of this collision will increase. This will cause multiple iterations. You can avoid that by following
1: You can use Universal unique identifiers if working on MySQL 8 but make sure length constraints matches as per your requirement.
UUID
UUID Short
2: Try the following
Unique Identifier = CONCAT(<some_random_string>, unix_timestamp, <primary_key_id>)
So if you are setting token for user = 4 then
SELECT LPAD(CONCAT(CONV(FLOOR(RAND()* POW(36,16)), 12, 36), UNIX_TIMESTAMP(), '4'), 25, 0);
If you are not generating more than one token in one sec then UNIX_TIMESTAMP will make it unique.
Further, if we are generating more than one token per sec then we reduce collision by generating a random number using CONV(FLOOR(RAND()* POW(36,16)), 12, 36)
If by chance, it generates the same random string for multiple, we have a unique id for each user, which will definitely make it unique.
Assumption: You are not creating multiple tokens for the same user in one second if so there might be chances of getting a collision.
Note: If you use the second approach, you can use two SQL statements for your complete procedure. Here is a way

MySQL Variable Assignment via Procedure Not Working Correctly

In the code below, I'm trying go through the results of endDateTable row by row, comparing the current row's endDate to the previous row's endDate. If there has been any change since the previous, we increment #revisionNum. However, upon populating the new table, all of the #revisionNum entries are 0. What am I doing wrong?
NOTE: I'm using prepared statements in this manner since doing a straightforward SELECT into a variable gives a syntax error due to the LIMIT clause not allowing a variable in our version of MySQL.
BEGIN
DECLARE _currentEndDate DATETIME DEFAULT now();
DECLARE _priorEndDate DATETIME DEFAULT now();
SET #ResultsCount = (SELECT COUNT(*) FROM mainTable);
SET #j = 0;
WHILE #j < #ResultsCount DO
SET #revisionNum = 0;
/*CURRENT END DATE*/
SET #appResultQueryCurrent = CONCAT('
SELECT
end_date
INTO _currentEndDate
FROM endDateTable
LIMIT ', #j, ', 1'
);
PREPARE currentQueryStmt FROM #appResultQueryCurrent;
EXECUTE currentQueryStmt;
/*PREVIOUS END DATE*/
SET #appResultQueryPrior = CONCAT('
SELECT
end_date
INTO _priorAppEndDate
FROM endDateTable
LIMIT ', IF(#j = 0, 0, #j - 1), ', 1'
);
PREPARE priorQueryStmt FROM #appResultQueryPrior;
EXECUTE priorQueryStmt;
SET #revisionNum = IF(
#j = 0 OR (_currentEndDate = _priorEndDate),
#revisionNum,
IF(
_currentEndDate != _priorEndDate,
#revisionNum + 1,
#revisionNum
)
);
INSERT INTO finalTable (RevisionNum)
SELECT
#revisionNum AS RevisionNum
FROM endDateTable;
SET #j = #j +1;
END WHILE;
END $$
You don't need a loop, you can use INSERT INTO ... SELECT ..., incrementing the variable in the select query.
You also need an ORDER BY criteria to specify how to order the rows when comparing one row to the previous row.
INSERT INTO finalTable (RevisionNum, otherColumn)
SELECT revision, otherColumn
FROM (
SELECT IF(end_date = #prev_end_date, #revision, #revision := #revision + 1) AS revision,
#prev_end_date := end_date,
otherColumn
FROM endDateTable
CROSS JOIN (SELECT #prev_end_date := NULL, #revision := -1) AS vars
ORDER BY id) AS x
DEMO
The offset value in the LIMIT clause is tenuous without an ORDER BY.
Without an ORDER BY clause, MySQL is free to return results in any sequence.
There is no guarantee that LIMIT 41,1 will return the row before LIMIT 42,1, or that it won't return the exact same row as LIMIT 13,1 did.
(A table in a relational database represents an unordered set of tuples, there is no guaranteed "order" or rows in a table.)
But just adding ORDER BY to the queries isn't enough to fix the Rube-Goldberg-esque rigmarole.
In the code shown, it looks like each time through the loop, we're inserting a copy of endDateTable into finalTable. If that's 1,000 rows in endDateTable, we're going to get 1,000,000 rows (1,000 x 1,000) inserted into finalTable. Not at all clear why we need so many copies.
Given the code shown, it's not clear what the objective is. Looks like we are conditionally incrementing revisionNum, the end result of which is the highest revision num. Just guessing here.
If there is some kind of requirement to do this in a LOOP construct, within a procedure, I'd think we'd do a cursor loop. And we can use procedure variables vs user-defined variables.
Something along these lines:
BEGIN
DECLARE ld_current_end_date DATETIME;
DECLARE ld_prior_end_date DATETIME;
DECLARE li_done INT;
DECLARE li_revision_num INT;
DECLARE lcsr_end_date CURSOR FOR SELECT t.end_date FROM `endDateTable` t ORDER BY NULL;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET li_done = TRUE;
SET li_done = FALSE;
SET li_revision_num = 0;
OPEN lcsr_end_date;
FETCH lcsr_end_date INTO ld_current_end_date;
SET ld_prior_end_date = ld_current_end_date;
WHILE NOT li_done DO
SET li_revision_num = li_revision_num + IF( ld_current_end_date <=> ld_prior_end_date ,0,1);
SET ld_prior_end_date := ld_current_end_date;
FETCH lcsr_end_date INTO ld_current_end_date;
END WHILE;
CLOSE lcsr_end_date;
INSERT INTO `finalTable` (revisionnum) VALUES (li_revision_num);
END $$
Note the "order by" clause on the SELECT, its not clear what the rows should be ordered on, so we're using a literal as a placeholder.
As the end result, we insert a single row into finalTable.
Again, it's not clear what the code in the question is supposed to achieve, but doing a cursor loop across ordered rows would be much more efficient than a bazillion dynamic SQL executions fetching individual rows.

SQL Server T-SQL breaking a string into a temp table for a join

We have a SQL Server Scalar Function and part of the process is to take one of the input values and do the following
'inputvalue'
Create a table variable and populate with the following rows
inputvalue
inputvalu
inputval
inputva
inputv
input
inpu
inp
Then this table is joined to a query, ordered by len of the inputvalue desc and returns the top 1. The actual code is here
DECLARE #Result NVARCHAR(20);
DECLARE #tempDialCodes TABLE (tempDialCode NVARCHAR(20));
DECLARE #counter INT = LEN(#PhoneNumber);
WHILE #counter > 2
BEGIN
INSERT INTO #tempDialCodes(tempDialCode) VALUES(#PhoneNumber);
SET #PhoneNumber = SUBSTRING(#PhoneNumber, 1, #counter - 1);
SET #counter = #counter - 1;
END
SET #Result = (SELECT TOP 1 [DialCodeID]
FROM DialCodes dc JOIN #tempDialCodes s
ON dc.DialCode = s.tempDialCode
ORDER BY LEN(DialCode) DESC);
RETURN #Result
It works fine but I am asking if there is a way to replace the while loop and somehow joining to the inputvalue to get the same result. When I say it works fine, it's too dam slow but it does work.
I'm stumped on how to break up this string without using a loop and to a table variable but my warning light tells me this is not efficient for running against a table with a million rows.
Are you familiar with tally tables? The speed difference can be incredible. I try to replace every loop with a tally table if possible. The only time I haven't been able to so far is when calling a proc from within a cursor. If using this solution I would recommend a permanent dbo.Tally table with a sufficiently large size rather than recreating every time in the function. You will find other uses for it!
declare #PhoneNumber nvarchar(20) = 'inputvalue';
declare #tempDialCodes table (tempDialCode nvarchar(20));
--create and populate tally table if you don't already a permanent one
--arbitrary 1000 rows for demo...you should figure out if that is enough
--this a 1-based tally table - you will need to tweak if you make it 0-based
declare #Tally table (N int primary key);
insert #Tally
select top (1000) row_number() over (order by o1.object_id) from sys.columns o1, sys.columns o2 order by 1;
--select * from #Tally order by N;
insert #tempDialCodes
select substring(#PhoneNumber, 1, t.N)
from #Tally t
where t.N between 3 and len(#PhoneNumber)
order by t.N desc;
select *
from #tempDialCodes
order by len(tempDialCode) desc;

how to optimise MySql function to work for concurrent users

I am new to MySQL. Please can you advice on how can i modify below function to ensure it does not throw locking errors when called by multiple users at the same time.
CREATE FUNCTION `get_val`(`P_TABLE` VARCHAR(50)) RETURNS int(11)
DETERMINISTIC
BEGIN
DECLARE pk_value INT DEFAULT 0;
DECLARE pk_found INT DEFAULT 0;
SELECT 1 INTO pk_found FROM pk_keys WHERE TABLE_NAME = P_TABLE;
IF pk_found = 1
THEN
UPDATE pk_keys SET TABLE_VALUE = (TABLE_VALUE + 1 ) WHERE TABLE_NAME = P_TABLE;
ELSE
INSERT INTO pk_keys VALUES ( P_TABLE, 1 );
END IF;
SELECT TABLE_VALUE INTO pk_value FROM pk_keys WHERE TABLE_NAME = P_TABLE;
RETURN pk_value;
END
thanks
CREATE FUNCTION `get_val`(`P_TABLE` VARCHAR(50)) RETURNS int(11)
DETERMINISTIC
MODIFIES SQL DATA
BEGIN
DECLARE pk_value INT DEFAULT 0;
IF EXISTS (SELECT 1 FROM pk_keys WHERE TABLE_NAME = P_TABLE)
THEN
SELECT TABLE_VALUE + 1 INTO pk_value FROM pk_keys WHERE TABLE_NAME = P_TABLE FOR UPDATE;
UPDATE pk_keys SET TABLE_VALUE = pk_value WHERE TABLE_NAME = P_TABLE;
ELSE
SET pk_value = 1;
INSERT INTO pk_keys VALUES ( P_TABLE, pk_value );
END IF;
RETURN pk_value;
END
Have a look at SELECT ... FOR UPDATE and SELECT ... LOCK IN SHARE MODE Locking Reads
Let us look at another example: We have an integer counter field in a
table child_codes that we use to assign a unique identifier to each
child added to table child. It is not a good idea to use either
consistent read or a shared mode read to read the present value of the
counter because two users of the database may then see the same value
for the counter, and a duplicate-key error occurs if two users attempt
to add children with the same identifier to the table.
Here, LOCK IN SHARE MODE is not a good solution because if two users
read the counter at the same time, at least one of them ends up in
deadlock when it attempts to update the counter.
To implement reading and incrementing the counter, first perform a
locking read of the counter using FOR UPDATE, and then increment the
counter. For example:
SELECT counter_field FROM child_codes FOR UPDATE;
UPDATE child_codes SET counter_field = counter_field + 1;
A SELECT ... FOR UPDATE reads the latest available data, setting
exclusive locks on each row it reads. Thus, it sets the same locks a
searched SQL UPDATE would set on the rows.
Also I replaced your if condition. EXISTS stops as soon as a row is found.

Sequel Pro multiple update query with case and random variable

Alright, so I have a table that has a count field that is between 0 and 100. Up to six of these fields are tied to a single id. I need to run an update that will decrease each of these rows by a different random number between 1 and 3.
I know I can get my random value with:
CAST(RAND() * 3 AS UNSIGNED)
And I know I can get my update to work with:
UPDATE Info SET Info.count = CASE WHEN Info.count < 2 THEN 0 ELSE Info.count - 2 END WHERE Info.id = $iid AND Info.type = 'Active';
(This just makes sure I will never dip below 0)
But I cannot combine them for the obvious reason that my random number will be different when evaluated then when it's set...
UPDATE Info SET Info.count = CASE WHEN Info.count < CAST(RAND() * 3 AS UNSIGNED) THEN 0 ELSE Info.count - CAST(RAND() * 3 AS UNSIGNED) END WHERE Info.id = $iid AND Info.type = 'Active';
Now I don't want to save just 1 variable, because I may need up to 6 different numbers...is there a way to do what I want to do in a single query? I know how I can do it in multiple, but I really shouldn't be running up to 6 queries every time I need to update one of these blocks...
The table structure I'm working off of is:
CREATE TABLE `Info` (
`id` int(40) DEFAULT NULL,
`count` int(11) DEFAULT NULL,
`type` varchar(40) DEFAULT NULL,
`AUTOINC` int(11) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`AUTOINC`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8;
The AUTOINC field at the moment is just so I can easily verify what's going on during testing.
Thanks!
You're probably best off writing a procedure to do this. So you could write something to the extent of (pseudo code)
create procedure update()
begin
declare id, count, sub int;
declare c cursor for select id, count floor(1+rand()*3) from info
where type='Active';
open c;
loop
fetch c into id, count, sub;
update info set case count - sub < 0 then 0 else count - sub end
where id = id;
end loop;
close c;
end
//
Or you can change the procedure to accept an ID like I had it before and simply use one select and one update statement.