I have a stored procedure with a select and an update. I would like to prevent multiple users, from executing it, at the same time, so I don't update, based on an incorrect select.
How do I lock it?
I've read various solutions (Transaction isolation, xlock), but I haven't been able to figure what I really want, and how to do it.
The easiest way is to forget about data locks but look at sp_getapplock to control access through the code
BEGIN TRY
EXEC sp_getapplock ...
SELECT ...
UPDATE ...
EXEC sp_releaseapplock
END TRY
...
Saying that, with thing like the OUTPUT clause and judicious use of ROWLOCK, UPDLOCK there is a good chance the UPDATE and SELECT can be one statement
Using the XLOCK table hint in the SELECT query:
CREATE TABLE [X]([x] INT NOT NULL)
GO
INSERT [X]([x]) SELECT 0
GO
CREATE PROCEDURE [ATOMIC]
AS
BEGIN
BEGIN TRAN
DECLARE #x INT = (
SELECT [x]
FROM [X] (XLOCK)
) + 1
UPDATE [X] SET [x] = #x
COMMIT TRAN
END
GO
You can then test this by running
EXEC [ATOMIC]
GO 10000
simultaneously from different sessions. You can test using
SELECT [x] FROM [X]
The value should be exactly 10 000 times the number of sessions you ran. If the number is less than expected you don't have atomic read + write, or some SPIDs may have been killed due to dead locking.
Related
I have a select statement that takes a long time to run (around 5 minutes). Because of this I only run the query every hour and save the results to a metadata table. Here is the query:
UPDATE `metadata` SET `value` = (select count(`id`) from `logs`) WHERE `key` = 'logs'
But this is the issue I have been having (And correct me if I am wrong). A select statement does not lock the database, but an update statement does. Now since I am running this long ruining select query inside of the update query, it ends up locking the DB for about 5 minutes.
Is there a better way to do this to run the select statement and save it to a variable and then once that is done then running the update query? This way it wont lock the DB.
Also note I don't care about dirty data.
The database has over 300 million rows and has data being added to it constantly.
Just to avoid the possibility that the server disconnects between the statement getting the count and the statement storing it, leaving your variable unset, beginning in mariadb 1.1 you can run multiple statements in a single request by putting them in a block:
begin not atomic
declare `logs_count` int;
select count(*) into `logs_count` from `logs`;
update `metadata` set `value`=`logs_count` where `key`='logs';
end
fiddle
I have found that setting this before the query runs seems to work and runs a whole lot faster. This should keep the DB from locking when executing the query. We then enable locking after it has completed.
(Please correct me if I have done something incorrect here)
BEGIN
SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
UPDATE `metadata` SET `value` = (select count(`id`) from `logs`) WHERE `key` = 'logs';
SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
END
I am writing a web application and came across the problem to keep a value in column that is identical in two rows only and both goes in single batch of execution. One way I came up with solution to read the MAX value in the column and increment by 1. Thus, I end-up writing the procedure to lock the table so that other user should not get the dirty read of MAX value.
Create table D(Id int , Name varchar(100))
Begin Tran
DECLARE #i int = (SELECT MAX(ID) FROM D with (tablockx, holdlock))
Print #i ;
Insert into D values ((#i + 1), 'ANAS')
SELECT * FROM D
--COMMIT
Rollback
This code lock the table until query commits or rollback. I have two question from this 1) Is this code guarantee to have exclusive lock on table? 2) In my quick read tablockx can help perform lock for read also whereas holdlock help to prevent the changes in row I am working in locked session, but is this rightly used because I think holdlock may actually not required
Not sure you are asking the right question. You need the current value not to be changed and no insert. With SERIALIZABLE you may not need (updlock).
I am not positive about this answer. You should do your own testing.
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
Begin Tran
DECLARE #i int = (SELECT MAX(ID) FROM D with (updlock))
Print #i ;
Insert into D values ((#i + 1), 'ANAS')
SELECT * FROM D
--COMMIT
Rollback
identity or sequence number are better approaches
I am building a "poor man's queuing system" using MySQL. It's a single table containing jobs that need to be executed (the table name is queue). I have several processes on multiple machines whose job it is to call the fetch_next2 sproc to get an item off of the queue.
The whole point of this procedure is to make sure that we never let 2 clients get the same job. I thought that by using the SELECT .. LIMIT 1 FOR UPDATE would allow me to lock a single row so that I could be sure it was only updated by 1 caller (updated such that it no longer fit the criteria of the SELECT being used to filter jobs that are "READY" to be processed).
Can anyone tell me what I'm doing wrong? I just had some instances where the same job was given to 2 different processes so I know it doesn't work properly. :)
CREATE DEFINER=`masteruser`#`%` PROCEDURE `fetch_next2`()
BEGIN
SET #id = (SELECT q.Id FROM queue q WHERE q.State = 'READY' LIMIT 1 FOR UPDATE);
UPDATE queue
SET State = 'PROCESSING', Attempts = Attempts + 1
WHERE Id = #id;
SELECT Id, Payload
FROM queue
WHERE Id = #id;
END
Code for the answer:
CREATE DEFINER=`masteruser`#`%` PROCEDURE `fetch_next2`()
BEGIN
SET #id := 0;
UPDATE queue SET State='PROCESSING', Id=(SELECT #id := Id) WHERE State='READY' LIMIT 1;
#You can do an if #id!=0 here
SELECT Id, Payload
FROM queue
WHERE Id = #id;
END
The problem with what you are doing is that there is no atomic grouping for the operations. You are using the SELECT ... FOR UPDATE syntax. The Docs say that it blocks "from reading the data in certain transaction isolation levels". But not all levels (I think). Between your first SELECT and UPDATE, another SELECT can occur from another thread. Are you using MyISAM or InnoDB? MyISAM might not support it.
The easiest way to make sure this works properly is to lock the table.
[Edit] The method I describe right here is more time consuming than using the Id=(SELECT #id := Id) method in the above code.
Another method would be to do the following:
Have a column that is normally set to 0.
Do an "UPDATE ... SET ColName=UNIQ_ID WHERE ColName=0 LIMIT 1. That will make sure only 1 process can update that row, and then get it via a SELECT afterwards. (UNIQ_ID is not a MySQL feature, just a variable)
If you need a unique ID, you can use a table with auto_increment just for that.
You can also kind of do this with transactions. If you start a transaction on a table, run UPDATE foobar SET LockVar=19 WHERE LockVar=0 LIMIT 1; from one thread, and do the exact same thing on another thread, the second thread will wait for the first thread to commit before it gets its row. That may end up being a complete table blocking operation though.
I have a stored procedure that basically looks like this:
do_insert (IN in_x varchar(64), IN in_y varchar(64))
BEGIN
declare v_x int(20) unsigned default -1;
declare v_y int(20) unsigned default -1;
select x into v_x from xs where x_col = in_x;
if v_x = 0
then
insert into xs (x_col) values(in_x);
select x into v_x from xs where x_col = in_x;
end if;
select y into v_y from ys where y_col = in_y;
if v_y = 0
then
insert into ys (y_col) values(in_y);
select y into v_y from ys where y_col = in_y;
end if;
insert ignore into unique_table (xId, yId) values(v_x, v_y);
END
Basically I look to see if I already have the varchars defined in their respective tables, and if so I select the id. If not, then I create them and get their IDs. Then I insert them into the unique_table ignoring if they're already there. (Yes I could probably put more logic in there to NOT have to do the final insert, but that shouldn't be an issue and KISS.)
The problem I have is that when I run this in a batch JDBC statement using Google Cloud SQL I get duplicate entries inserted into my xs table. The next time this stored proc is run I get the following exception:
java.sql.SQLException: Result consisted of more than one row Query: call do_insert(:x, :y)
So what I think is happening is that two calls with the same in_x values are occurring in the same batch statement. These two calls are being run in parallel, both selects come back with 0 as it's a new entry, then they both do an insert. The next run then fails.
Questions:
How do I prevent this?
Should I wrap my select (and possible insert) calls in a LOCK TABLE for that table to prevent this?
I've never noticed this on a local MySQL, is this Google Cloud SQL specific? Or just a fluke that I haven't seen it on my local MySQL?
I haven't tested this yet, but I'm guessing the batch statement is introducing some level of parallelism. So the 'select x into v_x' statement can race with the 'insert into vx' statement allowing the latter to be executed more than once.
Try changing the 'insert into xs' statement to an 'insert ignore' and add a unique index on the column.
I am trying to combine these two queries in twisted python:
SELECT * FROM table WHERE group_id = 1013 and time > 100;
and:
UPDATE table SET time = 0 WHERE group_id = 1013 and time > 100
into a single query. Is it possible to do so?
I tried putting the SELECT in a sub query, but I don't think the whole query returns me what I want.
Is there a way to do this? (even better, without a sub query)
Or do I just have to stick with two queries?
Thank You,
Quan
Apparently mysql does have something that might be of use, especially if you are only updating one row.
This example is from: http://lists.mysql.com/mysql/219882
UPDATE mytable SET
mycolumn = #mycolumn := mycolumn + 1
WHERE mykey = 'dante';
SELECT #mycolumn;
I've never tried this though, but do let me know how you get on.
This is really late to the party, but I had this same problem, and the solution I found most helpful was the following:
SET #uids := null;
UPDATE footable
SET foo = 'bar'
WHERE fooid > 5
AND ( SELECT #uids := CONCAT_WS(',', fooid, #uids) );
SELECT #uids;
from https://gist.github.com/PieterScheffers/189cad9510d304118c33135965e9cddb
You can't combine these queries directly. But you can write a stored procedure that executes both queries. example:
delimiter |
create procedure upd_select(IN group INT, IN time INT)
begin
UPDATE table SET time = 0 WHERE group_id = #group and time > #time;
SELECT * FROM table WHERE group_id = #group and time > #time;
end;
|
delimiter ;
So what you're trying to do is reset time to zero whenever you access a row -- sort of like a trigger, but MySQL cannot do triggers after SELECT.
Probably the best way to do it with one server request from the app is to write a stored procedure that updates and then returns the row. If it's very important to have the two occur together, wrap the two statements in a transaction.
There is a faster version of the return of updated rows, and more correct when dealing with highly loaded system asks for the execution of the query at the same time on the same database server
update table_name WITH (UPDLOCK, READPAST)
SET state = 1
OUTPUT inserted.
UPDATE tab SET column=value RETURNING column1,column2...