I would like to get a suggestion on improving my setup that is causing the sql server to return the deadlock message. I have multiple threading application that actually uses the TaskParallel library and each task will use a stored procedure to select an id from a table to use in its processing. I immediately delete that id from the table in the same statement and I think that is what is causing the deadlocks. The table consists of one column of uniques ids with no indexes. I thought of doing a batch delete periodically but that means keeping a tally of used ids across multiple servers.
here is my sql stored procedure:
CREATE PROCEDURE [dbo].[get_Ids]
#id nvarchar(20) OUTPUT
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
Select top 1 #id = siteid from siteids
delete siteids where siteid = #id
END
Is there any better way to do this? My processes work very fast and I used to request this id from a webrequest service but this took 3 seconds.
Some things to try:
Maybe try hinting to the DB that you will delete the record you just selected, this way it will grab the lock early. For this to work you'll need to wrap the whole procedure in a transaction, then hint the select. Should look something like:
BEGIN TRANSACTION
SELECT TOP 1 #id = siteid from siteids WITH (UPDLOCK, HOLDLOCK)
DELETE siteids WHERE siteid = #id
COMMIT TRANSACTION
Also make sure the siteid column is indexed(or tag it as primary key, since you say it is unique), otherwise it would have to scan the table to get the record to delete, which could make deadlocking worse since it spends a bunch more time deleting.
For deadlocks in general, run the SQL profiler and see what the deadlock graph looks like - might be something else is going on.
Related
I read about optimistic locking scheme, where clients can read the values, perform there computation and when write needs to happen, Updates are validated before being written to database.
Lets say If we employ version mechanism for Optimistic Locks then (In case two clients) both will be having update statements as :
update tableName Set field = val, version = oldVersion +1 where
version = OldVersion and Id = x;
Now lets consider the following scenario with Two Clients :
Both Clients read the values of field and version.
Both Clients compute something at there end. Generate new value of field.
Now Both Clients send query Request to Database Server.
As soon as it reaches database :
One Client Update Query starts executing.
But in the mean time interleaving happens and other Client Update
starts executing.
Will these query interleaving causes data races at table
I mean to say, we can't say that Optimistic Lock executes on its own, for example I understand the case where row level locking happens or other locking like table level locking happens, then its fine. But then its like Optimistic Locks doesn't work on its own, it needs pessimistic lock also(row level/ table level, which totally depends on underlying Storage Engine Implementation).
What happens when there is no Row / table level locks already there, but want to implement Optimistic Locking strategy. With query interleaving will it causes data races at table.(I mean to say only field is updated and version is not and then interleaving happens. Is this totally depends on what Isolation levels are set for query)?
I'm little bit confused with this scenario.
Also what is the right use case where optimistic Locking can be really helpful and increase the overall performance of application as compared to Pessimistic Locking.
The scenario in pseudo code for the worst case scenario: Two clients update the same record:
Scenario 1 (your scenario: optimistic locking):
Final constraints are checked at the server side. Optimistic locking is used only for presentation purposes.
Client one orders a product of which there is only 1 in stock.
Client two orders the same product of which there is only 1 in stock.
Both clients get this presented on the screen.
Products table:
CREATE TABLE products (
product_id VARCHAR(200),
stock INT,
price DOUBLE(5,2)
) ENGINE=InnoDB;
Presentation code:
-- Presentation:
SELECT * FROM products WHERE product_id="product_a";
-- Presented to client
Order code:
-- Verification of record (executed in the same block of code within
-- an as short time interval as possible):
SELECT stock FROM products WHERE product_id="product_a";
IF(stock>0) THEN
-- Client clicks "order" (one click method=also payment);
START TRANSACTION;
-- Gets a record lock
SELECT * FROM products WHERE product_id="product_a" FOR UPDATE;
UPDATE products SET stock=stock-1 WHERE product_id="product_a";
INSERT INTO orders (customer_id,product_id,price)
VALUES (customer_1, "product_a",price);
COMMIT;
END IF;
The result of this scenario is that both orders can succeed: They both get the stock>0 from the first select, and then execute the order placement. This is an unwanted situation (in almost any scenario). So this would then have to be addressed in code by cancelling the order, taking a few more transactions.
Scenario 2: Alternative to optimistic locking:
Final constraints are checked at the database side. Optimistic locking is used only for presentation purposes. Less database queries then in the previous optimistic locking scenario, less chance of redos.
Client one orders a product of which there is only 1 in stock.
Client two orders the same product of which there is only 1 in stock.
Both clients get this presented on the screen.
Products table:
CREATE TABLE products (
product_id VARCHAR(200),
stock INT,
price DOUBLE(5,2),
CHECK (stock>=-1) -- The constraint preventing ordering
) ENGINE=InnoDB;
Presentation code:
-- Presentation:
SELECT * FROM products WHERE product_id="product_a";
-- Presented to client
Order code:
-- Client clicks "order" (one click method=also payment);
START TRANSACTION;
-- Gets a record lock
SELECT * FROM products WHERE product_id="product_a" FOR UPDATE;
UPDATE products SET stock=stock-1 WHERE product_id="product_a";
INSERT INTO orders (customer_id,product_id,price)
VALUES (customer_1, "product_a",price);
COMMIT;
So now two customers get presented this product, and click order on the same time. The system executes both orders simultaneous. The result will be: One order will be placed, the other gets an exception since the constraint will fail to verify, and the transaction will be aborted. This abort (exception) will have to be handled in code but does not take any further queries or transactions.
I have several servers hitting a common MySQL box and I need exclusive access to a table of scheduled jobs.
After some reading here and elsewhere I was led to believe SELECT...FOR UPDATE was what I wanted, but now we are (very rarely) seeing multiple servers pick up the same record.
Here's the PROC (minus the BEGIN/END stuff because it was playing hell with my formatting):
CREATE DEFINER=`root`#`%` PROCEDURE `PopScheduledJob`(OUT `JobId` varchar(36) )
SELECT ScheduledJobId INTO JobId
FROM scheduledjob
WHERE
Status = 0
AND NextRun < UTC_TIMESTAMP()
ORDER BY StartDate
LIMIT 1
FOR UPDATE;
UPDATE scheduledjob
SET Status = 2
WHERE ScheduledJobId = JobId;
So the intent here is that it should only pick up a job with Status=0, and it sets it to 1 immediately.
My hope was that this would prevent any other thread/process from accessing the same record, but now it seems that's not the case.
EDIT: forgot to mention, we have an InnoDB backing store
(Please answer as generally as possible. But I am working in MS SQL Server and MySql, so if there is no general answer, go ahead and answer for one or both of those.)
Consider a reservation system implemented in a SQL database. I want to make sure that among many multiple users, only one user gets a reservation and no other user "thinks" they got it. It's a classic concurrency problem in DB work, but I'm not sure what the best answer is.
SPECIFICS:
Suppose each user has a UserID. We could imagine a few users are currently trying to make a reservation with UserID values of 1004, 1005, 1009, and 1011.
Suppose the resource and reservations are stored in a table, SEATS .
We could imagine at one point the SEATS table contains:
----- SEATS -----------------------------
SeatID UserID ResvTime
1 1017 2014.07.15 04:17:18.000
2 NULL NULL
3 NULL NULL
4 1012 2014.07.15 04:19:35.000
5 1003 2014.07.15 04:20:46.000
-----------------------------------------
Now suppose that "at the same time", users 1004 and 1005 try to grab SeatID 3.
I'm wanting to know what SQL will properly make sure that only one of them gets the seat and the other gets a refusal. The simplest version of the code I can think of, in T-SQL, would be:
PROC GRABSEAT #seatid INT, #userid INT, #obtained BIT OUTPUT
BEGIN
DECLARE #avail INT
SET #avail = (SELECT UserID FROM SEATS WHERE (SeatID = #seatid))
IF (#avail IS NULL)
BEGIN
UPDATE SEATS SET UserID = #userid, ResvTime = GETDATE() WHERE (SeatID = #seatid)
SET #obtained = 1
END
ELSE
SET #obtained = 0
END
But the question is how to prevent this from allowing multiple concurrent users, all executing this PROC, getting a TRUE return on the same seat (say SeatID = 3).
For example, if both users 1004 and 1005 execute this PROC nearly simultaneously, they could both do the SELECT and get #avail = NULL before either of them tries to set the UserID column. Then both of them would run the UPDATE statement. Assuming nothing even worse could result, then one of them would overwrite the other's set, both would think they got the seat, but actually only the one who ran the UPDATE statement last would have their data stored in the SEATS table. The other one would have their data overwritten. This is referred to as a "lost input" problem. But what is the way in a SQL database to prevent it? I have been assuming that each single SQL statement is executed as a TRANSACTION. A TRANSACTION has the four so-called "ACID" properties. These properties are what I need. So, I think the answer, in a SQL database, is:
BEGIN TRANSACTION
EXCEUTE GRABSEAT #seatid= <id1>, #userid = <id2>, #obtained
COMMIT
By doing so, the main property I need (isolation), will guarantee that the interleaved execution I'm worried about will not occur.
But I've seen articles that say it's not that simple at all. I think the big problem various articles point to is that not every TRANSACTION really runs in total atomicity and isolation. So, perhaps the above wrapping in a TRANSACTION will not achieve the desired result. If not, then what is needed?
A transaction is atomic by definition. But when a transaction's changes become visible to other users / connections / transactions depends on the isolation level. The default isolation in SQL Server is READ COMMITTED - see this question's answer for more info and links on how to change it.
For this type of scenario, you probably want SERIALIZABLE. The good news is that you can change the isolation level for a transaction with a SET TRANSACTION ISOLATION LEVEL statement in your stored proc. The bad news is that you have to be 100% sure that this is the only place in your code that ever updates the SEAT table.
Fundamentally, the issue you have is that there is a race condition. Just because you are in a transaction does not mean that two transactions can't both call the stored proc at the same time, then run the SELECT. Now both tx think it's ok to to do the UPDATE. Setting the isolation level to SERIALIZABLE locks the table for the tx that hits the SELECT first.
Instead of the SELECT statement, why don't you just do the update, with an extra filter on NULL so it can't replace if the value is null, and then return whether the query had any effect or not. That way, the transaction is atomic, since it's just one query.
PROC GRABSEAT #seatid INT, #userid INT, #obtained BIT OUTPUT
BEGIN
UPDATE SEATS SET UserID = #userid, ResvTime = GETDATE()
WHERE (SeatID = #seatid) AND UserID IS NULL
SET #obtained = ##ROWCOUNT
END
Due to rowlocking, two updates can't happen simultaneously, so one will work (return ##ROWCOUNT = 1, and the other will fail ##ROWCOUNT = 0.
This is a follow up on my previous question (you can skip it as I explain in this post the issue):
MySQL InnoDB SELECT...LIMIT 1 FOR UPDATE Vs UPDATE ... LIMIT 1
Environment:
JSF 2.1 on Glassfish
JPA 2.0 EclipseLink and JTA
MySQL 5.5 InnoDB engine
I have a table:
CREATE TABLE v_ext (
v_id INT NOT NULL AUTO_INCREMENT,
product_id INT NOT NULL,
code VARCHAR(20),
username VARCHAR(30),
PRIMARY KEY (v_id)
) ENGINE=InnoDB DEFAULT CHARSET=UTF8;
It is populated with 20,000 records like this one (product_id is 54 for all records, code is randomly generated and unique, username is set to NULL):
v_id product_id code username
-----------------------------------------------------
1 54 '20 alphanumerical' NULL
...
20,000 54 '20 alphanumerical' NULL
When a user purchase product 54, he gets a code from that table. If the user purchases multiple times, he gets a code each times (no unique constraint on username). Because I am preparing for a high activity I want to make sure that:
No concurrency/deadlock can occur
Performance is not impacted by the locking mechanism which will be needed
From the SO question (see link above) I found that doing such a query is faster:
START TRANSACTION;
SELECT v_id FROM v_ext WHERE username IS NULL LIMIT 1 FOR UPDATE;
// Use result for next query
UPDATE v_ext SET username=xxx WHERE v_id=...;
COMMIT;
However I found a deadlock issue ONLY when using an index on username column. I thought of adding an index would help in speeding up a little bit but it creates a deadlock after about 19,970 records (actually quite consistently at this number of rows). Is there a reason for this? I don't understand. Thank you.
From a purely theoretical point of view, it looks like you are not locking the right rows (different condition in the first statement than in the update statement; besides you only lock one row because of LIMIT 1, whereas you possibly update more rows later on).
Try this:
START TRANSACTION;
SELECT v_id FROM v_ext WHERE username IS NULL AND v_id=yyy FOR UPDATE;
UPDATE v_ext SET username=xxx WHERE v_id=yyy;
COMMIT;
[edit]
As for the reason for your deadlock, this is the probable answer (from the manual):
If you have no indexes suitable for your statement and MySQL must scan
the entire table to process the statement, every row of the table
becomes locked (...)
Without an index, the SELECT ... FOR UPDATE statement is likely to lock the entire table, whereas with an index, it only locks some rows. Because you didn't lock the right rows in the first statement, an additional lock is acquired during the second statement.
Obviously, a deadlock cannot happen if the whole table is locked (i.e. without an index).
A deadlock can certainly occur in the second setup.
First of all, the definition of the table is wrong. You have no tid column in the table, so i am suspecting the primary key is v_id.
Second of all, if you select for update, you lock the row. Any other select coming until the first transaction is done will wait for the row to be cleared, because it will hit the exact same record. So you will have waits for this row.
However, i pretty much doubt this can be a real serious problem in your case, because first of all, you have the username there, and second of all you have the product id there. It is extremly unlikely that you will have alot of hits on that exact same record you hit initially, and even if you do, the transaction should be running very fast.
You have to understand that by using transactions, you usually give up pretty much on concurrency for consistent data. There is no way to support consistency of data and concurrency at the same time.
I have innodb table read by lot of different instances (cloud)
Daemon in each instance takes 100 rows to "do things" of this table, but I don't want 2 (or more) instances to take the same things.
So I have a "status" column ("todo", "doing", "done").
INSTANCE 1: It takes 100 rows where status = "todo" ... Then I need to UPDATE these rows asap to status "doing", so INSTANCE 2,3,..x can't take the same rows.
How can i do it ?
Please, I would like a solution without LOCKING WHOLE table, but locking just the rows (that's because I use innodb) ... I have read a lot about that (LOCK SHARE MODE, FOR UPDATE, COMMITs ... ) but I do not get the right way ...
You should use LOCK TABLES and UNLOCK TABLES functions to do this:
http://dev.mysql.com/doc/refman/5.1/en/lock-tables.html
use a transaction and then SELECT ... FOR UPDATE when you read the records.
This way the records you read are locked. When you get all the data update the records to "doing" and COMMIT the transaction.
Maybe what you were missing is the use of a transaction, or the correct order of commands. Here is a basic example:
BEGIN TRANSACTION;
SELECT * FROM table WHERE STATUS = 'todo' FOR UPDATE;
// Loop over results in code, save necessary data to array/list..
UPDATE table SET STATUS ='doing' WHERE ...;
COMMIT;
// process the data...
UPDATE table SET STATUS ='done' WHERE ...;