Does the Laravel `increment()` lock the row? - mysql

Does calling the Laravel increment() on an Eloquent model lock the row?
For example:
$userPoints = UsersPoints::where('user_id','=',\Auth::id())->first();
if(isset($userPoints)) {
$userPoints->increment('points', 5);
}
If this is called from two different locations in a race condition, will the second call override the first increment and we still end up with only 5 points? Or will they add up and we end up with 10 points?

To answer this (helpful for future readers): the problem you are asking about depends on database configuration.
Most MySQL engines: MyISAM and InnoDB etc.. use locking when inserting, updating, or altering the table until this feature is explicitly turned off. (anyway this is the only correct and understandable implementation, for most cases)
So you can feel comfortable with what you got, because it will work correct at any number of concurrent calls:
-- this is something like what laravel query builder translates to
UPDATE users SET points += 5 WHERE user_id = 1
and calling this twice with starting value of zero will end up to 10

The answer is actually a tiny bit different for the specific case with ->increment() in Laravel:
If one would call $user->increment('credits', 1), the following query will be executed:
UPDATE `users`
SET `credits` = `credits` + 1
WHERE `id` = 2
This means that the query can be regarded as atomic, since the actual credits amount is retrieved in the query, and not retrieved using a separate SELECT.
So you can execute this query without running any DB::transaction() wrappers or lockForUpdate() calls because it will always increment it correctly.
To show what can go wrong, a BAD query would look like this:
# Assume this retrieves "5" as the amount of credits:
SELECT `credits` FROM `users` WHERE `id` = 2;
# Now, execute the UPDATE statement separately:
UPDATE `users`
SET `credits` = 5 + 1, `users`.`updated_at` = '2022-04-15 23:54:52'
WHERE `id` = 2;
Or in a Laravel equivalent (DONT DO THIS):
$user = User::find(2);
// $user->credits will be 5.
$user->update([
// Shown as "5 + 1" in the query above but it would be just "6" ofcourse.
'credits' => $user->credits + 1
]);
Now, THIS can go wrong easily since you are 'assigning' the credit value, which is dependent on the time that the SELECT statement took place. So 2 queries could update the credits to the same value while the intention was to increment it twice. However, you CAN correct this Laravel code the following way:
DB::transaction(function() {
$user = User::query()->lockForUpdate()->find(2);
$user->update([
'credits' => $user->credits + 1,
]);
});
Now, since the 2 queries are wrapped in a transaction and the user record with id 2 is READ-locked using lockForUpdate(), any second (or third or n-th) instance of this transaction that takes place in parallel should not be able to read using a SELECT query until the locking transaction is complete.

Related

MySQL - how to check reason for UPDATE rowCount = 0

I'm trying to solve some concurrency issues in a database. The application I'm developing needs to be able to work with MySQL, PostgreSQL and SQLite. When doing the UPDATE on a row, I check the resulting row count. There are two cases when the row count can be 0 - a) the row was already updated by another request with the same values or b) the row was deleted by another client. In scenario a there is no issue, but in scenario b the request should fail. Is there an easy/thread-safe way to tell the difference? It would be great if it could return rowCount=0 for scenario a and throw an actual error for scenario b, like INSERT INTO does if two clients concurrently insert a conflicting row. The only thing I can think of is to re-query if the row still exists, but this is 1) wasteful and inelegant and 2) still not thread safe - the row could get re-created between the UPDATE and SELECT. I am using the READ COMMITTED isolation level, so new data can appear during the request.
I noticed that when doing this via the command line you can tell, at least for MySQL. If it works with all three then maybe this is just more of a PHP-PDO question. PDO just has a single rowCount(). For MySQL at least the problem could be solved with PDO::MYSQL_ATTR_FOUND_ROWS but that doesn't seem to apply to Postgres/SQLite.
MariaDB [test]> update test set id=72;
Query OK, 0 rows affected (0.000 sec)
Rows matched: 1 Changed: 0 Warnings: 0
I have also found that I could follow up every UPDATE with a SELECT FOUND_ROWS() but this seems a bit hacky. I'd rather avoid two queries if possible. Ideally I'm looking for a way to have it just report the count of the rows matched, not rows affected.
It turns out that the behavior I'm looking for is already the default in PostgreSQL. The manual states "The count is the number of rows updated, including matched rows whose values did not change".
For MySQL, the desired behavior can be had with PDO::MYSQL_ATTR_FOUND_ROWS.
For Sqlite, the PDO documentation states that rowCount() "returns "0" (zero) with the SQLite driver at all times", but this does not seem to be true. I ran a test script and it seems to already work the same as PostgreSQL. My system is running libsqlite 3.35.5, PHP 7.4.27.
<?php
$db = new \PDO("sqlite:test.db");
$db->query("delete from test");
$result = $db->query("insert into test values(65)");
echo "insert: count: ".$result->rowCount()."\n";
$result = $db->query("update test set id=55 where id=65");
echo "update changed: count: ".$result->rowCount()."\n";
$result = $db->query("update test set id=55 where id=55");
echo "update not changed: count: ".$result->rowCount()."\n";
$result = $db->query("update test set id=55 where id=65");
echo "update not existing: count: ".$result->rowCount()."\n";
yields...
insert: count: 1
update changed: count: 1
update not changed: count: 1
update not existing: count: 0
I am not sure that this solution could help you or not:
Lets assume you have a table like this:
And you want to manage paralell update and deletes by different clients. So my suggestion would be change table structure. And make it client spesific:
So each client could just update/insert based on related data. And Create a view which just shows ID and Value - which works based on the latest status and filter out deleted records -
Query of view could be:
select ID, VALUE
from (
select ID,
VALUE,
STATUS,
ROW_NUMBER() OVER(PARTITION BY ID ORDER BY STATUS_DATE DESC) RN
from table m
)dt
where 1 = 1
and rn = 1
and STATUS <> 'Deleted'
Maybe it is not a good solution for your work. Just an idea maybe it could help.

Rails query executes 2 queries

I have a simple rails query like
a = A.where(type: 'user')
if a.count > 1
#Log Information
end
return a
Rails does lazy loading where it doesn't query the database unless some operation on the result set is executed. This is a fine behavior. But in my case rails ends up executing 2 queries because I call a.count before operating on a
SELECT COUNT(*) FROM `a` WHERE `a`.`type` = 'user';
SELECT `a`.* FROM `a` WHERE `a`.`type` = 'user';
Is there any way I can ask rails to perform the query immediately so that only the second query is executed and the count is returned from the dataset.
You can force the results into an array. I think that to_a will work for that, but entries is a clearer way express that intention since its job is to iterate over the items in an Enumerable and return an array of the enumerated results.
a = A.where(type: 'user').entries
if a.count > 1
#Log Information
end
return a

Select column to update based on value

What I am trying to do is reduce the time needed to aggregate data by producing a roll-up table of sorts. When I insert a record, an after insert trigger is fired which will update the correct row. I would update all of the columns of the roll-up table if I need to, but since there are 25 columns in the table and each insert will only update 2 of them, I would rather be able to dynamically select the columns to update. My current update statement in the after insert trigger looks similar to this:
update peek_at_chu.organization_data_state_log odsl
inner join ( select
lookUpID as org_data_lookup,
i.interval_id,
peek_at_chu.Get_Time_Durration_In_Interval1('s', new.start_time, new.end_time, i.start_time, i.end_time) as time_in_int,
new.phone_state_id
from
(peek_at_chu.interval_info i
join peek_at_chu.interval_step int_s on i.interval_step_id = int_s.interval_step_id)) as usl on odsl.org_date_lookup_id = usl.org_data_lookup
and odsl.interval_id = usl.interval_id
set
total_seconds = total_seconds + usl.time_in_int,
case new.phone_state_id
when 2 then
available_seconds = available_seconds + time_in_int
end;
In this, lookUpID is a variable previously declared in the trigger. The field that will dictate which field of the roll-up table to update is new.phone_state_id. The phone_state_id's are not consistent, that is some numbers are skipped in this table, so an update based on column number is out the window unless I create a mapping.
The case option throws an error but I am hoping to use something similar to that instead of 25 if statements if I can.
You have to update all the columns, but use a conditional to determine whether to give it a new value or keep the old value:
set total_seconds = total_seconds + usl.time_in_int,
available_seconds = IF(new.phone_state_id = 2, available_seconds + time_in_int, available_seconds)
Repeat the pattern in the last line for all the other columns that need to be updated conditionally.

Query execution was interrupted, error #1317

What I have is a table with a bunch of products (books, in this case). My point-of-sale system generates me a report that has the ISBN (unique product number) and perpetual sales.
I basically need to do an update that matches the ISBN from one table with the ISBN from the other and then add the sales from the one table to the other.
This needs to be done for about 30,000 products.
Here is the SQL statement that I am using:
UPDATE `inventory`,`sales`
SET `inventory`.`numbersold` = `sales`.`numbersold`
WHERE `inventory`.`isbn` = `sales`.`isbn`;
I am getting MySQL Error:
#1317 SQLSTATE: 70100 (ER_QUERY_INTERRUPTED) Query execution was interrupted
I am using phpMyAdmin provided by GoDaddy.com
I've probably come to this a bit late, but... It certainly looks like the query is being interrupted by an execution time limit. There may be no easy way around this, but here's a couple of ideas:
Make sure that inventory.isbn and sales.isbn are indexed. If they aren't, adding an index will reduce your execution time dramatically.
if that doesn't work, break the query down into blocks and run it several times:
UPDATE `inventory`,`sales`
SET `inventory`.`numbersold` = `sales`.`numbersold`
WHERE `inventory`.`isbn` = `sales`.`isbn`
AND substring(`inventory`.sales`,1,1) = '1';
The AND clause restricts the search to ISBNs starting with the digit 1. Run the query for each digit from '0' to '9'. For ISBNs you might find selecting on the last character gives better results. Use substring(inventory.sales,-1)`
try to use INNER JOIN in the two tables like that
UPDATE `inventory`
INNER JOIN `sales`
ON `inventory`.`isbn` = `sales`.`isbn`
SET `inventory`.`numbersold` = `sales`.`numbersold`
UPDATE inventory,sales
SET inventory.numbersold = sales.numbersold
WHERE inventory.isbn = sales.isbn
AND inventory.id < 5000
UPDATE inventory,sales
SET inventory.numbersold = sales.numbersold
WHERE inventory.isbn = sales.isbn
AND inventory.id > 5000 inventory.id < 10000
...
If the error, you can try to reduce the number to 1000, for example

using ssis to perform operation with high performance

Im trying to make an operation of creating user network based on call detail records in my CDR table.
To make things simple lets say Ive got CDR table :
CDRid
UserAId
UserBId
there is more than 100 mln records so table is quite big.
I reated user2user table:
UserAId
UserBId
NumberOfConnections
then using curos I iterate through each row in the table, then I make select statement:
if in user2user table there is record which has UserAId = UserAId from CDR record and UserBId = UserBId from CDR record then increase NumberOfConnections.
otherwise insert such a row which NumebrOfConnections = 1.
Quite simple task and it works as I said using cursor but it is very bad in performance (estimated time at my computer ~60 h).
I heard about Sql Server Integration Services that it has got better performance when we are talking about such big tables.
Problem is that I have no idea how to customize SSIS package for creating such task.
If anyone has got any idea how to help me, any good resources etc I would be really thankful.
Maybe there is any other good solution to make it work faster. I used indexes and variable tables and so on and performance is still pure.
thanks for help,
P.S.
This is script which I wrote and execution of this takes sth like 40 - 50 h.
DECLARE CDR_cursor CURSOR FOR
SELECT CDRId, SubscriberAId, BNumber
FROM dbo.CDR
OPEN CDR_cursor;
FETCH NEXT FROM CDR_cursor
INTO #CdrId, #SubscriberAId, #BNumber;
WHILE ##FETCH_STATUS = 0
BEGIN
--here I check if there is a user with this number (Cause in CDR i only have SubscriberAId --and BNumber so that I need to check which one user is this (I only have users from
--network so that each time I cant find this user I add one which is outide network)
SELECT #UserBId = (Select UserID from dbo.Number where Number = #BNumber)
IF (#UserBId is NULL)
BEGIN
INSERT INTO dbo.[User] (ID, Marked, InNetwork)
VALUES (#OutUserId, 0, 0);
INSERT into dbo.[Number](Number, UserId) values (#BNumber, #OutUserId);
INSERT INTO dbo.User2User
VALUES (#SubscriberAId, #OutUserId, 1)
SET #OutUserId = #OutUserId - 1;
END
else
BEGIN
UPDATE dbo.User2User
SET NumberOfConnections = NumberOfConnections + 1
WHERE User1ID = #SubscriberAId AND User2ID = #UserBId
-- Insert the row if the UPDATE statement failed.
if(##ROWCOUNT = 0)
BEGIN
INSERT INTO dbo.User2User
VALUES (#SubscriberAId, #UserBId, 1)
END
END
SET #Counter = #Counter + 1;
if((#Counter % 100000) = 0)
BEGIN
PRINT Cast (#Counter as NVarchar(12));
END
FETCH NEXT FROM CDR_cursor
INTO #CdrId, #SubscriberAId, #BNumber;
END
CLOSE CDR_cursor;
DEALLOCATE CDR_cursor;
The thing about SSIS is that it probably won't be much faster than a cursor. It's pretty much doing the same thing: reading the table record by record, processing the record and then moving to the next one. There are some advanced techniques in SSIS like sharding the data input that will help if you have heavy duty hardware, but without that it's going to be pretty slow.
A better solution would be to write an INSERT and an UPDATE statement that will give you what you want. With that you'll be better able to take advantage of indices on the database. They would look something like:
WITH SummaryCDR AS (UserAId, UserBId, Conns) AS
(
SELECT UserAId, UserBId, COUNT(1) FROM CDR
GROUP BY UserAId, UserBId)
UPDATE user2user
SET NumberOfConnections = NumberOfConnections + SummaryCDR.Conns
FROM SummaryCDR
WHERE SummaryCDR.UserAId = user2user.UserAId
AND SummaryCDR.UserBId = user2user.UserBId
INSERT INTO user2user (UserAId, UserBId, NumberOfConnections)
SELECT CDR.UserAId, CDR.UserBId, Count(1)
FROM CDR
LEFT OUTER JOIN user2user
ON user2user.UserAId = CDR.UserAId
AND user2user.UserBId = CDR.UserBId
WHERE user2user.UserAId IS NULL
GROUP BY CDR.UserAId, CDR.UserBId
(NB: I don't have time to test this code, you'll have to debug it yourself)
is this what you need?
select
UserAId, UserBId, count(CDRid) as count_connections
from cdr
group by UserAId, UserBId
Could you break the conditional update/insert into two separate statements and get rid of the cursor?
Do the INSERT for all the NULL rows and the UPDATE for all the NOT NULL rows.
Why are you even considering doing row-by-row processing on a table that size? You know you can use the merge statment and insert or update and it will be faster. Or you could write an update to insert all rows that need updating in one set-based stament and an insert to insert alll rows when the row doesn't exist in one set-based statement.
Stop using the values clause and use an insert with joins instead. Same thing with updates. If you need extra complexity the case stamenet will probably give you all you need.
In general stop thinking of row-by-row processing. If you can write a select for the cursor, you can write a set-based statement to do the work 99.9% of the time.
You may still want a cursor with a table this large but one to process batches of data (for instance a 1000 records at time) not one to run ro-by-row.