I'm trying to solve some concurrency issues in a database. The application I'm developing needs to be able to work with MySQL, PostgreSQL and SQLite. When doing the UPDATE on a row, I check the resulting row count. There are two cases when the row count can be 0 - a) the row was already updated by another request with the same values or b) the row was deleted by another client. In scenario a there is no issue, but in scenario b the request should fail. Is there an easy/thread-safe way to tell the difference? It would be great if it could return rowCount=0 for scenario a and throw an actual error for scenario b, like INSERT INTO does if two clients concurrently insert a conflicting row. The only thing I can think of is to re-query if the row still exists, but this is 1) wasteful and inelegant and 2) still not thread safe - the row could get re-created between the UPDATE and SELECT. I am using the READ COMMITTED isolation level, so new data can appear during the request.
I noticed that when doing this via the command line you can tell, at least for MySQL. If it works with all three then maybe this is just more of a PHP-PDO question. PDO just has a single rowCount(). For MySQL at least the problem could be solved with PDO::MYSQL_ATTR_FOUND_ROWS but that doesn't seem to apply to Postgres/SQLite.
MariaDB [test]> update test set id=72;
Query OK, 0 rows affected (0.000 sec)
Rows matched: 1 Changed: 0 Warnings: 0
I have also found that I could follow up every UPDATE with a SELECT FOUND_ROWS() but this seems a bit hacky. I'd rather avoid two queries if possible. Ideally I'm looking for a way to have it just report the count of the rows matched, not rows affected.
It turns out that the behavior I'm looking for is already the default in PostgreSQL. The manual states "The count is the number of rows updated, including matched rows whose values did not change".
For MySQL, the desired behavior can be had with PDO::MYSQL_ATTR_FOUND_ROWS.
For Sqlite, the PDO documentation states that rowCount() "returns "0" (zero) with the SQLite driver at all times", but this does not seem to be true. I ran a test script and it seems to already work the same as PostgreSQL. My system is running libsqlite 3.35.5, PHP 7.4.27.
<?php
$db = new \PDO("sqlite:test.db");
$db->query("delete from test");
$result = $db->query("insert into test values(65)");
echo "insert: count: ".$result->rowCount()."\n";
$result = $db->query("update test set id=55 where id=65");
echo "update changed: count: ".$result->rowCount()."\n";
$result = $db->query("update test set id=55 where id=55");
echo "update not changed: count: ".$result->rowCount()."\n";
$result = $db->query("update test set id=55 where id=65");
echo "update not existing: count: ".$result->rowCount()."\n";
yields...
insert: count: 1
update changed: count: 1
update not changed: count: 1
update not existing: count: 0
I am not sure that this solution could help you or not:
Lets assume you have a table like this:
And you want to manage paralell update and deletes by different clients. So my suggestion would be change table structure. And make it client spesific:
So each client could just update/insert based on related data. And Create a view which just shows ID and Value - which works based on the latest status and filter out deleted records -
Query of view could be:
select ID, VALUE
from (
select ID,
VALUE,
STATUS,
ROW_NUMBER() OVER(PARTITION BY ID ORDER BY STATUS_DATE DESC) RN
from table m
)dt
where 1 = 1
and rn = 1
and STATUS <> 'Deleted'
Maybe it is not a good solution for your work. Just an idea maybe it could help.
Related
my php code (the only relevant part):
$sql="UPDATE `posts` SET `likes`=`likes`+1 WHERE `id`='$id'";
is not working. i am trying to increment an int which is by default set to zero. the variable id is correctly set (a response from an html form) and the insert function works perfectly in the next line. I've researched this for a few hours but have come up empty handed :\ was wondering if anyone could assist me or point me in the right direction of a possible solution.
this is my query executed in the next line:
$res= $mysqli->query($sql);
$mysql being the database connection and $sql being the code snippet above.
this is the table
The issue is the NULL allowance your table has for the integer column. I would recommend not using NULL on an integer column. For your current issue there are at least two ways to resolve it:
Use coalesce to select the value, or use 0 if it is null:
update posts
set likes = coalesce(likes, 0) + 1
where id = 10;
Demo: http://sqlfiddle.com/#!9/54d9ae/1
Update the row so it actually has a 0 value.
update posts
set likes = 0
where id = 10;
... or you could just set it to 1 with that update. With this approach though you can take off the where clause and correct the whole table, assuming the NULL was incorrect.
Does calling the Laravel increment() on an Eloquent model lock the row?
For example:
$userPoints = UsersPoints::where('user_id','=',\Auth::id())->first();
if(isset($userPoints)) {
$userPoints->increment('points', 5);
}
If this is called from two different locations in a race condition, will the second call override the first increment and we still end up with only 5 points? Or will they add up and we end up with 10 points?
To answer this (helpful for future readers): the problem you are asking about depends on database configuration.
Most MySQL engines: MyISAM and InnoDB etc.. use locking when inserting, updating, or altering the table until this feature is explicitly turned off. (anyway this is the only correct and understandable implementation, for most cases)
So you can feel comfortable with what you got, because it will work correct at any number of concurrent calls:
-- this is something like what laravel query builder translates to
UPDATE users SET points += 5 WHERE user_id = 1
and calling this twice with starting value of zero will end up to 10
The answer is actually a tiny bit different for the specific case with ->increment() in Laravel:
If one would call $user->increment('credits', 1), the following query will be executed:
UPDATE `users`
SET `credits` = `credits` + 1
WHERE `id` = 2
This means that the query can be regarded as atomic, since the actual credits amount is retrieved in the query, and not retrieved using a separate SELECT.
So you can execute this query without running any DB::transaction() wrappers or lockForUpdate() calls because it will always increment it correctly.
To show what can go wrong, a BAD query would look like this:
# Assume this retrieves "5" as the amount of credits:
SELECT `credits` FROM `users` WHERE `id` = 2;
# Now, execute the UPDATE statement separately:
UPDATE `users`
SET `credits` = 5 + 1, `users`.`updated_at` = '2022-04-15 23:54:52'
WHERE `id` = 2;
Or in a Laravel equivalent (DONT DO THIS):
$user = User::find(2);
// $user->credits will be 5.
$user->update([
// Shown as "5 + 1" in the query above but it would be just "6" ofcourse.
'credits' => $user->credits + 1
]);
Now, THIS can go wrong easily since you are 'assigning' the credit value, which is dependent on the time that the SELECT statement took place. So 2 queries could update the credits to the same value while the intention was to increment it twice. However, you CAN correct this Laravel code the following way:
DB::transaction(function() {
$user = User::query()->lockForUpdate()->find(2);
$user->update([
'credits' => $user->credits + 1,
]);
});
Now, since the 2 queries are wrapped in a transaction and the user record with id 2 is READ-locked using lockForUpdate(), any second (or third or n-th) instance of this transaction that takes place in parallel should not be able to read using a SELECT query until the locking transaction is complete.
I am updating a field in a mysql column namend "frontpage", set it from 0 to 1.
No problem with this query:
mysql_query("UPDATE table SET frontpage='1' WHERE user_id='999' AND poll_id='555'");
What I'd like to accomplish is, in case user_id 999 got already other existing poll_id's set to 1 in the past, these rows should be set to 0 automatically.
As a beginner learning MySQL, I would run 2 queries, first one to set everything to frontpage='0' WHERE user_id='999' and the second query to set frontpage='1' WHERE user_id='999' AND poll_id='555'.
My question now is, could this be done by using only one query, and how?
PS: Not sure if it has something to do with my question, I've read these answers MySQL: Updating all rows setting a field to 0, but setting one row's field to 1 but I haven't really understood the logic, perhaps someone can explain it to a mysql beginner please.
I think you want this logic:
UPDATE table
SET frontpage = (case when poll_id = '555' then '1' else '0' end)
WHERE user_id = '999';
As a note: if the constants should really be integers, then drop the single quotes. In fact, you can then simplify the query to:
UPDATE table
SET frontpage = (poll_id = 555)
WHERE user_id = 999;
Good day.
For page navigation useally need use two query:
1) $res = mysql_query("SELECT * FROM Table");
-- query which get all count rows for make links on previous and next pages, example <- 2 3 4 5 6 ->)
2) $res = mysql_query("SELECT * FROM Table LIMIT 20, $num"); // where $num - count rows for page
Tell me please really use only one query to database for make links on previous and next pages ( <- 2 3 4 5 6 -> ) and output rows from page (sql with limit) ?
p.s.: i know that can use two query and SELECT * FROM Table LIMIT 20 - it not answer.
If you want to know how many rows would have been returned from a query while still using LIMIT you can use SQL_CALC_FOUND_ROWS and FOUND_ROWS():
A SELECT statement may include a LIMIT clause to restrict the number of rows the server returns to the client. In some cases, it is desirable to know how many rows the statement would have returned without the LIMIT, but without running the statement again. To obtain this row count, include a SQL_CALC_FOUND_ROWS option in the SELECT statement, and then invoke FOUND_ROWS() afterward:
$res = mysql_query("SELECT SQL_CALC_FOUND_ROWS, * FROM Table");
$count_result = mysql_query("SELECT FOUND_ROWS() AS found_rows");
$rows = mysql_fetch_assoc($rows);
$total_rows = $rows['found_rows'];
This is still two queries (which is inevitable) but is lighter on the DB as it doesn't actually have to run your main query twice.
Many database APIs don't actually grab all the rows of the result set until you access them.
For example, using Python's built-in sqlite:
q = cursor.execute("SELECT * FROM somehwere")
row1 = q.fetchone()
row2 = q.fetchone()
Of course the library is free to prefetch unknown number of rows to improve performance.
I have a large database with 300,000 rows (1.6 GB). I need to delete them all EXCEPT the ones that has the following features:
main_sec=118
main_sec=Companies
type=SWOT
Here is the code I prepared, but somehow, it's deleting all the rows of the table:
DELETE FROM `swots`
WHERE (main_sec <> '118') OR
(main_sec <> 'Companies') OR
(type <> 'SWOT');
Please help me to understand where the mistake is.
DELETE FROM `swots`
WHERE main_sec not in ('118', 'Companies')
and type <> 'SWOT'
It would be faster to insert the rows that you want to keep (assuming they are fewer then the remaining rows) in a new table like :
INSERT INTO main_sec_new
SELECT
*
FROM main_sec
WHERE main_sec IN ('118','Companies')
and type = 'SWOT'
And then just drop the old table
Try this:
DELETE FROM `swots`
WHERE (main_sec not in ('118', 'Companies')) OR
(type <> 'SWOT');
The problem is that main_sec is always not equal to one of those two values in a given record. So, every record meets the where condition in your version.