What I try to accomplish seems simple,
Db type: MyISAM
Table Structure: card_id, status
Query: select an unused card_id from a table, and set the row as "used".
Is it race condition that when two queries running at the same time, and before status is updated, the same card_id is fetched twice?
I did some search already. It seems Lock table is a solution, but it's overkill to me and need Lock Privilege.
Any Idea?
Thanks!
It really depends on what statements you are running.
For plain old UPDATE statements against a MyISAM table, MySQL will obtain a lock on the entire table, so there is no "race" condition between two sessions there. One session will wait until the lock is released, and then proceed with it's own update (or will wait for a specified period, and abort with a "timeout".)
BUT, if what you are asking about is two sessions both running a SELECT against a table, to retrieve an identifier for a row to be updated, and both sessions retrieving the same row identifier, and then both sessions attempting to update the same row, then yes, that's a definite possibility, and one which really does have to be considered.
If that condition is not addressed, then it's basically going to be a matter of "last update wins", the second session will (potentially) overwrite the changes made by a previous update.
If that's an untenable situation for your application, then that does need to be addressed, either with a different design, or with some mechanism that prevents the second update from overwriting the update applied by the first update.
One approach, as you mentioned, is to avoid this situation by first obtaining an exclusive lock on the table (using a LOCK TABLES statement), then running a SELECT to obtain an identifier, and then running an UPDATE to update the identified row, and then finally, releasing the lock (using an UNLOCK TABLES statement.)
That's a workable approach for some low volume, low concurrency applications. But it does have some significant drawbacks. Of primary concern is reduced concurrency, due to the exclusive locks obtained on a single resource, which has the potential to cause a performance bottleneck.
Another alternative is an strategy called "optimistic locking". (As opposed to the previously described approach, which could be described as "pessimistic locking".)
For an "optimistic locking" strategy, an additional "counter" column is added to the table. Whenever an update is applied to a row in the table, the counter for that row is incremented by one.
To make use of this "counter" column, when a query retrieves a row that will (or might) be updated later, that query also retrieves the value of the counter column.
When an UPDATE is attempted, the statement also compares the current value of the "counter" column in the row with the previously retrieved value of the counter column. (We just include a predicate (e.g. in the WHERE clause) of the UPDATE statement. For example,
UPDATE mytable
SET counter = counter + 1
, col = :some_new_value
WHERE id = :previously_fetched_row_identifier
AND counter = :previously_fetched_row_counter
If some other session has applied an update to the row we are attempting to update (sometime between the time our session retrieved the row and before our session is attempting to do the update), then the value of the "counter" column on that row will have been changed.
The predicate on our UPDATE statement checks for that, and if the "counter" has been changed, that will cause our update to NOT be applied. We can then detect this condition (i.e. the affected rows count will be a 0 rather than a 1) and our session can take some appropriate action. ("Hey! Some other session updated a row we were intending to update!")
There are some good write-ups on how to implement an "optimistic locking" strategy.
Some ORM frameworks (e.g. Hibernate, JPA) provide support for this type of locking strategy.
Unfortunately, MySQL does NOT provide support for a RETURNING clause in an UPDATE statement, such as:
UPDATE ...
SET status = 'used'
WHERE status = 'unused'
AND ROWNUM = 1
RETURNING card_id INTO ...
Other RDBMS (e.g. Oracle) do provide that kind of functionality. With that feature of the UPDATE statement available, we can simply run the UPDATE statement to both 1) locate a row with status = 'unused', 2) change the value of status = 'used', and 3) return the card_id (or whatever columns we want) of the row the we just updated.
That gets around the problem of having to run a SELECT and then running a separate UPDATE, with the potential of some other session updating the row between our SELECT and our UPDATE.
But the RETURNING clause is not supported in MySQL. And I've not found any reliable way of emulating this type functionality from within MySQL.
This may work for you
I'm not entirely sure why I previously abandoned this approach using user variables (I mentioned above that I had played around with this. I think maybe I needed something more general, which would update more than one row and return a set of of id values. Or, maybe there was something that wasn't guaranteed about the behavior of user variables. (Then again, I only reference user variables in carefully constructed SELECT statements; I don't use user variables in DML; it may be because I don't have a guarantee of their behavior.)
Since you are interested in exactly ONE row, this sequence of three statements may work for you:
SELECT #id := NULL ;
UPDATE mytable
SET card_id = (#id := card_id)
, status = 'used'
WHERE status = 'unused'
LIMIT 1 ;
SELECT ROW_COUNT(), #id AS updated_card_id ;
It's IMPORTANT that these three statements run in the SAME database session (i.e. keep a hold of the database session; don't let go of it and get a new one.)
First, we initialize a user variable (#id) to a value which we won't confuse with a real card_id value from the table. (A SET #id := NULL statement would work as well, without returning a result, like the SELECT statement does.)
Next, we run the UPDATE statement to 1) find one row where status = 'unused', 2) change the value of the status column to 'used', and 3) set the value of the #id user variable to the card_id value of the row we changed. (We'd want that card_id column to be integer type, not character, to avoid any possible character set translation issues.)
Next, we run a query get the number of rows changed by the previous UPDATE statement, using the ROW_COUNT() function (we are going to need to verify that this is 1 on the client side), and retrieve the value of the #id user variable, which will be the card_id value from the row that was changed.
After I post this questions, I thought of a solution which is exactly the same as the one you mentioned at the end. I used update statement, which is "update TABLE set status ='used' where status = 'unused' limit 1", which returns the primary Id of the TABLE, and then I can use this primary ID to get cart_id. Says there are two update statements occurs at the same time, as you said, "MySQL will obtain a lock on the entire table, so there is no "race" condition between two sessions there", so this should solve my issue. But I am not sure why you said, "MySQL does NOT provide support an style statement".
Related
I want to update a row in the table，before updating, do I need to check if there is any change in each column?
In MySQL, you do not need to check the value. MySQL does not update the record if there is no change. That means that MySQL does not incur the overhead of logging or writing the data. There is a slight cost of checking if any values changed, and I think it does run the triggers.
Other databases behave differently.
This is in an arcane location in the documentation, where mysql_affected_rows() is described:
For UPDATE statements, the affected-rows value by default is the
number of rows actually changed.
. . .
An integer greater than zero indicates the number of rows affected or
retrieved. Zero indicates that no records were updated for an UPDATE
statement, no rows matched the WHERE clause in the query or that no
query has yet been executed.
That would be totally up to you to check if anything is in there before hand. You can do updates based on a single row, multiple, or all of them.
An example where you update a specific row is like this:
UPDATE your_table
SET Username='TestName123'
WHERE UserID='12486';
where you would be changing the username where the userid is 12486
OR you can update all of the rows with data you want like
UPDATE Customers
SET Country='USA'
This would update every record to have the Country column be filled with USA.
Note: I'm new to databases and PHP
I have an order column that is set to auto increment and unique.
In my PHP script, I am using AJAX to get new data but the problem with that is, is that the order skips numbers and is substantially higher thus forcing me to manually update the numbers when the data is inserted. In this case I would end up changing 782 to 38.
$SQL = "INSERT IGNORE INTO `read`(`title`,`url`) VALUES\n ".implode( "\n,",array_reverse( $sql_values ) );
How can I get it to increment +1?
The default auto_increment behavior in MySQL 5.1 and later will "lose" auto-increment values if the INSERT fails. That is, it increments by 1 each time, but doesn't undo an increment if the INSERT fails. It's uncommon to lose ~750 values but not impossible (I consulted for a site that was skipping 1500 for every INSERT that succeeded).
You can change innodb_autoinc_lock_mode=0 to use MySQL 5.0 behavior and avoid losing values in some cases. See http://dev.mysql.com/doc/refman/5.1/en/innodb-auto-increment-handling.html for more details.
Another thing to check is the value of the auto_increment_increment config variable. It's 1 by default, but you may have changed this. Again, very uncommon to set it to something higher than 1 or 2, but possible.
I agree with other commenters, autoinc columns are intended to be unique, but not necessarily consecutive. You probably shouldn't worry about it so much unless you're advancing the autoinc value so rapidly that you could run out of the range of an INT (this has happened to me).
How exactly did you fix it skipping 1500 for ever insert?
The cause of the INSERT failing was that there was another column with a UNIQUE constraint on it, and the INSERT was trying to insert duplicate values in that column. Read the manual page I linked to for details on why this matters.
The fix was to do a SELECT first to check for existence of the value before attempting to INSERT it. This goes against common wisdom, which is to just try the INSERT and handle any duplicate key exception. But in this case, the side-effect of the failed INSERT caused an auto-inc value to be lost. Doing a SELECT first eliminated almost all such exceptions.
But you also have to handle a possible exception, even if you SELECT first. You still have a race condition.
You're right! innodb_autoinc_lock_mode=0 worked like a charm.
In your case, I would want to know why so many inserts are failing. I suspect that like many SQL developers, you aren't checking for success status after you do your INSERTs in your AJAX handler, so you never know that so many of them are failing.
They're probably still failing, you just aren't losing auto-inc id's as a side effect. You should really diagnose why so many fails occur. You could be either generating incomplete data, or running many more transactions than necessary.
After you change 782 in 38 you can reset the autoincrement with ALTER TABLE mytable AUTO_INCREMENT = 39. This way you continue at 39.
However, you should check why your gap is so high and change your design accordingly. Changing the autoincement should not be "default" behaviour.
I know the question has been answered already.. But if you have deleted rows in the table before, mysql will remember the used ID/Number because typically your Auto increment is Unique.. So therefore will not create duplicate increments.. To reindex and increment from the current max ID/integer you could perform:
ALTER TABLE TableName AUTO_INCREMENT=(SELECT max(order) + 1 FROM tablename)
auto increment doesn't care, if you delete some rows - everytime you insert a row, the value is incremented.
If you want a numbering without gaps, don't use auto increment and do it by yourself. You could use something like this to achive this for inserting
INSERT INTO tablename SET
`order` = (SELECT max(`order`) + 1 FROM (SELECT * from tablename) t),
...
and if you delete a row, you have to rearange the order column manually
I'm using MySQL database to store student projects (every single project has its own unique ID, as the first screenshot shows).
The 'ID' column is set as auto_increment and PRIMARY KEY.
All projects must be ordered (only) by their ID ascending. But every time I insert a new project into my database and set lower value in the 'Year' field (lower value than I entered last time, when I was inserting my previous project), my projects become automatically ordered by this 'Year' field (as shown on the second screenshot).
Is there any way, how to set my table for automatic ordering all newly added projects only by the 'ID' column? Yeah, I know that I can change the ordering with ALTER TABLE tablename ORDER BY columnname ASC;after I place every new record, but can it be done automatically?
Thx to everyone who helps.
Q: Is there any way, how to set my table for automatic ordering all newly added projects only by the 'ID' column?
A: There is no "automatic ordering" in a MySQL table. (Some storage engines, such as InnoDB, are "index organized" structures, and do store rows in order by the cluster key.) But this organization does not define or specify the order of rows returned by a SELECT statement.
Without an ORDER BY clause on a SELECT statement, then the MySQL server can return rows in any order it chooses to. There is no guarantee of any "automatic" or "default" ordering.
When we run a SQL SELECT statement without an ORDER BY clause, we do observe that rows tend to be returned in a consistent order. This behavior isn't guaranteed, and it isn't "automatic".
This consistent "ordering" behavior we observe is due to the MySQL server performing a consistent set of operations, on a consistent set of data.
Performing an ALTER TABLE statement to rebuild the entire table is not the solution to the "automatic ordering" issue.
If you want the MySQL server to return rows in a specific order, then the solution is to add an ORDER BY clause to the SELECT statement.
The client that executes the SELECT statement is free to do whatever it wants with the rows it retrieves. The client can perform operations such as filtering, ordering, or whatever, to come up with what gets returned in the user interface.
Some clients (like the mysql command line) don't implement any functions for "filtering" or "ordering" rows, so the only way for the client to specify an order that rows should be returned in is the ORDER BY clause on statement itself. (The MySQL command line client returns rows in the user interface in the same order that they are retrieved.)
I expect that phpMyAdmin does the same thing, it displays the rows in the same order that they are returned from the MySQL server.
Order in query results should be determined by the ORDER clause. Don't rely on default order applied by phpmyadmin (or some other tool).
You have to distinct retrieving from inserting/updating.
About automatic order INTO phpmyadmin, maybe using bookmarks on queries : How can I set a default sort for tables in PHPMyAdmin (i.e. always "Primary key - Descending")
I have a fairly basic query:
UPDATE the_table SET col1=[something], col2=[something else] WHERE col1 IS NULL AND col2 IS NULL LIMIT 1;
Immediately after issuing the query, the caller does:
SELECT col3 FROM the_table where col1=[something], col2=[something else];
Unfortunately, concurrent callers are claiming the same row.
I'd rather not do a SELECT FOR UPDATE, because the [select, update, select] would involve three rpcs to the database instead of two (which is bad enough.)
I gather that some dialects of sql allow UPDATE the_table WITH(UPDLOCK), but mine (galera/MySQL) does not. I find it appalling that I'd have to go through this many DB hits to execute such a basic concept. I find that most of my searching efforts end on pages that discuss dialects that DO support UPDLOCK.
Where does it go from here?
Do you have autocommit=1?
Without transactional integrity, some other connection can slip in and change the row before you execute the SELECT.
Note that there could be multiple NULL rows, so the UPDATE may be changing many rows.
Did you check the "rows affected" after the UPDATE? Maybe no rows were changed.
I think that it would be better to either execute all the queries in a transaction or to use a stored proc which will be responsible to make all the select and update stuff and then return back to you the respective data from the last select statement. Having such a flow out of transaction, raises issues as the one you describe. You need to lock the row in order not to allow other callers retrieve "dirty" (not up to date) data.
I have a cronjob that loops through and updates a MySQL table row by row. After the table is 'completed', I would like to execute the cronjob exactly 1 more time, to perform various cleanup activities.
In execute a cronjob exactly once, thaJeztah states:
It's best to set that value in the mysql database, e.g. needs_cleanup = 1. That way you can always find those records at a later time. Keeping it in the database allows to to recover, for example, if a cron-job wasn't executed or failed half-way the loop. – thaJeztah
I think this would be a good solution if its possible, as in my case I only need to set the flag once a day. If it is possible could someone point me to the sql commands nescesary to execute the placement of a simple binary flag, with values 0,1 in a mysql table?
UPDATE mytable SET needs_cleanup = 1
does it for all records of mytable. If you need for a single record, add a WHERE condition, e.g.
UPDATE mytable SET needs_cleanup = 1
WHERE id = 1