Question on locking scope in SQL Server (SQL Azure to be precise).
Scenario
A bunch of records are selected using a select statements.
We loop through the records
Each record is updated within a transactionscope -
(each record is independent of the other and there is no need for a table lock)
Am I right in assuming that the above would result in a row level lock of just that particular record row?
Framing the question within the context of a concrete example.
In the below example would each item in itemsToMove be locked one at a time?
var itemsToMove = ObjectContext.Where(emp => emp.ExpirationDate < DateTime.Now)
foreach(Item expiredItem in itemsToMove)
{
bool tSuccess = false;
using (TransactionScope transaction = new TransactionScope())
{
try
{
//We push this to another table. In this case Azure Storage.
bool bSuccess = PushToBackup();
if(bSuccess)
{
ObjectContext.DeleteObject(expiredItem);
}
else
{
//throw an exception or return
return false;
}
ObjectContext.SaveChanges();
transaction.Complete();
tSuccess = true;
}
catch (Exception e)
{
return cResults;
}
}
}
if (tSuccess)
{
ObjectContext.AcceptAllChanges();
}
Provided that there isn't any outer / wrapper transaction calling your code, each call to transaction.Complete() should commit and release any locks.
Just a couple of quick caveats
SQL will not necessarily default to row level locking - it may use page level or higher locks (recommend that you leave SQL to its own devices, however)
Note that the default isolation level of a new TransactionScope() is read serializable. This might be too pessimistic for your scenario.
Related
Multiple consumers are trying to update 'processedRecords' and 'erredRecords' fields of JobTracker. But I am facing issue where random values updated in these columns. Mysql is on default isolation level(MySQL version 5.7)
I thought of locking row by using PESSIMISTIC_WRITE. But following code doesn't give any exception and also row values are not updated.
Someone please help me in resolving this.
Please note:
1. Any solution that changes isolation level for entire DB is not helpful as it would effect other flows in the application.
2. Both 'processedRecords' and 'erredRecords' are of type Long.
#Override
#Transactional
public Boolean updateRecords(Long jobTrackerId, Long processedRecords, Long erredRecords) {
try{
JobTracker jobTracker = getEntityManager(true).find(JobTracker.class, jobTrackerId);
getEntityManager(true).lock(jobTracker, LockModeType.PESSIMISTIC_WRITE);
if(jobTracker == null){
return false;
}
if(processedRecords!=null){
jobTracker.setProcessedRecords(processedRecords);
}
if(erredRecords!=null){
jobTracker.setErredRecords(erredRecords);
}
if(jobTracker.getErredRecords() != null && jobTracker.getProcessedRecords() != null && jobTracker.getTotalRecords() != null){
if(jobTracker.getErredRecords() + jobTracker.getProcessedRecords() == jobTracker.getTotalRecords()){
jobTracker.setStatus("Completed");
}else if(jobTracker.getErredRecords() + jobTracker.getProcessedRecords() > jobTracker.getTotalRecords()){
jobTracker.setStatus("Erred");
}
}
getEntityManager(true).merge(jobTracker);
return true;
}catch(PersistenceException e){
logger.error("For job id:" + jobTrackerId + " Exception while updating the jobTracker records data", e);
return false;
}catch (Exception e){
logger.error("For job id:" + jobTrackerId + "Generic Exception while updating the jobTracker records data",e);
return false;
}
}
Using the #Transactional annotation we can set the isolation level of a Spring managed transactional method. This means that the transaction in this method is executed will have that isolation level. In your case, all you have to do is the following.
#Transactional(isolation=Isolation.READ_COMMITTED)
Now if you were to set the isolation level for your whole DB you would do the following.
<property name="hibernate.connection.isolation">2</property>
I've a table with a column that needs to be constantly recomputed and I want this table to be scallable. Users must be able to write on it as well.
It's difficult to test this type of things without having a server and concurrent users, at least I don't know how.
So is one of those two options viable ?
#ApplicationScoped
public class Abean {
#EJB
private MyService myService;
#Asynchronous
public void computeTheData(){
long i = 1;
long numberOfRows = myService.getCountRows(); // gives the number of row in the table
while(i<numberOfRows){
myService.updateMyRow(i);
}
computeTheData(); // recursion so it never stops, I'm wondering if this wouldn't spawn more threads and if it would be an issue.
}
}
public class MyService implements MyServiceInterface{
...
public void updateMyRows(int row){
Query query = em.createQuery("SELECT m FROM MyEntity WHERE m.id=:id");
Query.setParameter("id", row);
List<MyEntity> myEntities = (MyEntity) query.getResultList();
myEntity.computeData();
}
}
VS
#ApplicationScoped
public class Abean {
#EJB
private MyService myService;
#Asynchronous
public void computeTheData(){
myService.updateAllRows();
}
}
public class MyService implements MyServiceInterface{
...
public void updateAllRows(int page){
Query query = em.createQuery("SELECT m FROM MyEntity");
List<MyEntity> myEntities = (MyEntity) query.getResultList();
myEntity.computeData();
}
}
Is any of this viable ? I'm using mysql and the engine for tables is innoDB.
You should use pessimistic locking to lock modified rows before update, so that manual modifications by a user are not in conflict with background updates. If you did not use locking, your user's modifications would sometimes be rolled back, if they collide with background job having modified the same row.
Also, with pessimistic locking, your user may encounter rollback if her transaction waits to acquire the lock for longer than timeout happends. To prevent this, you should make all transactions, which use pessimistic locks, as short as possible. Therefore, the background job should create a new transaction for every row or small group of rows, if it may run longer than reasonable time. Locks are released only after the transaction finishes (User will wait until lock is released).
Example of how your MyService could look like, running every update in separate transaction (in reality, you may run multiple updates in batch in single transaction, passing list or range of ids as parameter to updateMyRows):
public class MyService implements MyServiceInterface{
...
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW) // this will create a new transaction when running this method from another bean, e.g. from Abean
public void updateMyRows(int row){
TypedQuery<MyEntity> query = em.createQuery(SELECT m FROM MyEntity WHERE m.id=:id", MyEntity.class);
query.setParameter("id", row);
query.setLockMode(LockModeType.PESSIMISTIC_WRITE); // this will lock all entities retrieved by the query
List<MyEntity> myEntities = query.getResultList();
if (!myEntities.isEmpty()) {
myEntities.get(0).computeData();
}
}
}
When you use only id in where condition, you may consider em.find(row, MyEntity.class, LockModeType.PESSIMISTIC_WRITE).computeData() instead of using a query (add null pointer check after em.find())
Other notes:
It is not clear from the question how you trigger the background job. Running the job infinitely, as you wrote in the example, would on one hand NOT create additional threads (as you call the methond on the same bean, annotations are not considered recursively). On the other hand, if there is an exception, your background job should at least handle exceptions so that it will not be stopped. You may also want to add some wait time between subsequent executions.
It is better to run background jobs as scheduled jobs. One possible option is #Schedule annotation instead of #Asynchronous. You may specify the frequency, in which the job will be executed in background. It is then good to check in the beginning of your job, whether previous execution is finished. Another option with Java EE 7 is to use ManagedScheduledExecutorService to trigger a background job periodically at specified interval.
I had an issue with MySQL db where the setAutocommit were being logged many times even though there was no autocommit mode change. For example, if I call setautocommit(false) 5 times, I see 5 queries in the query log saying set autommit=0;
This is usually not supposed to happen as the autocommit mode hasnt changed. The query should be sent only when I am changing the autocommit mode i.e. say 1 to 0.
When I did some looking in the MySQL connection class implementation, I figured that they check useLocalSessionState value to decide whether they want to execute the query or not?
if ((getUseLocalSessionState()) && (this.autoCommit == autoCommitFlag)) {
needsSetOnServer = false;
}
So, even though this.autocommit & autocommit flag are same, needsSetOnServer is NOT set to false becuase useLocalSessionState is defaulted to false.
Once I add useLocalSessionState = true in my connection URL, I dont see the unnecessary query logs.
So, my questions are :
What is the significance of useLocalSessionState ? Why is it used generally?
Is it safe to set useLocalSessionState = true?
PS : I see that SQL Server handles this kind of a scenario without any such dependencies i.e. (Code snippet from SQLServerConnection
if (paramBoolean == this.databaseAutoCommitMode) {
return;
}
Referring to http://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html:
Should the driver refer to the internal values of autocommit and transaction isolation that are set by Connection.setAutoCommit() and Connection.setTransactionIsolation() and transaction state as maintained by the protocol, rather than querying the database or blindly sending commands to the database for commit() or rollback() method calls?
My interpretation of this (after looking to the code of the MySQL JDBC driver) is that instead of issuing e.g. a SELECT ##session.tx_read_only, it checks the readOnly property of the connection class:
ConnectionImpl.java (MySQL JDBC connecotr 5.1.31):
public boolean isReadOnly() throws SQLException {
return this.isReadOnly(true);
}
public boolean isReadOnly(boolean useSessionStatus) throws SQLException {
if(useSessionStatus && /*...*/ && !this.getUseLocalSessionState()) {
//...
rs = stmt.executeQuery("select ##session.tx_read_only");
//...
} else {
return this.readOnly;
}
}
Furthermore, it only issues a SQL command for setting the read only state if useLocalSessionState is false:
public void setReadOnlyInternal(boolean readOnlyFlag) throws SQLException {
if((/*...*/ !this.getUseLocalSessionState()) {
this.execSQL((StatementImpl)null, "set session transaction " + (readOnlyFlag?"read only":"read write"), -1, (Buffer)null, 1003, 1007, false, this.database, (Field[])null, false);
}
this.readOnly = readOnlyFlag;
}
I assume that the same applies for the auto-commit and transaction isolation level properties.
I want to perform a certain set of operations on multiple models/tables using Doctrine with Symfony. Here is what I'm doing:
public function myFunc()
{
$conn = Doctrine_Manager::connection();
try {
$conn->beginTransaction();
$prop_trans->save($conn);
self::doSomething1($conn);
$bill_appor->save($conn);
// Final Committ
$conn->commit();
return $prop_trans;
} catch (Exception $exc) {
if ($conn)
$conn->rollback();
throw $exc;
}
}
public function doSomething($conn)
{
$obj = new Trans();
// this function might create & save another child record
$obj->doSomething2($conn);
$obj->save($conn);
}
However, when there is an exception (code or db level), I wonder if the rollback works, as I see some records being saved.
My understanding is that, as long as I've opened a connection, begun a transaction, all methods using the connection $conn are running in the same transaction. If something fails, all rollback.
I also tried to use savepoints, but I haven't been able to work with them. Can someone pl tell me if passing the connection around is enough to make everything run the transaction?
Is it because a MySQL auto_committ attribute is set or something?
Thanks
Well, transactions are supported only over InnoDB tables and I believe, that some of your tables are MyISAM. So, rollback works only on InnoDB ones and you see changes to MyISAM ones left intact.
just try
catch (Exception $exc) {
$conn->rollback();
}
in your try ctach
or
just put try catch on your other function as well
Hallo,
I have web service that has multiple methods that can be called. Each time one of these methods is called I am logging the call to a statistics database so we know how many times each method is called each month and the average process time.
Each time I log statistic data I first check the database to see if that method for the current month already exists, if not the row is created and added. If it already exists I update the needed columns to the database.
My problem is that sometimes when I update a row I get the "Row not found or changed" exception and yes I know it is because the row has been modified since I read it.
To solve this I have tried using the following without success:
Use using around my datacontext.
Use using around a TransactionScope.
Use a mutex, this doesn’t work because the web service is (not sure I am calling it the right think) replicated out on different PC for performance but still using the same database.
Resolve concurrency conflict in the exception, this doesn’t work because I need to get the new database value and add a value to it.
Below I have added the code used to log the statistics data. Any help would be appreciated very much.
public class StatisticsGateway : IStatisticsGateway
{
#region member variables
private StatisticsDataContext db;
#endregion
#region Singleton
[ThreadStatic]
private static IStatisticsGateway instance;
[ThreadStatic]
private static DateTime lastEntryTime = DateTime.MinValue;
public static IStatisticsGateway Instance
{
get
{
if (!lastEntryTime.Equals(OperationState.EntryTime) || instance == null)
{
instance = new StatisticsGateway();
lastEntryTime = OperationState.EntryTime;
}
return instance;
}
}
#endregion
#region constructor / initialize
private StatisticsGateway()
{
var configurationAppSettings = new System.Configuration.AppSettingsReader();
var connectionString = ((string)(configurationAppSettings.GetValue("sqlConnection1.ConnectionString", typeof(string))));
db = new StatisticsDataContext(connectionString);
}
#endregion
#region IStatisticsGateway members
public void AddStatisticRecord(StatisticRecord record)
{
using (db)
{
var existing = db.Statistics.SingleOrDefault(p => p.MethodName == record.MethodName &&
p.CountryID == record.CountryID &&
p.TokenType == record.TokenType &&
p.Year == record.Year &&
p.Month == record.Month);
if (existing == null)
{
//Add new row
this.AddNewRecord(record);
return;
}
//Update
existing.Count += record.Count;
existing.TotalTimeValue += record.TotalTimeValue;
db.SubmitChanges();
}
}
I would suggest letting SQL Server deal with the concurrency.
Here's how:
Create a stored procedure that accepts your log values (method name, month/date, and execution statistics) as arguments.
In the stored procedure, before anything else, get an application lock as described here, and here. Now you can be sure only one instance of the stored procedure will be running at once. (Disclaimer! I have not tried sp_getapplock myself. Just saying. But it seems fairly straightforward, given all the examples out there on the interwebs.)
Next, in the stored procedure, query the log table for a current-month's entry for the method to determine whether to insert or update, and then do the insert or update.
As you may know, in VS you can drag stored procedures from the Server Explorer into the DBML designer for easy access with LINQ to SQL.
If you're trying to avoid stored procedures then this solution obviously won't be for you, but it's how I'd solve it easily and quickly. Hope it helps!
If you don't want to use the stored procedure approach, a crude way of dealing with it would simply be retrying on that specific exception. E.g:
int maxRetryCount = 5;
for (int i = 0; i < maxRetryCount; i++)
{
try
{
QueryAndUpdateDB();
break;
}
catch(RowUpdateException ex)
{
if (i == maxRetryCount) throw;
}
}
I have not used the sp_getapplock, instead I have used HOLDLOCK and ROWLOCK as seen below:
CREATE PROCEDURE [dbo].[UpdateStatistics]
#MethodName as varchar(50) = null,
#CountryID as varchar(2) = null,
#TokenType as varchar(5) = null,
#Year as int,
#Month as int,
#Count bigint,
#TotalTimeValue bigint
AS
BEGIN
SET NOCOUNT ON;
BEGIN TRAN
UPDATE dbo.[Statistics]
WITH (HOLDLOCK, ROWLOCK)
SET Count = Count + #Count
WHERE MethodName=#MethodName and CountryID=#CountryID and TokenType=#TokenType and Year=#Year and Month=#Month
IF ##ROWCOUNT=0
INSERT INTO dbo.[Statistics] (MethodName, CountryID, TokenType, TotalTimeValue, Year, Month, Count) values (#MethodName, #CountryID, #TokenType, #TotalTimeValue, #Year, #Month, #Count)
COMMIT TRAN
END
GO
I have tested it by calling my web service methods by multiple threads simultaneous and each call is logged without any problems.