I have a game where I use (Spring, Hibernate and MySQL 5.7) in the back-end. At the end of each game, I execute a native query to bulk update balance of the winners. Most games the update executes successfully for all winners, but for a few games I face the problem of updating the balance for most of them except for one or two (random). The update doesn't throw an error and as said Most of the winners' balance is updated successfully. Here is the method in my DAO (I summed it up) with the native query:
#Override
public Integer addAmountToPlayerBalance(List<Long> playerIds, Double amount, Long gameId) {
try
{
StringBuilder queryNative = new StringBuilder();
queryNative.append("update game_player_user set balance = ifnull(balance,0) + :amount where id in (:playerIds)");
Session session = teleEM.unwrap(Session.class);
org.hibernate.Query query = session.createSQLQuery(queryNative.toString());
query.setParameter("amount", amount);
query.setParameterList("playerIds", playerIds);
int numOfUpdatedRecords = query.executeUpdate();
return numOfUpdatedRecords;
}
catch (NoResultException e)
{
return null;
}
}
Notes:
1) I added the code for returning the number of updated records a week ago but it hasn't occurred on production since then. but as I said, it happens occasionally.
2) The list of player Ids includes all of the winners even in the games where the issue appears.
3) The amount is double but rounded to 2 decimal places.
Related
Using EF Core 2.2.6 and Pomelo.EntityFrameworkCore.MySql 2.2.6 (with MySqlConnector 0.59.2)). I have a model for UserData:
public class UserData
{
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public ulong ID { get; private set; }
[Required]
public Dictionary<string, InventoryItem> Inventory { get; set; }
public UserData()
{
Data = new Dictionary<string, string>();
}
}
I have a REST method that can be called that will add items to the user inventory:
using (var transaction = context.Database.BeginTransaction())
{
UserData data = await context.UserData.FindAsync(userId);
// there is code here to detect duplicate entries/etc, but I've removed it for brevity
foreach (var item in items) data.Inventory.Add(item.ItemId, item);
context.UserData.Update(data);
await context.SaveChangesAsync();
transaction.Commit();
}
If two or more calls to this method are made with the same user id then I get concurrent accesses (despite the transaction). This causes the data to sometimes be incorrect. For example, if the inventory is empty and then two calls are made to add items simultaneously (item A and item B), sometimes the database will only contain either A or B, and not both. From logging it appears that it is possible for EF to read from the database while the other read/write is still occurring, causing the code to have the incorrect state of the inventory for when it tries to write back to the db. So I tried marking the isolation level as serializable.
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
Now I sometimes see an exception:
MySql.Data.MySqlClient.MySqlException (0x80004005): Deadlock found when trying to get lock; try restarting transaction
I don't understand how this code could deadlock... Anyways, I tried to proceed by wrapping this whole thing in a try/catch, and retry:
public static async Task<ResponseError> AddUserItem(Controller controller, MyContext context, ulong userId, List<InventoryItem> items, int retry = 5)
{
ResponseError result = null;
try
{
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
UserData data = await context.UserData.FindAsync(userId);
// there is code here to detect duplicate entries/etc, but I've removed it for brevity
foreach (var item in items) data.Inventory.Add(item.ItemId, item);
context.UserData.Update(data);
await context.SaveChangesAsync();
transaction.Commit();
}
}
catch (Exception e)
{
if (retry > 0)
{
await Task.Delay(SafeRandomGenerator(10, 500));
return await AddUserItem(controller, context, userId, items, retry--);
}
else
{
// store exception and return error
}
}
return result;
}
And now I am back to the data being sometimes correct, sometimes not. So I think the deadlock is another problem, but this is the only method accessing this data. So, I'm at a loss. Is there a simple way to read from the database (locking the row in the process) and then writing back (releasing the lock on write) using EF Core? I've looked at using concurrency tokens, but this seems overkill for what appears (on the surface to me) to be a trivial task.
I added logging for mysql connector as well as asp.net server and can see the following failure:
fail: Microsoft.EntityFrameworkCore.Database.Command[20102]
=> RequestId:0HLUD39EILP3R:00000001 RequestPath:/client/AddUserItem => Server.Controllers.ClientController.AddUserItem (ServerSoftware)
Failed executing DbCommand (78ms) [Parameters=[#p1='?' (DbType = UInt64), #p0='?' (Size = 4000)], CommandType='Text', CommandTimeout='30']
UPDATE `UserData` SET `Inventory` = #p0
WHERE `ID` = #p1;
SELECT ROW_COUNT();
A total hack is to just delay the arrival of the queries by a bit. This works because the client is most likely to generate these calls on load. Normally back-to-back calls aren't expected, so spreading them out in time by delaying on arrival works. However, I'd rather find a correct approach, since this just makes it less likely to be an issue:
ResponseError result = null;
await Task.Delay(SafeRandomGenerator(100, 500));
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
// etc
This isn't a good answer, because it isn't what I wanted to do, but I'll post it here as it did solve my problem. My problem was that I was trying to read the database row, modify it in asp.net, and then write it back, all within a single transaction and while avoiding deadlocks. The backing field is JSON type, and MySQL provides some JSON functions to help modify that JSON directly in the database. This required me to write SQL statements directly instead of using EF, but it did work.
The first trick was to ensure I could create the row if it didn't exist, without requiring a transaction and lock.
INSERT INTO UserData VALUES ({0},'{{}}','{{}}') ON DUPLICATE KEY UPDATE ID = {0};
I used JSON_REMOVE to delete keys from the JSON field:
UPDATE UserData as S set S.Inventory = JSON_REMOVE(S.Inventory,{1}) WHERE S.ID = {0};
and JSON_SET to add/modify entries:
UPDATE UserData as S set S.Inventory = JSON_SET(S.Inventory,{1},CAST({2} as JSON)) WHERE S.ID = {0};
Note, if you're using EF Core and want to call this using FromSql then you need to return the entity as part of your SQL statement. So you'll need to add something like this to each SQL statement:
SELECT * from UserData where ID = {0} LIMIT 1;
Here is a full working example as an extension method:
public static async Task<UserData> FindOrCreateAsync(this IQueryable<UserData> table, ulong userId)
{
string sql = "INSERT INTO UserData VALUES ({0},'{{}}','{{}}') ON DUPLICATE KEY UPDATE ID = {0}; SELECT * FROM UserData WHERE ID={0} LIMIT 1;";
return await table.FromSql(sql, userId).SingleOrDefaultAsync();
}
public static async Task<UserData> JsonRemoveInventory(this DbSet<UserData> table, ulong userId, string key)
{
if (!key.StartsWith("$.")) key = $"$.\"{key}\"";
string sql = "UPDATE UserData as S set S.Inventory = JSON_REMOVE(S.Inventory,{1}) WHERE S.ID = {0}; SELECT * from UserData where ID = {0} LIMIT 1;";
return await table.AsNoTracking().FromSql(sql, userId, key).SingleOrDefaultAsync();
}
Usage:
var data = await context.UserData.FindOrCreateAsync(userId);
await context.UserData.JsonRemoveInventory(userId, itemId);
I have two transactions with shareable count down latch. Each transaction is performed in separate thread. I need such mechanism to reproduce "dirty read" situation.
The first transaction do update of "target entity" and flush changes without commit, then I set thread to waiting state (countDownLatch).
In the second transaction I fetch "target entity" and copy dirty field to another entity and do save. After save operation I make countDown to continue the first transaction and rollback it.
Below the code samples of transaction methods.
The first:
#Transactional
public void updatePrice() {
log.info("Start price updating. Thread '{}'", Thread.currentThread().getName());
Collection<Product> products = productRepository.findAll();
products.forEach(this::updatePrice);
productRepository.saveAll(products);
entityManager.flush();
log.info("Flash changes and wait for order transaction");
try {
latch.await();
} catch (InterruptedException e) {
log.error("Something wrong");
Thread.currentThread().interrupt();
}
log.error("Rollback changes");
throw new RuntimeException("Unexpected exception");
}
The second: I have main service that is bounded by transaction and wrapper service on it to count down the latch.
Wrapper:
public void doOrder(Long productId, Long amount) {
log.info("Start do order. Thread '{}'", Thread.currentThread().getName());
orderService.doOrder(productId, amount);
log.info("Order transaction committed");
latch.countDown();
log.info("Finish order process");
}
Main service:
#Transactional
public void doOrder(Long productId, Long amount) {
Product product = productRepository.findById(productId).get();
log.info("Get product");
Order order = Order.builder().price(product.getPrice()).amount(amount).productId(new Long(productId)).build();
orderRepository.save(order);
log.info("Save order");
}
So, at the line
orderRepository.save(order);
the thread has became frozen. I see only insert statement in the logs.
But if I remove relation between 'order' and 'product', the insert doesn't become frozen.
I guess that there is deadlock on 'product' and countDownLatch. But this issue occurs only with mysql jdbc. If I switch onto postgres jdbc, then no issue there.
PS:
The first transaction executes following queries:
select * from product;
update product set price = ...
The second one executes:
select * from product where id = ...;
insert into product_order(product_id, amount, price) values (...)
I am using spring NamedParameterJdbcTemplate with my sql.I am executing 2 queries through jbdc template it is taking 1.1 and 4 s respectively. but the same query if I am running in my sql it is taking 0.5 and 1 s respectively I don't understand what could be my bottleneck. my application and db resides on same server so there can be no network overhead, I have my connection pooled. I can say it is working because query with less amount of data taking 50 ms through application. please let me know what could be my bottleneck
below is my code of namedjdbcTemplate
MapSqlParameterSource parameters = new MapSqlParameterSource();
parameters.addValue("organizationIds", list);
parameters.addValue("fromDate", fromDate);
parameters.addValue("toDate", toDate);
long startTime = System.currentTimeMillis();
List<MappingUI> list =namedParameterJdbcTemplate.query(QueryList.CALCULATE_SCORE,parameters,new RowMapper<MappingUI>() {
#Override
public MappingUI mapRow(ResultSet rs, int rowNum) throws SQLException {
MappingUI mapping= new MappingUI();
mapping.setCompetitor(rs.getInt("ID"));
mapping.setTotalRepufactScore(rs.getFloat("AVG_SCORE"));
return mapping;
}
});
below is my query
SELECT AVG(SCORE.SCORE) AS AVG_SCORE,ANALYSIS.ID AS ID FROM SCORE SCORE,QC_ANALYSIS ANALYSIS WHERE SCORE.TAG_ID = ANALYSIS.ID AND ANALYSIS.ORGANIZATION_ID IN (:organizationIds) AND DATE(ANALYSIS.DATES) BETWEEN DATE(:fromDate) AND DATE(:toDate) GROUP BY ANALYSIS.ORGANIZATION_ID
I am using primefaces autocomplete component with pojos and which is filled from a database table with huge number of rows.
When I select value from database which contains millions of entries (SELECT synonym FROM synonyms WHERE synonym like '%:query%') it takes a very long time to find the word on autocomplete because of huge database entries on my table and it will be bigger in future.
Is there any suggestions on making autocomplete acting fast.
Limiting the number of rows is a great way to speed-up autocomplete. I'm not clear on why you'd limit to 1000 rows though: you can't show 1000 entries in a dropdown; shouldn't you be limiting to maybe 10 entries?
Based on your comments below, here is an example database query that you should be able to adapt to your situation:
String queryString = "select distinct b.title from Books b where b.title like ':userValue'";
Query query = entityManager.createQuery(queryString);
query.setParameter("userValue", userValue + "%");
query.setMaxResults(20);
List<String> results = query.getResultList();
I finally went to using an index solar for doing fast requests while my table will contains more than 4 million entries which must be parsed fastly and without consuming a lot of memory.
Here's I my solution maybe someone will have same problem as me.
public List<Synonym> completeSynonym(String query) {
List<Synonym> filteredSynonyms = new ArrayList<Synonym>();
// ResultSet result;
// SolrQuery solrQ=new SolrQuery();
String sUrl = "http://......solr/synonym_core";
SolrServer solr = new HttpSolrServer(sUrl);
ModifiableSolrParams parameters = new ModifiableSolrParams();
parameters.set("q", "*:*"); // query everything
parameters.set("fl", "id,synonym");// send back just the id
//and synonym values
parameters.set("wt", "json");// this in json format
parameters.set("fq", "synonym:\"" + query+"\"~0"); //my conditions
QueryResponse response;
try {
if (query.length() > 1) {
response = solr.query(parameters);
SolrDocumentList dl = response.getResults();
for (int i = 0; i < dl.size(); i++) {
Synonym s = new Synonym();
s.setSynonym_id((int) dl.get(i).getFieldValue("id"));
s.setSynonymName(dl.get(i).getFieldValue("synonym")
.toString());
filteredSynonyms.add(s);
}
}
} catch (SolrServerException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
return filteredSynonyms;
}
Hallo,
I have web service that has multiple methods that can be called. Each time one of these methods is called I am logging the call to a statistics database so we know how many times each method is called each month and the average process time.
Each time I log statistic data I first check the database to see if that method for the current month already exists, if not the row is created and added. If it already exists I update the needed columns to the database.
My problem is that sometimes when I update a row I get the "Row not found or changed" exception and yes I know it is because the row has been modified since I read it.
To solve this I have tried using the following without success:
Use using around my datacontext.
Use using around a TransactionScope.
Use a mutex, this doesn’t work because the web service is (not sure I am calling it the right think) replicated out on different PC for performance but still using the same database.
Resolve concurrency conflict in the exception, this doesn’t work because I need to get the new database value and add a value to it.
Below I have added the code used to log the statistics data. Any help would be appreciated very much.
public class StatisticsGateway : IStatisticsGateway
{
#region member variables
private StatisticsDataContext db;
#endregion
#region Singleton
[ThreadStatic]
private static IStatisticsGateway instance;
[ThreadStatic]
private static DateTime lastEntryTime = DateTime.MinValue;
public static IStatisticsGateway Instance
{
get
{
if (!lastEntryTime.Equals(OperationState.EntryTime) || instance == null)
{
instance = new StatisticsGateway();
lastEntryTime = OperationState.EntryTime;
}
return instance;
}
}
#endregion
#region constructor / initialize
private StatisticsGateway()
{
var configurationAppSettings = new System.Configuration.AppSettingsReader();
var connectionString = ((string)(configurationAppSettings.GetValue("sqlConnection1.ConnectionString", typeof(string))));
db = new StatisticsDataContext(connectionString);
}
#endregion
#region IStatisticsGateway members
public void AddStatisticRecord(StatisticRecord record)
{
using (db)
{
var existing = db.Statistics.SingleOrDefault(p => p.MethodName == record.MethodName &&
p.CountryID == record.CountryID &&
p.TokenType == record.TokenType &&
p.Year == record.Year &&
p.Month == record.Month);
if (existing == null)
{
//Add new row
this.AddNewRecord(record);
return;
}
//Update
existing.Count += record.Count;
existing.TotalTimeValue += record.TotalTimeValue;
db.SubmitChanges();
}
}
I would suggest letting SQL Server deal with the concurrency.
Here's how:
Create a stored procedure that accepts your log values (method name, month/date, and execution statistics) as arguments.
In the stored procedure, before anything else, get an application lock as described here, and here. Now you can be sure only one instance of the stored procedure will be running at once. (Disclaimer! I have not tried sp_getapplock myself. Just saying. But it seems fairly straightforward, given all the examples out there on the interwebs.)
Next, in the stored procedure, query the log table for a current-month's entry for the method to determine whether to insert or update, and then do the insert or update.
As you may know, in VS you can drag stored procedures from the Server Explorer into the DBML designer for easy access with LINQ to SQL.
If you're trying to avoid stored procedures then this solution obviously won't be for you, but it's how I'd solve it easily and quickly. Hope it helps!
If you don't want to use the stored procedure approach, a crude way of dealing with it would simply be retrying on that specific exception. E.g:
int maxRetryCount = 5;
for (int i = 0; i < maxRetryCount; i++)
{
try
{
QueryAndUpdateDB();
break;
}
catch(RowUpdateException ex)
{
if (i == maxRetryCount) throw;
}
}
I have not used the sp_getapplock, instead I have used HOLDLOCK and ROWLOCK as seen below:
CREATE PROCEDURE [dbo].[UpdateStatistics]
#MethodName as varchar(50) = null,
#CountryID as varchar(2) = null,
#TokenType as varchar(5) = null,
#Year as int,
#Month as int,
#Count bigint,
#TotalTimeValue bigint
AS
BEGIN
SET NOCOUNT ON;
BEGIN TRAN
UPDATE dbo.[Statistics]
WITH (HOLDLOCK, ROWLOCK)
SET Count = Count + #Count
WHERE MethodName=#MethodName and CountryID=#CountryID and TokenType=#TokenType and Year=#Year and Month=#Month
IF ##ROWCOUNT=0
INSERT INTO dbo.[Statistics] (MethodName, CountryID, TokenType, TotalTimeValue, Year, Month, Count) values (#MethodName, #CountryID, #TokenType, #TotalTimeValue, #Year, #Month, #Count)
COMMIT TRAN
END
GO
I have tested it by calling my web service methods by multiple threads simultaneous and each call is logged without any problems.