DbUpdateConcurrencyException using Entity Framework 6 with MySql - mysql

I'm having trouble with concurrency checks using EF6 and MySQL.
The problem I'm having is that I get a concurrency exception thrown when I try to save data to the database. If you examine the sql that is output to the console it tries to query the concurrency field from the database using the old value in the where clause. Because this field has been updated by the database.
Environment:
Windows 7 64 bit
Visual Studio 2013
Nuget packages installed:
EF 6.0.1
MySql.ConnectorNET.Data 6.8.3.2
MySql.ConnectorNET.Entity 6.8.3.2
Demo Database SQL:
DROP DATABASE IF EXISTS `bugreport`;
CREATE DATABASE IF NOT EXISTS `bugreport`;
USE `bugreport`;
DROP TABLE IF EXISTS `test`;
CREATE TABLE IF NOT EXISTS `test` (
`TestId` int(10) NOT NULL AUTO_INCREMENT,
`AStringField` varchar(50) DEFAULT NULL,
`DateModified` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`TestId`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
INSERT INTO `test` (`TestId`, `AStringField`, `DateModified`) VALUES
(1, 'Initial Value', '2014-07-11 09:15:52');
Demo code:
using System;
using System.Data.Entity.Infrastructure;
using System.Linq;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
namespace BugReport
{
class Program
{
static void Main(string[] args)
{
using (var context = new BugReportModel())
{
context.Database.Log = (s => Console.WriteLine(s));
var firstTest = context.tests.First();
firstTest.AStringField = "First Value";
// Exception is thrown when changes are saved.
context.SaveChanges();
Console.ReadLine();
}
}
}
public class BugReportModel : DbContext
{
public BugReportModel()
: base("name=Model1")
{
}
public virtual DbSet<test> tests { get; set; }
}
[Table("test")]
public class test
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int TestId { get; set; }
[StringLength(50)]
public string AStringField { get; set; }
[ConcurrencyCheck()]
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[Column(TypeName = "timestamp")]
public System.DateTime DateModified { get; set; }
}
}
Update:
Filed bug with MySql.

You should be trying to use the DB Timestamp / Rowversion feature.
In EF you declare a ByteArray and nominate it as the Concurrency check field.
DB sets the value on creation. All subsequent updates can check the value hasnt changed
DB updates rowversion as appropriate. This approach works on SQL server.
It should behave the same way on MYSql.
public abstract class BaseObject {
[Key]
[Required]
public virtual int Id { set; get; }
[ConcurrencyCheck()]
public virtual byte[] RowVersion { get; set; }
}
or via fluent if you like
// Primary Key
this.HasKey(t => t.Id);
// Properties
//Id is an int allocated by DB , with string keys, no db generation now
this.Property(t => t.Id).HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity); // default to db generated
this.Property(t => t.RowVersion)
.IsRequired()
.IsFixedLength()
.HasMaxLength(8)
.IsRowVersion(); //<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Docu on the optimistic concurrency pattern

Workarround with Interceptor
I see that MySQL Connector bug pointed in question do not have a fix yet (since 2014) and I wrote that "solution" (I know this is ugly) until the they fix it.
I create an DBCommandInterceptor and override the ReaderExecuting to replace the equal operator (=) in last WHERE to a not equal operator (<>) because the pattern for the update is something like "UPDATE ...; SELECT ... WHERE (row_version_field = #parameter)"
In the code bellow replace row_version in regular expression with the name of your row version field.
public class ConcurrencyFixInterceptor : DbCommandInterceptor
{
private static Regex concurrencyPattern =
new Regex(#"^UPDATE[\S\s]+SELECT[\S\s]+\(.?row_version.?\s(=)\s#[\w\d]+\)$",
RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.Multiline);
public override void ReaderExecuting(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
Match macth = concurrencyPattern.Match(command.CommandText);
if (macth.Success)
{
command.CommandText =
command.CommandText.
Remove(macth.Groups[1].Index, 1).
Insert(macth.Groups[1].Index, "<>");
}
base.ReaderExecuting(command, interceptionContext);
}
}
I use a row version in MySQL with a TIMESTAMP(5) field type.

I have just submitted a PR to MySQL .NET Connector v6.9.10 that provides a workaround solution for this issue.
The workaround avoids use of TIMESTAMP or DATETIME values to perform optimistic locking using a safer BIGINT RowVersion value that is incremented via a BEFORE UPDATE trigger. This fix will now support optimistic locking with an external (non-EF) application. If I can fix a 2nd bug related to TIMESTAMP / DATETIME then ConcurrencyCheck should work with these types as well.
EF6:
public class MyTable
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public virtual int Id { get; set; }
[Required, MaxLength(45)]
public virtual string Name { get; set; }
[ConcurrencyCheck, DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[Column(TypeName = "bigint")]
public virtual long RowVersion { get; set; }
}
SQL:
CREATE TABLE IF NOT EXISTS `mytable` (
Id int(11) NOT NULL,
Name varchar(45) NOT NULL,
RowVersion bigint NOT NULL DEFAULT 0,
PRIMARY KEY (`Id`)
) ENGINE=InnoDB
CREATE TRIGGER `trg_mytable_before_update`
BEFORE UPDATE ON `mytable`
FOR EACH ROW SET NEW.RowVersion = OLD.RowVersion + 1;
TIMESTAMP Solution?
I'm also investigating how to performing optimistic locking with a TIMESTAMP field.
Firstly, you need to use a more fine grained timestamp value.
So for example if you use the following, your timestamp value will be truncated to the nearest second (not very safe for optimistic locking).
UpdatedAt TIMESTAMP NOT NULL DEFAULT CURRENT_TIME ON UPDATE CURRENT_TIME
Instead you should use following to record microsecond precision.
UpdatedAt TIMESTAMP(6) NOT NULL DEFAULT NOW(6) ON UPDATE NOW(6)
Secondly, I'm observing a bug that I'm reproducing within the environment of the MySQL .NET Connector unit test suite combined with the PR patch I've just submitted. EF6 now generates the correct optimistic locking SQL to perform an UPDATE followed by the SELECT (now fixed) that returns the updated TIMESTAMP field. However the MySQL connector returns a zero TIMESTAMP (0000-00-00 00:00:00.000000) even though executing the exact same UPDATE and SELECT in MySQL Workbench it returns a valid non-zero TIMESTAMP value. I've observed the packets read via the connection socket return the string '0000-00-00 00:00:00.000000' so its probably related to the MySQL session configuration in some way. Hints welcome! I'm currently testing this with MySQL v5.6.26 (Windows).
Multiple optimistic lock fields
In our case, we have a legacy MS-Access app that uses a TIMESTAMP in most tables to perform optimistic locking. This is a convenient solution for MS-Access as it detects the presence of any TIMESTAMP column and automatically applies optimistic locking to this column when it finds one.
Since we currently don't have optimistic locking working with EF6 for TIMESTAMP columns we've added a second optimistic lock column on each table we care about by creating a BIGINT RowVersion column as that is incremented via a BEFORE INSERT trigger. So now for each UPDATE both the existing TIMESTAMP column and the new RowVersion column are update so either can be used to detect a change. Not ideal but it works!

Related

Query from database with date range in .Net Core

I receive a MySql database and one table inside it have a Date column in string format, now I need to build a .Net core server with Pomelo and EF Core and requirement is my server can query data from that table in a range of date, but because Date column of that table is in string format so I don't know how to query it, please help.
Thank you!
You are going to have to get that string into a date in order to query it.
I would probably add a new datetime column to the table and then create a simple console app that reads in each string date, try to parse this as a datetime and save it to the new datetime column.
Then you should see how many rows have valid datetimes and correct the others
Finally, you can then query using Entity Framework
how to convert a string to date in mysql?
As was told here
You can Query string to date
SELECT STR_TO_DATE(yourdatefield, '%m/%d/%Y')
FROM yourtable
With database schema change
If you can (i.e. are allowed) to change the schema of the table in question, then just add a new datetime or date column, copy the data over from the old column to the new one, and drop the column:
ALTER TABLE `YourTable` ADD COLUMN `NewDateColumn` date NOT NULL;
UPDATE `YourTable` SET `NewDateColumn` = STR_TO_DATE(`OldDateColumn`,'%Y-%m-%d');
ALTER TABLE `YourTable` DROP COLUMN `OldDateColumn`;
You can run these statements just using MySQLWorkbench or the commmand line tool. Of course you first test them with a local copy, to see that everything works fine.
With value converter
If you cannot change the schema of the table, then you can still query date ranges from the database, as long as the date strings in the database are in a string format, that sorts alphabetically (e.g. YYYY-MM-DD). In that case, you can just use a value converter in your actual app code and don't need to alter the database at all:
public class SomeModel
{
public int SomeModelId {get; set;}
public DateTime YourDateProperty {get; set;} // <-- the type you want to use in .NET
}
public class Context : DbContext
{
public virtual DbSet<SomeModel> SomeModels { get; set; }
// ...
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<SomeModel>(
entity =>
{
entity.Property(e => e.YourDateProperty)
.HasColumnType("varchar(255)") // <-- the type it has in the database table
.HasConversion(
v => v.ToString(#"yyyy\-MM\-dd"),
v => DateTime.Parse(v, CultureInfo.InvariantCulture));
});
}
}
// Here is how a sample query in your app would look like:
var query = context.SomeModels
.Where(m => m.YourDateProperty >= new DateTime(2020, 9, 1) &&
m.YourDateProperty < new DateTime(2020, 9, 10))
.ToList();

ActiveJDBC, MySQL and null timestamp producing SQLException

I'm having trouble reading a Model from db using ActiveJDBC.
Basically, i'm doing ModelClass.findFirst(...) and a SQLException pops up. There seem to be no problems if the timestamp field is non-zero.
Said Model has a mysql timestamp (tried also datetime) field with zero / null value in it. This example is a timestamp with value '0000-00-00 00:00:00'. Model object is populated without errors if i update the value to a real date/time.
Caused by: java.sql.SQLException: Value '10151payment100.0012002017-01-16 02:06:530000-00-00 00:00:002017-01-16 03:36:43noFirst Last
+358 40 123456b989e4dce9e639eaadbed3b64e2c3eb' can not be represented as java.sql.Timestamp
My question is - i need to store an arbitrary date+time in this column, and for convenience reasons it should be null or zero if a value has not been stored. Is this a possible combination with ActiveJDBC?
EDIT: A minimal example to reproduce below.
Table
CREATE TABLE test (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
ts timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
insert into test (id) values (1);
Model source
import org.javalite.activejdbc.Model;
import org.javalite.activejdbc.annotations.Table;
#Table("test")
public class Test extends Model {
}
Main class
import com.zaxxer.hikari.HikariDataSource;
import javax.sql.DataSource;
import org.javalite.activejdbc.Base;
public class ActiveJDBCTest {
private HikariDataSource dbds;
public ActiveJDBCTest() {
try {
init();
} catch (Exception e) {
System.out.println(e);
}
}
public void init() throws ClassNotFoundException {
Class.forName("com.mysql.jdbc.Driver");
dbds = new HikariDataSource();
dbds.setJdbcUrl("jdbc:mysql://localhost:3306/test" +
"?useSSL=false" +
"&characterEncoding=UTF-8");
dbds.setUsername("test");
dbds.setPassword("test");
dbds.setConnectionInitSql("SET ##session.time_zone = '+00:00'");
Base.open(dbds);
Test first = Test.findFirst("id = ?", 1);
Base.close();
}
public static void main(String[] args) {
new ActiveJDBCTest();
}
}
Result (please see embedded image for actual exception message - null characters?)
org.javalite.activejdbc.DBException: java.sql.SQLException: Value '10000-00-00 00:00:00' can not be represented as java.sql.Timestamp
Result
Without a full stack trace and DDL, I can only see that you have the following value in your column:
10151payment100.0012002017-01-16 02:06:530000-00-00 00:00:002017-01-16 03:36:43noFirst Last
+358 40 123456b989e4dce9e639eaadbed3b64e2c3eb
The MySQL driver is telling you that it can not be represented as java.sql.Timestamp. I think you some sort of data corruption issue. I do not think it is related to ActiveJDBC.
Ohh hell, i'm sorry for wasting your time. I apparently figured it out.
The underlying db has at some point changed from mysql to mariadb - it's some driver incompatibility while using mysql connector/j with mariadb. Using mariadb driver makes the problem go away.

ASP.NET - Invalid object name 'dbo.ForumSections'

I am trying to read data from my database. I have created a table called ForumSections, here is its table definition.
CREATE TABLE [dbo].[ForumSection] (
[Id] INT IDENTITY (0, 1) NOT NULL,
[SectionName] NVARCHAR (200) NOT NULL,
[TopicsinSection] INT NOT NULL,
[SectionDescription] NVARCHAR (MAX) NOT NULL,
PRIMARY KEY CLUSTERED ([Id] ASC)
);
When I try to call from the database I get a EntityCommandExecutionException with a InnerException message of Invalid object name 'dbo.ForumSections'.
Here is the code used to call the database.
First setting up the database context:
public class MainDbContext : DbContext
{
public MainDbContext()
: base("name=DefaultConnection")
{
}
public DbSet<ForumSection> ForumSection { get; set; }
}
Then calling it:
var db = new MainDbContext();
db.ForumSection.Find(0); //Error happens on this line
The row with index of 0 is appropriately filled out.
What are my options for resolving this?
By default, EntityFramework tries to look for the table pluralizing the name of the class. That's why it is adding the 's' to the ForumSection.
You can disable that option. This will help you solve your problem:
Why does EF 5.x use plural name for table?

Enable hbm2ddl.keywords=auto-quote in Fluent NHibernate

I have made a tiny software tool that allows me to display or run SQL generated from NHibernate. I made this because hbm2ddl.auto is not recommended for production.
I have one problem: when I generate the SQL I always get the infamous Index column unquoted, because I need .AsList() mappings. This prevents me to run the SQL.
In theory, if I had an XML configuration of NHibernate I could use hbm2ddl.keywords tag, but unfortunately since my tool is designed as a DBA-supporting tool for multiple environments, I must use a programmatic approach.
My approach (redundant) is the following:
private static Configuration BuildNHConfig(string connectionString, DbType dbType, out Dialect requiredDialect)
{
IPersistenceConfigurer persistenceConfigurer;
switch (dbType)
{
case DbType.MySQL:
{
persistenceConfigurer =
MySQLConfiguration
.Standard
.Dialect<MySQL5Dialect>()
.Driver<MySqlDataDriver>()
.FormatSql()
.ShowSql()
.ConnectionString(connectionString);
requiredDialect = new MySQL5Dialect();
break;
}
case DbType.MsSqlAzure:
{
persistenceConfigurer = MsSqlConfiguration.MsSql2008
.Dialect<MsSqlAzure2008Dialect>()
.Driver<SqlClientDriver>()
.FormatSql()
.ShowSql()
.ConnectionString(connectionString);
requiredDialect = new MsSqlAzure2008Dialect();
break;
}
default:
{
throw new NotImplementedException();
}
}
FluentConfiguration fc = Fluently.Configure()
.Database(persistenceConfigurer)
.ExposeConfiguration(
cfg => cfg.SetProperty("hbm2ddl.keywords", "keywords")
.SetProperty("hbm2ddl.auto", "none"))
.Mappings(
m => m.FluentMappings.AddFromAssemblyOf<NHibernateFactory>());
Configuration ret = fc.BuildConfiguration();
SchemaMetadataUpdater.QuoteTableAndColumns(ret);
return ret;
}
...
public static void GenerateSql(MainWindowViewModel viewModel)
{
Dialect requiredDialect;
Configuration cfg = BuildNHConfig(viewModel.ConnectionString, viewModel.DbType.Value, out requiredDialect);
StringBuilder sqlBuilder = new StringBuilder();
foreach (string sqlLine in cfg.GenerateSchemaCreationScript(requiredDialect))
sqlBuilder.AppendLine(sqlLine);
viewModel.Sql = sqlBuilder.ToString();
}
Explanation: when I want to set the ViewModel's SQL to display on a TextBox (yea, this is WPF) I initialize the configuration programmatically with connection string given in ViewModel and choose the dialect/provider accordingly. When I Fluently Configure NHibernate I both set hbm2ddl.keywords (tried both auto-quote and keywords, this being the default) and, following this blog post, I also use the SchemaMetadataUpdater.
The result is that I'm always presented with SQL like
create table `OrderHistoryEvent` (Id BIGINT NOT NULL AUTO_INCREMENT, EventType VARCHAR(255) not null, EventTime DATETIME not null, EntityType VARCHAR(255), Comments VARCHAR(255), Order_id VARCHAR(255), Index INTEGER, primary key (Id))
where the guilty Index column is not quoted.
The question is: given a programmatic and fluent configuration of NHibernate, how do I tell NHibernate to quote any reserved word in the SQL exported by GenerateSchemaCreationScript?
I have found a workaround: when I generate the update script (the one that runs with hbm2ddl.auto=update) the script is correctly quoted.
The infamous Index column has been already discussed and from my findings it's hardcoded in FNH (ToManyBase.cs, method public T AsList()).
Since the update script is a perfectly working creational script on an empty database, changing the code to generate an update script on an empty DB should equal generating a creational script.
This happens only because I want to generate the script on my own. There is probably a bug in NHibernate that only activates when you call GenerateSchemaCreationScript and not when you let your SessionFactory build the DB for you

TIMESTAMP column not updating

I am using Struts2, Spring and Hibernate in my application and database is MySQL 5.5. I have this table in database:
create table if not exists campaigns(
id int(10) not null auto_increment,
campaignId int(25) not null unique,
createdBy int(25) not null REFERENCES users(userId),
campaignName varchar(255) not null,
subject varchar(500),
body varchar(50000),
modifiedOn TIMESTAMP,
triggeredOn date,
numberOfTargets int(10),
primary key (id, campaignId)
);
And I save and update the "Campaign" objects with the following methods (hibernate-mapping through hbm files) :
public boolean addCampaign(long createdBy, String campaignName) throws NoSuchAlgorithmException {
Campaign campaignObject = new Campaign();
SecureRandom generatedHash = SecureRandom.getInstance("SHA1PRNG");
campaignObject.setCampaignId(new Integer(generatedHash.nextInt()));
campaignObject.setCreatedBy(createdBy);
campaignObject.setCampaignName(campaignName);
getHibernateTemplate().save(campaignObject);
getSession().flush();
return true;
}
public Date updateCampaign(String campaignId, String subject, String body) throws NumberFormatException {
Campaign campaign = getCampaignByCampaignId(Long.parseLong(campaignId));
if(campaign != null) {
campaign.setSubject(subject);
campaign.setBody(body);
getHibernateTemplate().save(campaign);
getSession().flush();
return campaign.getModifiedOn();
}
return null;
}
The "modifiedOn" column updates when I run a update query on database. But hibernate is failing to update it. Thanks for your time.
First of all, save is for inserting a new entity. You should not use it when updating an attached entity. An attached entity's state is automatically written in the database (if changed) at flush time. You don't need to call anything to have the state updated.
And Hibernate won't magically re-read the row it has inserted/updated to get the generated timestamp. A specific #Generated annotation is needed to do that. But it will decrease the performance of the application.
I would use a pre-insert/pre-update hook to set the modifiedOn value programmatically in the entity, and avoid auto-modified timestamps in the database.
In addition to JB Nizet's response, if modifiedOn is being updated by a trigger, take a look at org.hibernate.Session#refresh().
In case the field is updated by a trigger, when hibernate saves the entity it has the old date, the trigger udpates the record at DB level (not the hibernate entity), and then when the Session closes, at commit time, Hibernate sees the entity as dirty because the modifiedOn field has a different value in DB. So, another update is launched and it is as if the trigger never updated the field. refresh() will update the entity's state with the one from the DB after the update, and the trigger execution, so they can be in synch at commit time.
public Date updateCampaign(String campaignId, String subject, String body)
throws NumberFormatException {
Campaign campaign = getCampaignByCampaignId(Long.parseLong(campaignId));
if(campaign != null) {
campaign.setSubject(subject);
campaign.setBody(body);
getHibernateTemplate().update(campaign);
getSession().flush();
getSession.refresh(campaign);
return campaign.getModifiedOn();
}
return null;
}