I'm having trouble reading a Model from db using ActiveJDBC.
Basically, i'm doing ModelClass.findFirst(...) and a SQLException pops up. There seem to be no problems if the timestamp field is non-zero.
Said Model has a mysql timestamp (tried also datetime) field with zero / null value in it. This example is a timestamp with value '0000-00-00 00:00:00'. Model object is populated without errors if i update the value to a real date/time.
Caused by: java.sql.SQLException: Value '10151payment100.0012002017-01-16 02:06:530000-00-00 00:00:002017-01-16 03:36:43noFirst Last
+358 40 123456b989e4dce9e639eaadbed3b64e2c3eb' can not be represented as java.sql.Timestamp
My question is - i need to store an arbitrary date+time in this column, and for convenience reasons it should be null or zero if a value has not been stored. Is this a possible combination with ActiveJDBC?
EDIT: A minimal example to reproduce below.
Table
CREATE TABLE test (
id int(10) unsigned NOT NULL AUTO_INCREMENT,
ts timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
insert into test (id) values (1);
Model source
import org.javalite.activejdbc.Model;
import org.javalite.activejdbc.annotations.Table;
#Table("test")
public class Test extends Model {
}
Main class
import com.zaxxer.hikari.HikariDataSource;
import javax.sql.DataSource;
import org.javalite.activejdbc.Base;
public class ActiveJDBCTest {
private HikariDataSource dbds;
public ActiveJDBCTest() {
try {
init();
} catch (Exception e) {
System.out.println(e);
}
}
public void init() throws ClassNotFoundException {
Class.forName("com.mysql.jdbc.Driver");
dbds = new HikariDataSource();
dbds.setJdbcUrl("jdbc:mysql://localhost:3306/test" +
"?useSSL=false" +
"&characterEncoding=UTF-8");
dbds.setUsername("test");
dbds.setPassword("test");
dbds.setConnectionInitSql("SET ##session.time_zone = '+00:00'");
Base.open(dbds);
Test first = Test.findFirst("id = ?", 1);
Base.close();
}
public static void main(String[] args) {
new ActiveJDBCTest();
}
}
Result (please see embedded image for actual exception message - null characters?)
org.javalite.activejdbc.DBException: java.sql.SQLException: Value '10000-00-00 00:00:00' can not be represented as java.sql.Timestamp
Result
Without a full stack trace and DDL, I can only see that you have the following value in your column:
10151payment100.0012002017-01-16 02:06:530000-00-00 00:00:002017-01-16 03:36:43noFirst Last
+358 40 123456b989e4dce9e639eaadbed3b64e2c3eb
The MySQL driver is telling you that it can not be represented as java.sql.Timestamp. I think you some sort of data corruption issue. I do not think it is related to ActiveJDBC.
Ohh hell, i'm sorry for wasting your time. I apparently figured it out.
The underlying db has at some point changed from mysql to mariadb - it's some driver incompatibility while using mysql connector/j with mariadb. Using mariadb driver makes the problem go away.
Related
I'm having trouble with concurrency checks using EF6 and MySQL.
The problem I'm having is that I get a concurrency exception thrown when I try to save data to the database. If you examine the sql that is output to the console it tries to query the concurrency field from the database using the old value in the where clause. Because this field has been updated by the database.
Environment:
Windows 7 64 bit
Visual Studio 2013
Nuget packages installed:
EF 6.0.1
MySql.ConnectorNET.Data 6.8.3.2
MySql.ConnectorNET.Entity 6.8.3.2
Demo Database SQL:
DROP DATABASE IF EXISTS `bugreport`;
CREATE DATABASE IF NOT EXISTS `bugreport`;
USE `bugreport`;
DROP TABLE IF EXISTS `test`;
CREATE TABLE IF NOT EXISTS `test` (
`TestId` int(10) NOT NULL AUTO_INCREMENT,
`AStringField` varchar(50) DEFAULT NULL,
`DateModified` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`TestId`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
INSERT INTO `test` (`TestId`, `AStringField`, `DateModified`) VALUES
(1, 'Initial Value', '2014-07-11 09:15:52');
Demo code:
using System;
using System.Data.Entity.Infrastructure;
using System.Linq;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
namespace BugReport
{
class Program
{
static void Main(string[] args)
{
using (var context = new BugReportModel())
{
context.Database.Log = (s => Console.WriteLine(s));
var firstTest = context.tests.First();
firstTest.AStringField = "First Value";
// Exception is thrown when changes are saved.
context.SaveChanges();
Console.ReadLine();
}
}
}
public class BugReportModel : DbContext
{
public BugReportModel()
: base("name=Model1")
{
}
public virtual DbSet<test> tests { get; set; }
}
[Table("test")]
public class test
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int TestId { get; set; }
[StringLength(50)]
public string AStringField { get; set; }
[ConcurrencyCheck()]
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[Column(TypeName = "timestamp")]
public System.DateTime DateModified { get; set; }
}
}
Update:
Filed bug with MySql.
You should be trying to use the DB Timestamp / Rowversion feature.
In EF you declare a ByteArray and nominate it as the Concurrency check field.
DB sets the value on creation. All subsequent updates can check the value hasnt changed
DB updates rowversion as appropriate. This approach works on SQL server.
It should behave the same way on MYSql.
public abstract class BaseObject {
[Key]
[Required]
public virtual int Id { set; get; }
[ConcurrencyCheck()]
public virtual byte[] RowVersion { get; set; }
}
or via fluent if you like
// Primary Key
this.HasKey(t => t.Id);
// Properties
//Id is an int allocated by DB , with string keys, no db generation now
this.Property(t => t.Id).HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity); // default to db generated
this.Property(t => t.RowVersion)
.IsRequired()
.IsFixedLength()
.HasMaxLength(8)
.IsRowVersion(); //<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Docu on the optimistic concurrency pattern
Workarround with Interceptor
I see that MySQL Connector bug pointed in question do not have a fix yet (since 2014) and I wrote that "solution" (I know this is ugly) until the they fix it.
I create an DBCommandInterceptor and override the ReaderExecuting to replace the equal operator (=) in last WHERE to a not equal operator (<>) because the pattern for the update is something like "UPDATE ...; SELECT ... WHERE (row_version_field = #parameter)"
In the code bellow replace row_version in regular expression with the name of your row version field.
public class ConcurrencyFixInterceptor : DbCommandInterceptor
{
private static Regex concurrencyPattern =
new Regex(#"^UPDATE[\S\s]+SELECT[\S\s]+\(.?row_version.?\s(=)\s#[\w\d]+\)$",
RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.Multiline);
public override void ReaderExecuting(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
Match macth = concurrencyPattern.Match(command.CommandText);
if (macth.Success)
{
command.CommandText =
command.CommandText.
Remove(macth.Groups[1].Index, 1).
Insert(macth.Groups[1].Index, "<>");
}
base.ReaderExecuting(command, interceptionContext);
}
}
I use a row version in MySQL with a TIMESTAMP(5) field type.
I have just submitted a PR to MySQL .NET Connector v6.9.10 that provides a workaround solution for this issue.
The workaround avoids use of TIMESTAMP or DATETIME values to perform optimistic locking using a safer BIGINT RowVersion value that is incremented via a BEFORE UPDATE trigger. This fix will now support optimistic locking with an external (non-EF) application. If I can fix a 2nd bug related to TIMESTAMP / DATETIME then ConcurrencyCheck should work with these types as well.
EF6:
public class MyTable
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public virtual int Id { get; set; }
[Required, MaxLength(45)]
public virtual string Name { get; set; }
[ConcurrencyCheck, DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[Column(TypeName = "bigint")]
public virtual long RowVersion { get; set; }
}
SQL:
CREATE TABLE IF NOT EXISTS `mytable` (
Id int(11) NOT NULL,
Name varchar(45) NOT NULL,
RowVersion bigint NOT NULL DEFAULT 0,
PRIMARY KEY (`Id`)
) ENGINE=InnoDB
CREATE TRIGGER `trg_mytable_before_update`
BEFORE UPDATE ON `mytable`
FOR EACH ROW SET NEW.RowVersion = OLD.RowVersion + 1;
TIMESTAMP Solution?
I'm also investigating how to performing optimistic locking with a TIMESTAMP field.
Firstly, you need to use a more fine grained timestamp value.
So for example if you use the following, your timestamp value will be truncated to the nearest second (not very safe for optimistic locking).
UpdatedAt TIMESTAMP NOT NULL DEFAULT CURRENT_TIME ON UPDATE CURRENT_TIME
Instead you should use following to record microsecond precision.
UpdatedAt TIMESTAMP(6) NOT NULL DEFAULT NOW(6) ON UPDATE NOW(6)
Secondly, I'm observing a bug that I'm reproducing within the environment of the MySQL .NET Connector unit test suite combined with the PR patch I've just submitted. EF6 now generates the correct optimistic locking SQL to perform an UPDATE followed by the SELECT (now fixed) that returns the updated TIMESTAMP field. However the MySQL connector returns a zero TIMESTAMP (0000-00-00 00:00:00.000000) even though executing the exact same UPDATE and SELECT in MySQL Workbench it returns a valid non-zero TIMESTAMP value. I've observed the packets read via the connection socket return the string '0000-00-00 00:00:00.000000' so its probably related to the MySQL session configuration in some way. Hints welcome! I'm currently testing this with MySQL v5.6.26 (Windows).
Multiple optimistic lock fields
In our case, we have a legacy MS-Access app that uses a TIMESTAMP in most tables to perform optimistic locking. This is a convenient solution for MS-Access as it detects the presence of any TIMESTAMP column and automatically applies optimistic locking to this column when it finds one.
Since we currently don't have optimistic locking working with EF6 for TIMESTAMP columns we've added a second optimistic lock column on each table we care about by creating a BIGINT RowVersion column as that is incremented via a BEFORE INSERT trigger. So now for each UPDATE both the existing TIMESTAMP column and the new RowVersion column are update so either can be used to detect a change. Not ideal but it works!
I am using Struts2, Spring and Hibernate in my application and database is MySQL 5.5. I have this table in database:
create table if not exists campaigns(
id int(10) not null auto_increment,
campaignId int(25) not null unique,
createdBy int(25) not null REFERENCES users(userId),
campaignName varchar(255) not null,
subject varchar(500),
body varchar(50000),
modifiedOn TIMESTAMP,
triggeredOn date,
numberOfTargets int(10),
primary key (id, campaignId)
);
And I save and update the "Campaign" objects with the following methods (hibernate-mapping through hbm files) :
public boolean addCampaign(long createdBy, String campaignName) throws NoSuchAlgorithmException {
Campaign campaignObject = new Campaign();
SecureRandom generatedHash = SecureRandom.getInstance("SHA1PRNG");
campaignObject.setCampaignId(new Integer(generatedHash.nextInt()));
campaignObject.setCreatedBy(createdBy);
campaignObject.setCampaignName(campaignName);
getHibernateTemplate().save(campaignObject);
getSession().flush();
return true;
}
public Date updateCampaign(String campaignId, String subject, String body) throws NumberFormatException {
Campaign campaign = getCampaignByCampaignId(Long.parseLong(campaignId));
if(campaign != null) {
campaign.setSubject(subject);
campaign.setBody(body);
getHibernateTemplate().save(campaign);
getSession().flush();
return campaign.getModifiedOn();
}
return null;
}
The "modifiedOn" column updates when I run a update query on database. But hibernate is failing to update it. Thanks for your time.
First of all, save is for inserting a new entity. You should not use it when updating an attached entity. An attached entity's state is automatically written in the database (if changed) at flush time. You don't need to call anything to have the state updated.
And Hibernate won't magically re-read the row it has inserted/updated to get the generated timestamp. A specific #Generated annotation is needed to do that. But it will decrease the performance of the application.
I would use a pre-insert/pre-update hook to set the modifiedOn value programmatically in the entity, and avoid auto-modified timestamps in the database.
In addition to JB Nizet's response, if modifiedOn is being updated by a trigger, take a look at org.hibernate.Session#refresh().
In case the field is updated by a trigger, when hibernate saves the entity it has the old date, the trigger udpates the record at DB level (not the hibernate entity), and then when the Session closes, at commit time, Hibernate sees the entity as dirty because the modifiedOn field has a different value in DB. So, another update is launched and it is as if the trigger never updated the field. refresh() will update the entity's state with the one from the DB after the update, and the trigger execution, so they can be in synch at commit time.
public Date updateCampaign(String campaignId, String subject, String body)
throws NumberFormatException {
Campaign campaign = getCampaignByCampaignId(Long.parseLong(campaignId));
if(campaign != null) {
campaign.setSubject(subject);
campaign.setBody(body);
getHibernateTemplate().update(campaign);
getSession().flush();
getSession.refresh(campaign);
return campaign.getModifiedOn();
}
return null;
}
Not sure if I should raise an issue regarding this, so thought I would ask if anybody knew a simple workaround for this first. I am getting an error when I try to use Dapper with OleDbConnection when used in combination with MS Access 2003 (Jet.4.0) (not my choice of database!)
When running the test code below I get an exception 'OleDbException : Data type mismatch in criteria expression'
var count = 0;
using (var conn = new OleDbConnection(connString)) {
conn.Open();
var qry = conn.Query<TestTable>("select * from testtable where CreatedOn <= #CreatedOn;", new { CreatedOn = DateTime.Now });
count = qry.Count();
}
I believe from experience in the past with OleDb dates, is that when setting the DbType to Date, it then changes internally the value for OleDbType property to OleDbTimeStamp instead of OleDbType.Date. I understand this is not because of Dapper, but what 'could' be considered a strange way of linking internally in the OleDbParameter class
When dealing with this either using other ORMs, raw ADO or my own factory objects, I would clean up the command object just prior to running the command and change the OleDbType to Date.
This is not possible with Dapper as far as I can see as the command object appears to be internal. Unfortunately I have not had time to learn the dynamic generation stuff, so I could be missing something simple or I might suggest a fix and contribute rather than simply raise an issue.
Any thoughts?
Lee
It's an old thread but I had the same problem: Access doesn't like DateTime with milliseconds, so you have to add and extension method like this :
public static DateTime Floor(this DateTime date, TimeSpan span)
{
long ticks = date.Ticks / span.Ticks;
return new DateTime(ticks * span.Ticks, date.Kind);
}
And use it when passing parameters:
var qry = conn.Query<TestTable>("select * from testtable where CreatedOn <= #CreatedOn;", new { CreatedOn = DateTime.Now.Floor(TimeSpan.FromSeconds(1)) });
Unfortunately, with current Dapper version (1.42), we cannot add custom TypeHandler for base types (see #206).
If you can modify Dapper (use the cs file and not the DLL) merge this pull request and then you do not have to use Floor on each parameters :
public class DateTimeTypeHandler : SqlMapper.TypeHandler<DateTime>
{
public override DateTime Parse(object value)
{
if (value == null || value is DBNull)
{
return default(DateTime);
}
return (DateTime)value;
}
public override void SetValue(IDbDataParameter parameter, DateTime value)
{
parameter.DbType = DbType.DateTime;
parameter.Value = value.Floor(TimeSpan.FromSeconds(1));
}
}
SqlMapper.AddTypeHandler<DateTime>(new DateTimeTypeHandler());
I have MySql table that has a date field with zeroes ("0000-00-00") as its default value (field cannot be null, I can't change table structure). Hibernate doesn't like zero dates and throws exception during read or save.
I managed to make it read records by setting MySql connection setting "zeroDateTimeBehavior=convertToNull" that converts zero dates to nulls while retrieving records. It is all working fine until I try to save the record that has null date - it throws exception that date cannot be null.
So the question is - how to save record through Hibernate so date will appear as zeroes in a table?
Thanks.
I'd try to add an Hibernate Interceptor (API, Doc) and try to implement something in the onSave() method.
The following code may work:
static final Date ZERO_DATE = //0000-00-00
public boolean onSave(Object entity,
Serializable id,
Object[] state,
String[] propertyNames,
Type[] types)
throws CallbackException {
for(int i = 0; i< propertyNames.length; i++) {
if(propertyNames[i].equals("dateFieldName") && state[i]==null) {
state[i] = ZERO_DATE;
return; //or may continue, if there are several such fields.
}
}
}
Ready and working solution for DATE '0000-00-00' and TIME '00:00:00': How to map MySQL DATE '0000-00-00' & TIME '00:00:00' with Hibernate
Thanks Preston for the code and ChssPly76 for useful comments.
I have the following JPA SqlResultSetMapping:
#SqlResultSetMappings({
#SqlResultSetMapping(name="GroupParticipantDTO",
columns={
#ColumnResult(name="gpId"),
#ColumnResult(name="gpRole"),
// #ColumnResult(name="gpRemarks")
}
)
Which is used like this:
StringBuilder sbQuery = new StringBuilder("Select ");
sbQuery.append(" gpId, ");
sbQuery.append(" gpRole, ");
// sbQuery.append(" gpRemarks ");
sbQuery.append(" FROM v_group_participants_with_details ");
Query query = em.createNativeQuery(sbQuery.toString(), "GroupParticipantDTO");
The view is like this:
DROP VIEW IF EXISTS `v_group_participants_with_details`;
CREATE VIEW `v_group_participants_with_details`
AS
SELECT
gp.id AS gpId,
gp.role AS gpRole,
gp.remarks AS gpRemarks
FROM GroupParticipation gp
;
The GroupParticipation table has the remarks column defined as LONGTEXT (I'm using Mysql 5.x)
Now for the problem:
When the remarks field is commented out from the query everything works perfectly, but if I try to include the remarks field in the query, I get the following error:
javax.persistence.PersistenceException: org.hibernate.MappingException:
No Dialect mapping for JDBC type: -1
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException
(AbstractEntityManagerImpl.java:614)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:76)
What gives? How can I get a LONGTEXT column from a native query?
This problem is reported in HHH-1483 and HHH-3892. In short, Hibernate does not know, how to map a LONGVARCHAR column returned by a native query.
This problem is fixed in Hibernate 3.5.0+. For previous versions, a workaround would be to extend the MysqlDialect to register the correct Hibernate Type for a LONGVARCHAR:
import java.sql.Types;
import org.hibernate.Hibernate;
public class MyMySQL5Dialect extends org.hibernate.dialect.MySQL5Dialect {
public MyMySQL5Dialect() {
super();
// register additional hibernate types for default use in scalar sqlquery type auto detection
registerHibernateType(Types.LONGVARCHAR, Hibernate.TEXT.getName());
}
}