I'm trying to add a long list of file into mysql and use spring ACL service with grails to attach permission.
So, in my controller i have:
Files.withTransaction {
Files file = new Files(dataStore:ds,created:new Date(),path:target,name:fileName,contentType:contentType,contentLength:contentLength,isFolder:false).save(flush:true)
file = Files.lock(file.id)
filesService.addPermission(file, username ,BasePermission.ADMINISTRATION)
}
the i have no worries with the Files domain object, it supports huge amount of data ,(specially since i 've disabled versioning in mysql), the problem is on filesService which uses aclUtilService,
#Transactional
#PreAuthorize("hasRole('ROLE_ADMIN') or hasRole('ROLE_USER') or hasRole('ROLE_GROUP_OWNER')")
def addPermission(Files f, String username,Permission permission) {
aclUtilService.addPermission f,username,permission
}
Randomly i have the following deadlock (optmistic?)error :
Deadlock found when trying to get lock; try restarting transaction.
Stacktrace follows:
com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException:
Deadlock found when trying to get lock; try restarting transaction at
com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at
com.mysql.jdbc.Util.getInstance(Util.java:381) at
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1045) at
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at
com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3558) at
com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3490) at
com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959) at
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2109) at
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2648) at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2077)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2362)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2280)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2265)
at
org.grails.plugins.springsecurity.service.acl.AclService.save(AclService.groovy:330)
at
org.grails.plugins.springsecurity.service.acl.AclService.createEntries(AclService.groovy:198)
at
org.grails.plugins.springsecurity.service.acl.AclService.updateAcl(AclService.groovy:176)
at
GrailsMelodyGrailsPlugin$_closure5_closure18_closure19.doCall(GrailsMelodyGrailsPlugin.groovy:170)
at
org.grails.plugins.springsecurity.service.acl.AclUtilService.addPermission(AclUtilService.groovy:90)
at
org.grails.plugins.springsecurity.service.acl.AclUtilService.addPermission(AclUtilService.groovy:67)
at
GrailsMelodyGrailsPlugin$_closure5_closure18_closure19.doCall(GrailsMelodyGrailsPlugin.groovy:170)
at
xxxxxxxxxxxxx.FilesService.addPermission(FilesService.groovy:34)
at
GrailsMelodyGrailsPlugin$_closure5_closure18_closure19.doCall(GrailsMelodyGrailsPlugin.groovy:170)
at
xxxxxxxxxxxxxQuantumController$_uploadToS3_closure1$$ENzPdDAW.doCall(QuantumController.groovy:87)
at
org.grails.datastore.gorm.GormStaticApi.withTransaction(GormStaticApi.groovy:686)
at
xxxxxxxxxxxxx.QuantumController$$ENzPdDAW.uploadToS3(QuantumController.groovy:84)
at
grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
at
grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:149) at
net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:259)
at
net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:202)
at
net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:175)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Any help please ?
just encapsulated the transaction in controller and use #Transactional in the service to solve the issue:
Files.withTransaction {
Files file = new Files(
...
if (file.validate()) {
file.save()
filesService.addPermission(file,username,BasePermission.ADMINISTRATION)
...}
}
Related
I have a stub that I can see under /__admin/ but when making any sort of rest calls nothing registers in either __admin/requests or __admin/requests/unmatched which I feel is strange. Is Wiremock failing to intercept the request?
Simplified code:
class MyTestClass {
#Test
void testStuff(){
WireMock.configureFor("localhost", 9050); // Running in docker with custom port
WireMock.reset();
stubFor(
get("/getStuff?include-archived=yes")
.withPort(<myServicePort>)
.willReturn(
aResponse()
.withHeader("Content-Type", "application/json")
.withStatus(200)
)
);
}
}
Error:
Error has been observed at the following site(s): *__checkpoint ⇢
Request to GET
http://localhost:<myServicePort>/getStuff?include-archived=yes
[DefaultWebClient]
...
[org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver]
: Resolved
[org.springframework.web.reactive.function.client.WebClientRequestException:
Connection refused: no further information:
localhost/127.0.0.1:<myServicePort>; nested exception is
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
refused: no further information: localhost/127.0.0.1:<myServicePort>]
Finally found the issue, the port that I was trying to intercept wasn't open in the container thus why I got no entries in either __admin/requests or __admin/requests/unmatched
I implement a Dashboard functionality that checks every time at program start a list of Requirement-Objects for a bunch of different characteristics like progress, missing data and alike and sets for each characteristic a dedicated beacon on the UI.
protected void initializePerformanceIndicator() {
try {
updateA();
updateB();
...
updateF();
updateG();
} catch (Exception e) {
ErrorHandler.showError("Cannot show KPI Performance", e);
}
}
The checks have different compute demands some are faster some slower, therefore each of this checks runs in a dedicated Task to provide some feedback to the user. The skeleton of such a Task is always the same
protected void updateA() throws Exception {
Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
embeddedBudgetKPIController.setHyperlink("Budget", null);
embeddedBudgetKPIController.setToolTip("...");
ObservableList<UserRequirement> issues = FXCollections.observableArrayList();
List<UserRequirement> requirements = reqService.getAllUserRequirements(false); // all requirements of the selected product
for(UserRequirement req: requirements) {
if(*some criteria*) {
issues.add(req);
}
}
if(issues.isEmpty()) {
embeddedBudgetKPIController.setBeaconColor(Color.GREEN);
} else {
embeddedBudgetKPIController.setBeaconColor(Color.RED);
}
return null;
};
};
task.setOnSucceeded(e -> {
// Nothing to do
});
Thread tt = new Thread(task);
tt.start();
}
Before initializePerformanceIndicator is called, I retrieved already elsewhere the data from the database querying a number Spring Repositories:
protected final ObservableList<UserRequirement> allUserRequirements = FXCollections.observableArrayList();
public synchronized ObservableList<UserRequirement> getAllUserRequirements(boolean forceUpdate) throws Exception {
logger.debug(""); // show that this method is called
Product selectedProduct = SelectedScope.getSelectedProduct();
if(selectedProduct == null) {
throw new Exception("No selProduct selected");
}
if(forceUpdate || allUserRequirements.isEmpty()) {
allUserRequirements.clear();
allUserRequirements.addAll(epicRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(themeRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(userStoryRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
allUserRequirements.addAll(tangibleRepository.findByProductAndRevisionSuccessorIsNull(selectedProduct));
}
return allUserRequirements;
}
and as you see updateBudgetKPIController calls getallUserRequirements with the parameter false. Therefore it returns the buffered result set and is not re-fetching data from database. So far everything is fine.
I can run each of these Tasks individually without problem. I tried a number combinations with 2 Tasks. Works fine, but the program will never show more than three or four beacons. Which ones are shown differs as well - what is expected as a consequence of the different Tasks. If I exceed three or four Tasks I often get no error at all, but the UI is just not showing more than three to four beacons.
Sometimes I do get an error message, which is
WARN 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 08:14 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
I debugged it, and realized that I was generating way too many select statements. The UserRequirement entity has almost a dozen OneToMany relations, some where defined with FetchType.LAZY, so I thought it would be better anyway to configure all these relations as
#OneToMany(fetch = FetchType.LAZY, mappedBy="parent", cascade = CascadeType.ALL)
Because of the LAZY loading, every Task tries to load additional data in the if(*some criteria*) part.
The problem did not disappear but I get more information, as the error is now
WARN 11:02 o.h.c.i.AbstractPersistentCollection.withTemporarySessionIfNeeded:278: Unable to close temporary session used to load lazy collection associated to no session
WARN 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:137: SQL Error: 0, SQLState: S1009
ERROR 11:02 o.h.e.j.s.SqlExceptionHelper.logExceptions:142: No operations allowed after statement closed.
So I do have a LAZY loading issue.
I am using Spring Boot 2.1.6, MySQL 8.0.15 Community Server, Hibernate Core {5.3.10.Final}, Java 1.8.0_211 and the com.mysql.cj.jdbc.Driver
From a former issue, I have in my properties file the following configuration
# Prevent LazyInitializationException
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
Don't know whether this has a side effect?!
Probably changing the LAZY loading to EAGER will fix it - haven't tried yet - but it would delay program start significantly. Therefore I would prefer a solution with LAZY loading.
Any ideas? I also appreciate any ideas regarding how to further isolate the root cause as the error message is not really explicit and I can't see which part of my code triggers it. Plus when I debug it, the behavior changes as I compute all Tasks sequentially rather then in parallel. Thank you in advance.
The issue was caused by different Tasks accessing the same getter of some of the entities. If the first getter call opened a connection, the second call got on it, and then the first call closed the ResultSet, the second call one was in trouble. Synchronizing the getter method solved the problem.
I just learned Hibernate's optimistic locking last week and introduced it to my application written with Spring JPA and MySQL DB.
My Entity looks like this, the Version annotation was just added,
#Entity
public class Instance {
...
#javax.persistence.Version
private Date updateTime;
...
pubic Instance() {
this.updateTime = new Date();
}
}
And I have a org.springframework.data.repository.CrudRepository used for persisting the data. While I verified updating existing record works fine, the problem is it'll throw exception when persisting a new object,
inst = new Inst();
instanceRepo.save(inst);
The exception thrown is
org.springframework.orm.ObjectOptimisticLockingFailureException: Object of class [Instance] with identifier [a5deddb9-d76c-433f-8b0d-e50cbf8f601e]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) :
at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:2541)
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3285)
at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3183)
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3525)
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:425)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:177)
at org.hibernate.jpa.internal.TransactionImpl.commit(TransactionImpl.java:77)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:517)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:131)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy139.save(Unknown Source)
This is simply the first persistence call and I don't think there are multiple threads trying to update this record, so I'm confused why this exception is thrown.
Also I tried to trace into the underlying code, and here is some findings might be helpful,
When calling the underlying org.springframework.data.jpa.repository.support.SimpleJpaRepository#save(S) as pasted below, the code will go to the merge logic even though this is a new record,
public <S extends T> S save(S entity) {
if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}
The exception happens when it tries to commit the transaction and flush the DB.
The JPA spec said that
The following types are supported for version properties: int, Integer, short, Short, long, Long, java.sql.Timestamp.
There may be problem when you use java.util.Date type.
#Version annotation can also be used for Date Fields if your JPA provider supports that .But You would have to map the date field with #Temporal annotation too as below
#javax.persistence.Version
#Temporal(TemporalType.TIMESTAMP)
private Date updateTime;
For more details please visit below link
http://www.byteslounge.com/tutorials/jpa-entity-versioning-version-and-optimistic-locking
In short, what I am trying solve is how to recover from certain database errors in a Grails application using Hibernate and continue on with the transaction skipping over the failed row updates that are part of a batch of changes.
The application uses Grails 2.3.11 but I have also tried with version 1.3.8 with similar failed results.
Basically there is a Grails service class that iterates over a list of imported records and attempts to update associated master records appropriately. In certain situations exceptions might occur during the domain.save(flush:true) call e.g. org.hibernate.exception.DataException thrown due to issues like (Data truncation: Data too long for column ...).
At this point I have tried:
Disabling transactions
Using domainObj.withTransaction() for each individual record
Trying various #Transactional annotations
Calling domain.clearErrors() and domain.discard() after catching the exception
Tried using a nested service with Transactional annotation with noRollbackFor as shown below
A number of other approaches but nothing I've tried has worked
Example code:
#Transactional
class UpdateService {
public updateBatch(Integer batchId) {
...
list.each { record ->
record.value = 123
try {
nestedService.saveDomain()
} catch (e) {
record.clearErrors()
record.discard()
}
}
batch.status = "POSTED"
batch.save()
}
}
#Transactional
class NestedService {
#Transactional(propagation = Propagation.REQUIRED, noRollbackFor = RuntimeException.class)
public void saveDomain(domainObj) throws RuntimeException {
if (domainObj.validate() && domainObj.save(flush:true) {
log.info "domain $domain was saved"
}
}
}
Once an error occurs I can't seem to clear out the hibernate session. On each subsequent record being updated I receive the error:
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction
where it indicates the original failed domain id.
Revision:
Vahid, Thanks for the suggestions. I have tried that. I realized one issue is that I am passing objects across transactional boundaries. So I experimented with the NestedService class do something along the lines of:
#Transactional(propagation = Propagation.REQUIRE_NEW)
public void saveDomain(domainObj) {
def newObj = new Domain.get(domainObj.id)
newObj.properties = domainObj.properties
if (newObj.validate() && newObj.save(force:true) ) { ... }
I expected that to work but the original domainObj still fails even though I'm not calling the save on it. Very strange...
A simple approach would be to loop and then use validate(). If it does fail, then just store the id of the failed entity and proceed.
if(!domainObject.validate()){
// store Id for trying it again later ?
}else{
// Save
}
I am trying to use flyway to create and manage a MySQL database. Here is the code i have got so far.
FlywayMigration.java : Class that applys the migration
public class FlywayMigration
{
public FlywayMigration(DatabaseConfiguration configuration, Flyway flyway)
{
flyway.setDataSource(configuration.getDataSource());
flyway.migrate();
}
public static void main(String[] args)
{
new FlywayMigration(new DatabaseConfiguration("database.properties"), new Flyway());
}
}
DatabaseConfiguration.java : Configuration class, this class will configure the datasource to be applyed to the Flyway.setDataSource method
public class DatabaseConfiguration
{
private final Logger LOGGER = LoggerFactory.getLogger(this.getClass());
private PropertiesUtil prop = null;
public DatabaseConfiguration(String file)
{
prop = new PropertiesUtil(file);
}
public String getDataSourceClass()
{
return prop.getProperty("mysql.data.source.class");
}
public String getURL ()
{
return prop.getProperty("mysql.url");
}
public String getHostName()
{
return prop.getProperty("mysql.host.name");
}
public String getDatabaseName()
{
return prop.getProperty("mysql.database.name");
}
public DataSource getDataSource()
{
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setURL(getURL());
dataSource.setUser(prop.getProperty("mysql.user.name"));
dataSource.setPassword(null);
return dataSource;
}
}
database.properties is the file where i store the database information, password can be null
mysql.data.source.class=com.mysql.jdbc.Driver
mysql.url=jdbc:mysql://localhost:3306/vmrDB
mysql.host.name=localhost
mysql.database.name=vmrDB
mysql.user.name=root
And i get the folowing error in my trace
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1144)
at org.flywaydb.core.Flyway.migrate(Flyway.java:811)
at com.bt.sitb.vmr.migration.FlywayMigration.<init>(FlywayMigration.java:10)
at com.bt.sitb.vmr.migration.FlywayMigration.main(FlywayMigration.java:15)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Can someone please tell me why the DataSource from MySQL is not connecting?
It looks like Flyway cannot connect to the database.
One reason for this is that the database in the database URL does not exist.
Question: does your database schema exist?
If your answer is no, then:
connect to jdbc:mysql://localhost:3306/mysql
also specify the schema to use for migration with flyway.setSchemas(configuration.getDatabaseName())
you also need flyway.init() before you can initialize migration of your database.
Ran into this same issue. Apparently, the problem was with my .properties file. The jar was using the one packaged with it and not the external one. So I moved my external properties file out of the resources folder and into the root directory of the jar and problem solved!
Hope this helps someone.
I had this same issue when working on a Java application in Debian 10 using Tomcat Application server.
I defined the connection strings for the database in the context.xml file, however, when I start out the application and try to log into the application, I get the error:
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1144)
Here's what I figured out:
I finally realized that the application was using internally defined database connection strings that were packaged with it. The internally defined database connection strings were different from my own database connection strings defined in the context.xml file.
The solution for me was to either modify the internally defined database connection strings that were packaged with the application or use the same internally defined database connection strings that were packaged with application in my context.xml file.
That's all.
I hope this helps.