I am using shedlock library 4.20.0.
net.javacrumbs.shedlock shedlock-spring 4.20.0 net.javacrumbs.shedlock shedlock-provider-jdbc-template 2.1.0
The scheduler job is,
#scheduled(fixedRate = 5000)
#SchedulerLock(name = "TaskScheduler__scheduledTask", lockAtLeastForString = "PT5M", lockAtMostForString = "PT14M")
public void reportCurrentTime() {
LockAssert.assertLocked();
log.info("The time is now {} {}", dateFormat.format(new Date()), dataSource);
}
It shows #SchedulerLock as deprecated.
And the spring boot class,
#SpringBootApplication
#EnableScheduling
#EnableSchedulerLock(defaultLockAtMostFor = "PT30S")
public class DMSCaseEmulatorSpringApplication {
public static void main(String[] args) {
SpringApplication.run(DMSCaseEmulatorSpringApplication.class, args);
}
}
When i execute the spring boot class, it triggers shedlock and creates a record in database table but in logs i keep getting as below,
19:54:39.188 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Locked TaskScheduler__scheduledTask.
19:54:39.188 [scheduling-1] INFO u.g.h.c.d.s.ScheduledTasks - The time is now 19:54:39 HikariDataSource (HikariPool-1)
19:54:39.205 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Unlocked TaskScheduler__scheduledTask.
19:54:44.065 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Not executing TaskScheduler__scheduledTask. It's locked.
19:54:49.062 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Not executing TaskScheduler__scheduledTask. It's locked.
Any thoughts will be appreciated?
The issue is caused by lockAtLeastForString = "PT5M" By specifying that, you are saying that the lock should be held at least for 5 minutes even if the task finishes sooner.
Regarding the Deprecation warning, please consult the JavaDoc.
Related
I have a Spring Batch Task running on our cloud platform that will launch with the provided command line parameters, and then skip over the execution of the first Step with the following error:
[OUT] The job execution id 992 was run within the task execution 1325
[OUT] Step already complete or not restartable, so no action to execute:
StepExecution: id=1071, version=3, name=OFileStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0,
writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
I have investigated the metadata tables in the MySQL instance that Spring Batch uses to find that the JOB_INSTANCE_ID is the same between multiple executions, when it should increment by 1 each time.
The #Bean that I have defined for the Job Configuration is:
#Bean
public Job job() {
return jobBuilderFactory.get(OTaskConstants.JOB_NAME)
.listener(listener())
.incrementer(new RunIdIncrementer())
.start(dataTransferTaskStep())
.next(controlMTaskStep())
.build();
}
Is anyone aware of what could be causing this behavior?
Below line clearly says it all.
Step already complete or not restartable, so no action to execute:
Meaning the step/job already complete and can not be restarted. This is the behavior of Spring Batch. In order to by pass this we need to pass an unique argument.
In your case i see you already have RunIdIncrementer. Now question is why it is not working.
Can you see BATCH_JOB_PARMS table to see what arguments are getting passed to the job? May be you are missing something.
You can also use SimpleIncrementor. See below code for explanation.
https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#JobParametersIncrementer
Remove #Bean annotation on Job.
It causes the Job to be launched with no parameters every time you launch/start application as spring tries to load the bean and which in-turn launches the batch job.
Remove the annotation and use spring scheduler to schedule the jobs.
I had the same issue. Below code helped me resolve it. By adding params in job launcher a new job_instance_id is created for every run.
#SpringBootApplication
public class App implements CommandLineRunner {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
#Override
public void run(String... args) throws Exception {
JobParameters params = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
jobLauncher.run(job, params);
}
}
Solution
Refer error message above “If you want to run this job again, change the parameters.” The formula is JobInstance = JobParameters + Job. If you do not have any parameters for JobParameters, just pass a current time as parameter to create a new JobInstance. For example,
CustomJobLauncher.java
//...
#Component
public class CustomJobLauncher {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
public void run() {
try {
JobParameters jobParameters =
new JobParametersBuilder()
.addLong("time",System.currentTimeMillis()).toJobParameters();
JobExecution execution = jobLauncher.run(job, jobParameters);
System.out.println("Exit Status : " + execution.getStatus());
} catch (Exception e) {
e.printStackTrace();
}
}
}
Source : https://mkyong.com/spring-batch/spring-batch-a-job-instance-already-exists-and-is-complete-for-parameters/
I have a docker-compose setup to start my SpringBoot application and a MySQL database. If the database starts first, then my application can connect successfully. But if my application starts first, no database exists yet, so the application throws the following exception and exits:
app_1 | 2018-05-27 14:15:03.415 INFO 1 --- [ main]
com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
app_1 | 2018-05-27 14:15:06.770 ERROR 1 --- [ main]
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization
app_1 | com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure
I could edit my docker-compose file to make sure the database is always up before the application starts up, but I want the application to be able to handle this case on its own, and not immediately exit when it cannot reach the database address.
There are ways to configure the datasource in the application.properties file to make the application reconnect to the database, as answered here and here. But that doesn't work for a startup connection to the datasource.
How can I make my SpringBoot application retry the connection at startup to the database at a given interval until it successfully connects to the database?
Set HikariCP's initializationFailTimeout property to 0 (zero), or a negative number. As documented here:
⌚initializationFailTimeout
This property controls whether the pool will "fail fast" if the pool cannot be seeded with an initial connection successfully. Any positive number is taken to be the number of milliseconds to attempt to acquire an initial connection; the application thread will be blocked during this period. If a connection cannot be acquired before this timeout occurs, an exception will be thrown. This timeout is applied after the connectionTimeout period. If the value is zero (0), HikariCP will attempt to obtain and validate a connection. If a connection is obtained, but fails validation, an exception will be thrown and the pool not started. However, if a connection cannot be obtained, the pool will start, but later efforts to obtain a connection may fail. A value less than zero will bypass any initial connection attempt, and the pool will start immediately while trying to obtain connections in the background. Consequently, later efforts to obtain a connection may fail. Default: 1
There is an alternative way to do this, which doesn't rely on a specific Connection Pool library or a specific database. Note that you will need to use spring-retry to achieve the desired behaviour with this approach
First you need to add spring-retry to your dependencies :
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>${spring-retry.version}</version>
</dependency>
Then you can create a decorator over DataSource that will extends AbstractDataSource like bellow :
#Slf4j
#RequiredArgsConstructor
public class RetryableDataSource extends AbstractDataSource {
private final DataSource dataSource;
#Override
#Retryable(maxAttempts = 5, backoff = #Backoff(multiplier = 1.3, maxDelay = 10000))
public Connection getConnection() throws SQLException {
log.info("getting connection ...");
return dataSource.getConnection();
}
#Override
#Retryable(maxAttempts = 5, backoff = #Backoff(multiplier = 2.3, maxDelay = 10000))
public Connection getConnection(String username, String password) throws SQLException {
log.info("getting connection by username and password ...");
return dataSource.getConnection(username, password);
}
}
Then you will need to inject this custom DataSource decorator into Spring context by creating a custom BeanPostProcessor :
#Slf4j
#Order(value = Ordered.HIGHEST_PRECEDENCE)
#Component
public class RetryableDatabasePostProcessor implements BeanPostProcessor {
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
if(bean instanceof DataSource) {
log.info("-----> configuring a retryable datasource for beanName = {}", beanName);
return new RetryableDataSource((DataSource) bean);
}
return bean;
}
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
}
Last but not least you will need to enable Spring retry by adding #EnableRetry annotation to spring main class, example :
#EnableRetry
#SpringBootApplication
public class RetryableDbConnectionApplication {
public static void main(String[] args) {
SpringApplication.run(RetryableDbConnectionApplication.class, args);
}
}
I have a listener test, where i post a message in a parallel thread and check with LatchCountDownAndCallRealMethodAnswer if the all were processed successfully. Running the test alone, it works perfectly, however if you run all other tests together, it fails because it failed to leave the counter at zero, but the listener received and processed the message normally. Does anyone have any ideas?
My Test Class
#RunWith(SpringRunner.class)
#SpringBootTest
#RabbitListenerTest
#ActiveProfiles("test")
public class EventListenerTest {
EventListener eventListener;
#Autowired
protected RabbitListenerTestHarness harness;
#Autowired
private EventStoreRepository repository;
#SpyBean
private DomainEventPublisher publisher;
#Before
public void setUp() {
MockitoAnnotations.initMocks(this);
DomainRegister.setDomainEventPublisher(publisher);
eventListener = this.harness.getSpy("eventListenerId");
}
#Test
public void storeEventsListenerTest() throws Exception {
LatchCountDownAndCallRealMethodAnswer answer = new LatchCountDownAndCallRealMethodAnswer(1);
doAnswer(answer).when(eventListener).storeEvents(any(BalanceReserved.class));
publisher.publish(new BalanceReserved("12233", 150.0, BigDecimal.ZERO), "");
assertTrue(answer.getLatch().await(10, TimeUnit.SECONDS));
verify(eventListener, times(1)).storeEvents(any(BalanceReserved.class));
}
#After
public void tearDown() {
DomainRegister.setDomainEventPublisher(null);
reset(eventListener);
repository.deleteAll();
}
}
Error
java.lang.AssertionError
If you have other tests using the same queue, you need to shut down the application context for each test so the test's listeners are stopped. By default, the Spring Test framework caches the application context for reuse. This will cause other tests to "steal" messages.
Add #DirtiesContext to each test class that uses #RabbitListeners, to tell the test framework to shutdown the context.
In my case, I'm using spring-boot with gradle and added flyway by simply putting compile 'org.flywaydb:flyway-core' to the build.gradle.
For a simulator run, which is in test, I would like to clear the database before each run. I've put a reset script in /src/test/resources/db/migration/V1.0__Reset.sql (with the real init sql-script at /src/main/resources/db/migration/V1.1__Init.sql), but receive a SyntaxException due to the reset script, which doesn't occur when I run it from the MySQL Workbench.
How can I reset or clear the database at startup?
-- UPDATE --
I've tried to use a Spring DataSourceInitializer, but it seems Flyway scripts are executed before the DS init, so it results in Hibernate Syntax error because the tables aren't found.
#Resource
DataSource ds;
#Bean
public DataSourceInitializer dbInit() throws FileNotFoundException, URISyntaxException {
public DataSourceInitializer dbInit() throws FileNotFoundException, URISyntaxException {
DataSourceInitializer re = new DataSourceInitializer();
re.setDataSource(ds);
re.setEnabled(true);
String str = "classpath:sql/V1.0__Reset.sql";
URL url = ResourceUtils.getURL(str);
org.springframework.core.io.Resource resi = new PathResource(url.toURI());
// new org.springframework.core.io.ClassPathResource(str)
re.setDatabasePopulator(new ResourceDatabasePopulator(resi));
return re;
}
Go for Flyway.clean(). It does exactly what you want. No need to write your own reset script.
You can use ApplicationRunner to run just after the startup and inside it do whatever you want with flyway. You'll also probably want to run migrate after clean:
#Component
public class CleanDatabase implements ApplicationRunner {
#Autowired
private Flyway flyway;
#Override
public void run(ApplicationArguments args) throws Exception {
flyway.clean();
flyway.migrate();
}
}
I got a valid issue when I run my package. It runs failed in my PC and success in anyone else.
The error is caused by Script Component (turned red), and it is in Post Execute phase, not the post execute in the script componet, but in the runtime of package. The error is:
Information: 0x40043008 at Data Flow Task, SSIS.Pipeline: Post Execute phase is beginning.
Error: 0xC0047062 at Data Flow Task, Script Component [263]: System.InvalidCastException: Unable to cast COM object of type 'System.__ComObject' to interface type 'Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSVariables100'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{22992C1D-393D-48FB-9A9F-4E4C62441CCA}' failed due to the following error: The application called an interface that was marshalled for a different thread. (Exception from HRESULT: 0x8001010E (RPC_E_WRONG_THREAD)).
I guess the issue is related to variables, because when I remove all the code related to variables, the package run successfully. The code in script component:
private int scheduled550;
private int scheduled620;
private int scheduled720;
private int scheduled820;
public override void PreExecute()
{
base.PreExecute();
scheduled550 = Variables.Count550;
scheduled620 = Variables.Count620;
scheduled720 = Variables.Count720;
scheduled820 = Variables.Count820;
}
public override void PostExecute()
{
base.PostExecute();
}
Did anyone ever encounter the same issue? Can anyone tell me what will POST Execute phase do? Thanks
More info: I have tried to re-install SQL Server, but this is not help. And not all the script component with variables failed running in my SSIS (not in the same package with the error one)
All the tasks/containers in an SSIS have the same lifecycle. You can see some of this by watching the Event Handlers fire. In a script component, inside a Data Flow Task, is going to under go various steps. Part of that is Validation (this contract says I should have a column from this table that is an integer type- can I connect, does it exist, is it the right type, etc).
After validation, tasks will have setup and tear down steps to perform. Since you appear to be working with SSIS Variables in your script, part of that pre/post execute time is spent allowing the translation of Variable (SSIS) to variable (.net) and back again.
Until I see the specific code in your PostExecute method that was causing the failure, I can't state what the code issue might have been.
I cannot recreate your issue. While this is the 2012 release of Integration Services, the Script Component as you are using it will behave the same. I did not send my output to Excel but that should not matter given it's the Script that is failing.
My Script component. I have selected my Variable, User::Count550 as a ReadOnly variable in the menu before editing my code.
public class ScriptMain : UserComponent
{
private int scheduled550;
public override void PreExecute()
{
base.PreExecute();
this.scheduled550 = Variables.Count550;
}
public override void PostExecute()
{
base.PostExecute();
//this.Variables.Count550 = scheduled550;
//this.VariableDispenser.LockForWrite("Count550");
}
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
if (true)
{
this.scheduled550++;
}
}
}