Spring retry connection until datasource is available - mysql

I have a docker-compose setup to start my SpringBoot application and a MySQL database. If the database starts first, then my application can connect successfully. But if my application starts first, no database exists yet, so the application throws the following exception and exits:
app_1 | 2018-05-27 14:15:03.415 INFO 1 --- [ main]
com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
app_1 | 2018-05-27 14:15:06.770 ERROR 1 --- [ main]
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization
app_1 | com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure
I could edit my docker-compose file to make sure the database is always up before the application starts up, but I want the application to be able to handle this case on its own, and not immediately exit when it cannot reach the database address.
There are ways to configure the datasource in the application.properties file to make the application reconnect to the database, as answered here and here. But that doesn't work for a startup connection to the datasource.
How can I make my SpringBoot application retry the connection at startup to the database at a given interval until it successfully connects to the database?

Set HikariCP's initializationFailTimeout property to 0 (zero), or a negative number. As documented here:
⌚initializationFailTimeout
This property controls whether the pool will "fail fast" if the pool cannot be seeded with an initial connection successfully. Any positive number is taken to be the number of milliseconds to attempt to acquire an initial connection; the application thread will be blocked during this period. If a connection cannot be acquired before this timeout occurs, an exception will be thrown. This timeout is applied after the connectionTimeout period. If the value is zero (0), HikariCP will attempt to obtain and validate a connection. If a connection is obtained, but fails validation, an exception will be thrown and the pool not started. However, if a connection cannot be obtained, the pool will start, but later efforts to obtain a connection may fail. A value less than zero will bypass any initial connection attempt, and the pool will start immediately while trying to obtain connections in the background. Consequently, later efforts to obtain a connection may fail. Default: 1

There is an alternative way to do this, which doesn't rely on a specific Connection Pool library or a specific database. Note that you will need to use spring-retry to achieve the desired behaviour with this approach
First you need to add spring-retry to your dependencies :
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>${spring-retry.version}</version>
</dependency>
Then you can create a decorator over DataSource that will extends AbstractDataSource like bellow :
#Slf4j
#RequiredArgsConstructor
public class RetryableDataSource extends AbstractDataSource {
private final DataSource dataSource;
#Override
#Retryable(maxAttempts = 5, backoff = #Backoff(multiplier = 1.3, maxDelay = 10000))
public Connection getConnection() throws SQLException {
log.info("getting connection ...");
return dataSource.getConnection();
}
#Override
#Retryable(maxAttempts = 5, backoff = #Backoff(multiplier = 2.3, maxDelay = 10000))
public Connection getConnection(String username, String password) throws SQLException {
log.info("getting connection by username and password ...");
return dataSource.getConnection(username, password);
}
}
Then you will need to inject this custom DataSource decorator into Spring context by creating a custom BeanPostProcessor :
#Slf4j
#Order(value = Ordered.HIGHEST_PRECEDENCE)
#Component
public class RetryableDatabasePostProcessor implements BeanPostProcessor {
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
if(bean instanceof DataSource) {
log.info("-----> configuring a retryable datasource for beanName = {}", beanName);
return new RetryableDataSource((DataSource) bean);
}
return bean;
}
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
}
Last but not least you will need to enable Spring retry by adding #EnableRetry annotation to spring main class, example :
#EnableRetry
#SpringBootApplication
public class RetryableDbConnectionApplication {
public static void main(String[] args) {
SpringApplication.run(RetryableDbConnectionApplication.class, args);
}
}

Related

ShedLock - Not Executing

I am using shedlock library 4.20.0.
net.javacrumbs.shedlock shedlock-spring 4.20.0 net.javacrumbs.shedlock shedlock-provider-jdbc-template 2.1.0
The scheduler job is,
#scheduled(fixedRate = 5000)
#SchedulerLock(name = "TaskScheduler__scheduledTask", lockAtLeastForString = "PT5M", lockAtMostForString = "PT14M")
public void reportCurrentTime() {
LockAssert.assertLocked();
log.info("The time is now {} {}", dateFormat.format(new Date()), dataSource);
}
It shows #SchedulerLock as deprecated.
And the spring boot class,
#SpringBootApplication
#EnableScheduling
#EnableSchedulerLock(defaultLockAtMostFor = "PT30S")
public class DMSCaseEmulatorSpringApplication {
public static void main(String[] args) {
SpringApplication.run(DMSCaseEmulatorSpringApplication.class, args);
}
}
When i execute the spring boot class, it triggers shedlock and creates a record in database table but in logs i keep getting as below,
19:54:39.188 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Locked TaskScheduler__scheduledTask.
19:54:39.188 [scheduling-1] INFO u.g.h.c.d.s.ScheduledTasks - The time is now 19:54:39 HikariDataSource (HikariPool-1)
19:54:39.205 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Unlocked TaskScheduler__scheduledTask.
19:54:44.065 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Not executing TaskScheduler__scheduledTask. It's locked.
19:54:49.062 [scheduling-1] DEBUG n.j.s.c.DefaultLockingTaskExecutor - Not executing TaskScheduler__scheduledTask. It's locked.
Any thoughts will be appreciated?
The issue is caused by lockAtLeastForString = "PT5M" By specifying that, you are saying that the lock should be held at least for 5 minutes even if the task finishes sooner.
Regarding the Deprecation warning, please consult the JavaDoc.

manual set the properties of connection manager by c3p0

I get a connection from c3p0 pool for batch insert and i set the autoCommit=false,after using,i put it back to the poll. If the autoCommit=false still valid for this connection?
No. According the the JDBC spec, new Connections are autocommit=true, and c3p0 implements transparent Connection pooling, meaning an application that functions correctly in a c3p0 application should also function correctly using an unpooled Connection source.
If you wish, you can override this behavior by defining a ConnectionCustomizer. Something like...
package mypkg;
import com.mchange.v2.c3p0.AbstractConnectionCustomizer;
import java.sql.Connection;
public class AutocommitConnectionCustomizer extends AbstractConnectionCustomizer {
#Override
public void onCheckOut(Connection c, String parentDataSourceIdentityToken) throws Exception {
c.setAutoCommit( false );
}
}
Then, in your config...
c3p0.connectionCustomizerClassName=mypkg.AutocommitConnectionCustomizer
I hope this helps!

MySQL Max Connections error: User already has more than 'max_user_connections' active connections

I am working on a school project involving a MySQL database. We have encountered a problem where when we try to connect to our server we get the following error:
Severe: Exception while deploying the app [FrienDB-Server]
Severe: Exception during lifecycle processing
org.glassfish.deployment.common.DeploymentException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Error in allocating a connection. Cause: Connection could not be allocated because: User nwong already has more than 'max_user_connections' active connections
We are using our schools SQL server so we do not have access to changing the amount of max connections. I have tried killing connections on MySQL Workbench but that does not work. How can we fix this?
Here is my DatabaseConnection class
package friendb.server.util;
import java.util.Map;
import java.util.HashMap;
import org.eclipse.persistence.config.PersistenceUnitProperties;
import javax.persistence.EntityManagerFactory;
import javax.persistence.EntityManager;
import javax.persistence.Persistence;
/**
* Utility class for database connections.
* #author Evan Guby
*/
public final class DatabaseConnection
{
private static final String PERSISTENCE_UNIT_NAME = "FrienDB";
private static Map PROPERTIES = new HashMap();
private static final String USERNAME = "nwong";
private static final String PASSWORD = "108857304";
/**
* Creates and returns an entity manager connected to the persistence unit.
* #return
*/
public static EntityManager getEntityManager()
{
PROPERTIES.put(PersistenceUnitProperties.JDBC_USER, USERNAME);
PROPERTIES.put(PersistenceUnitProperties.JDBC_PASSWORD, PASSWORD);
EntityManagerFactory emf =
Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME, PROPERTIES);
return emf.createEntityManager();
}
}
In the rare occasions where it does run when I try to access something from the database through the server from our client I get this error:
Caused by: javax.transaction.xa.XAException: com.sun.appserv.connectors.internal.api.PoolingException: javax.resource.spi.LocalTransactionException: Communications link failure
The last packet successfully received from the server was 1,575,262 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.

Azure service bus with qpid JMS java client messages not taking from queue

I am using Azure service-bus queues (AMQP Protocol) with Apache Qpid (0.3) as Java client.
I am also using Spring JmsTemplate to produce messages and DefaultMessageListenerContainer to manage my consumers, spring JMS 4.0.6.
Spring configurations:
#PostConstruct
private void JndiLookup() throws NamingException {
// Configure JNDI environment
Hashtable<String, String> envPrp = new Hashtable<String, String>();
envPrp.put(Context.INITIAL_CONTEXT_FACTORY,
PropertiesFileInitialContextFactory.class.getName());
envPrp.put("connectionfactory.SBCF", "amqps://owner:{parimeryKey}#{namespace}.servicebus.windows.net");
envPrp.put("queue.STORAGE_NEW_QUEUE", "QueueName");
context = new InitialContext(envPrp);
}
#Bean
public ConnectionFactory connectionFactory() throws NamingException {
ConnectionFactory cf = (ConnectionFactory) context.lookup("SBCF");
return cf;
}
#Bean
public DefaultMessageListenerContainer messageListenerContainer() throws NamingException {
DefaultMessageListenerContainer messageListenerContainer = new DefaultMessageListenerContainer();
messageListenerContainer.setConnectionFactory(connectionFactory());
Destination queue = (Destination) context.lookup("QueueName");
messageListenerContainer.setDestination(queue);
messageListenerContainer.setConcurrency("3-10");
MessageListenerAdapter adapter = new MessageListenerAdapter();
adapter.setDelegate(new MessageWorker());
adapter.setDefaultListenerMethod("onMessage");
messageListenerContainer.setMessageListener(adapter);
return messageListenerContainer;
}
#Bean
public JmsTemplate jmsTemplate() throws NamingException {
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setConnectionFactory(connectionFactory());
return jmsTemplate;
}
Nothing fancy in the configurations just straight forward.
Running the code and everything seems to be working .. but after few minutes without traffic in the queue it seems like the consumers are losing connection with the queue and not taking messages.
I dont know if it is related but every 5 minutes in am getting the following warning:
Fri Nov 07 15:23:53 +0000 2014, (DefaultMessageListenerContainer.java:842) WARN : Setup of JMS message listener invoker failed for destination 'org.apache.qpid.amqp_1_0.jms.impl.QueueImpl#8fb0427b' - trying to recover. Cause: Force detach the link because the session is remotely ended.
Fri Nov 07 15:23:56 +0000 2014, (DefaultMessageListenerContainer.java:891) INFO : Successfully refreshed JMS Connection
I have messages being in the queue for hours and not being handled by the consumers only when I restart the app the consumers renewing the connection properly and taking the messages.
Is it possible that the problem is with the Spring Listener container properties or qpid connection factory or is it an issue with Azure service bus??
Couldn't find related post to my situation will appreciate the help!!

Flyway migration, Unable to obtain Jdbc connection from DataSource

I am trying to use flyway to create and manage a MySQL database. Here is the code i have got so far.
FlywayMigration.java : Class that applys the migration
public class FlywayMigration
{
public FlywayMigration(DatabaseConfiguration configuration, Flyway flyway)
{
flyway.setDataSource(configuration.getDataSource());
flyway.migrate();
}
public static void main(String[] args)
{
new FlywayMigration(new DatabaseConfiguration("database.properties"), new Flyway());
}
}
DatabaseConfiguration.java : Configuration class, this class will configure the datasource to be applyed to the Flyway.setDataSource method
public class DatabaseConfiguration
{
private final Logger LOGGER = LoggerFactory.getLogger(this.getClass());
private PropertiesUtil prop = null;
public DatabaseConfiguration(String file)
{
prop = new PropertiesUtil(file);
}
public String getDataSourceClass()
{
return prop.getProperty("mysql.data.source.class");
}
public String getURL ()
{
return prop.getProperty("mysql.url");
}
public String getHostName()
{
return prop.getProperty("mysql.host.name");
}
public String getDatabaseName()
{
return prop.getProperty("mysql.database.name");
}
public DataSource getDataSource()
{
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setURL(getURL());
dataSource.setUser(prop.getProperty("mysql.user.name"));
dataSource.setPassword(null);
return dataSource;
}
}
database.properties is the file where i store the database information, password can be null
mysql.data.source.class=com.mysql.jdbc.Driver
mysql.url=jdbc:mysql://localhost:3306/vmrDB
mysql.host.name=localhost
mysql.database.name=vmrDB
mysql.user.name=root
And i get the folowing error in my trace
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1144)
at org.flywaydb.core.Flyway.migrate(Flyway.java:811)
at com.bt.sitb.vmr.migration.FlywayMigration.<init>(FlywayMigration.java:10)
at com.bt.sitb.vmr.migration.FlywayMigration.main(FlywayMigration.java:15)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Can someone please tell me why the DataSource from MySQL is not connecting?
It looks like Flyway cannot connect to the database.
One reason for this is that the database in the database URL does not exist.
Question: does your database schema exist?
If your answer is no, then:
connect to jdbc:mysql://localhost:3306/mysql
also specify the schema to use for migration with flyway.setSchemas(configuration.getDatabaseName())
you also need flyway.init() before you can initialize migration of your database.
Ran into this same issue. Apparently, the problem was with my .properties file. The jar was using the one packaged with it and not the external one. So I moved my external properties file out of the resources folder and into the root directory of the jar and problem solved!
Hope this helps someone.
I had this same issue when working on a Java application in Debian 10 using Tomcat Application server.
I defined the connection strings for the database in the context.xml file, however, when I start out the application and try to log into the application, I get the error:
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1144)
Here's what I figured out:
I finally realized that the application was using internally defined database connection strings that were packaged with it. The internally defined database connection strings were different from my own database connection strings defined in the context.xml file.
The solution for me was to either modify the internally defined database connection strings that were packaged with the application or use the same internally defined database connection strings that were packaged with application in my context.xml file.
That's all.
I hope this helps.