I am trying to use flyway to create and manage a MySQL database. Here is the code i have got so far.
FlywayMigration.java : Class that applys the migration
public class FlywayMigration
{
public FlywayMigration(DatabaseConfiguration configuration, Flyway flyway)
{
flyway.setDataSource(configuration.getDataSource());
flyway.migrate();
}
public static void main(String[] args)
{
new FlywayMigration(new DatabaseConfiguration("database.properties"), new Flyway());
}
}
DatabaseConfiguration.java : Configuration class, this class will configure the datasource to be applyed to the Flyway.setDataSource method
public class DatabaseConfiguration
{
private final Logger LOGGER = LoggerFactory.getLogger(this.getClass());
private PropertiesUtil prop = null;
public DatabaseConfiguration(String file)
{
prop = new PropertiesUtil(file);
}
public String getDataSourceClass()
{
return prop.getProperty("mysql.data.source.class");
}
public String getURL ()
{
return prop.getProperty("mysql.url");
}
public String getHostName()
{
return prop.getProperty("mysql.host.name");
}
public String getDatabaseName()
{
return prop.getProperty("mysql.database.name");
}
public DataSource getDataSource()
{
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setURL(getURL());
dataSource.setUser(prop.getProperty("mysql.user.name"));
dataSource.setPassword(null);
return dataSource;
}
}
database.properties is the file where i store the database information, password can be null
mysql.data.source.class=com.mysql.jdbc.Driver
mysql.url=jdbc:mysql://localhost:3306/vmrDB
mysql.host.name=localhost
mysql.database.name=vmrDB
mysql.user.name=root
And i get the folowing error in my trace
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1144)
at org.flywaydb.core.Flyway.migrate(Flyway.java:811)
at com.bt.sitb.vmr.migration.FlywayMigration.<init>(FlywayMigration.java:10)
at com.bt.sitb.vmr.migration.FlywayMigration.main(FlywayMigration.java:15)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Can someone please tell me why the DataSource from MySQL is not connecting?
It looks like Flyway cannot connect to the database.
One reason for this is that the database in the database URL does not exist.
Question: does your database schema exist?
If your answer is no, then:
connect to jdbc:mysql://localhost:3306/mysql
also specify the schema to use for migration with flyway.setSchemas(configuration.getDatabaseName())
you also need flyway.init() before you can initialize migration of your database.
Ran into this same issue. Apparently, the problem was with my .properties file. The jar was using the one packaged with it and not the external one. So I moved my external properties file out of the resources folder and into the root directory of the jar and problem solved!
Hope this helps someone.
I had this same issue when working on a Java application in Debian 10 using Tomcat Application server.
I defined the connection strings for the database in the context.xml file, however, when I start out the application and try to log into the application, I get the error:
Exception in thread "main" org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1144)
Here's what I figured out:
I finally realized that the application was using internally defined database connection strings that were packaged with it. The internally defined database connection strings were different from my own database connection strings defined in the context.xml file.
The solution for me was to either modify the internally defined database connection strings that were packaged with the application or use the same internally defined database connection strings that were packaged with application in my context.xml file.
That's all.
I hope this helps.
Related
I'm using Pomelo Entity framework core with MySqlConnector in my asp.net 5 app to connect to my MySql database - using custom DbContext classes. This normally works fine.
However, I have a need to connect to another database than the one in the connection string (for instance 'INFORMATION_SCHEMA').
I can of course change the connection string, replacing the database name, but:
That creates an extra connection pool - one per connection string!
I'm trying to avoid that - having only one connection pool per website.
I was messing around with 'SetDefaultSchema' and other attempts that all fail miserably.
How can I change the database name, the DbContext uses so I only have one connection pool and still each DbContext has its own database to connect to?
The solution is actually quite simple: Use connection interceptor (available from Entity Framework Core 3.0+).
The code below switches the database after the connection has been opened.
Now each DbContext class can use its own database and with only one connection pool in use.
First you create an interceptor class inherited from DbConnectionInterceptor. The constructor takes the database name, you want to switch to, as parameter:
using Microsoft.EntityFrameworkCore.Diagnostics;
using System.Data.Common;
using System.Threading.Tasks;
public class MySqlConnectionInterceptor : DbConnectionInterceptor
{
public MySqlConnectionInterceptor(string databaseName)
{
database = databaseName;
}
readonly string database;
public override void ConnectionOpened(DbConnection connection, ConnectionEndEventData eventData)
{
if (database != null)
{
connection.ChangeDatabase(database); // The 'magic' code
}
base.ConnectionOpened(connection, eventData);
}
public override async Task ConnectionOpenedAsync(DbConnection connection, ConnectionEndEventData eventData, CancellationToken cancellationToken = default)
{
if (database != null)
{
await connection.ChangeDatabaseAsync(database); // The 'magic' code
}
await base.ConnectionOpenedAsync(connection, eventData, cancellationToken);
}
}
Now all you have to is include one line in your DbContext class's OnConfiguring method:
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.AddInterceptors(new MySqlConnectionInterceptor("yourDatabase"));
}
Now the connection will switch to 'yourDatabase' database every time, it's opened.
And it will only use one connection pool (total)! That way the number of 'sleeping' connections are kept at a minimum.
This works because Pomelo Entity Framework Core always resets a connection before reusing it from the pool (unless you specifically sets 'Connectionreset=false' - which is bad anyway). It sets the database back to the one in the connection string, which you of course can override again).
Of course you don't have to hard code the database name. If you for instance use a base DbContext class, that your other DbContexts inherits from, you can create a constructor that takes the database name as parameter, like this:
public class BaseDbContext : DbContext
{
public BaseDbContext (string databaseName)
{
database = databaseName;
}
string database;
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.AddInterceptors(new MySqlConnectionInterceptor(database));
}
}
The code has been tested in Asp.Net 5+6 and .Net Windows Forms.
I have a docker-compose setup to start my SpringBoot application and a MySQL database. If the database starts first, then my application can connect successfully. But if my application starts first, no database exists yet, so the application throws the following exception and exits:
app_1 | 2018-05-27 14:15:03.415 INFO 1 --- [ main]
com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
app_1 | 2018-05-27 14:15:06.770 ERROR 1 --- [ main]
com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization
app_1 | com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure
I could edit my docker-compose file to make sure the database is always up before the application starts up, but I want the application to be able to handle this case on its own, and not immediately exit when it cannot reach the database address.
There are ways to configure the datasource in the application.properties file to make the application reconnect to the database, as answered here and here. But that doesn't work for a startup connection to the datasource.
How can I make my SpringBoot application retry the connection at startup to the database at a given interval until it successfully connects to the database?
Set HikariCP's initializationFailTimeout property to 0 (zero), or a negative number. As documented here:
⌚initializationFailTimeout
This property controls whether the pool will "fail fast" if the pool cannot be seeded with an initial connection successfully. Any positive number is taken to be the number of milliseconds to attempt to acquire an initial connection; the application thread will be blocked during this period. If a connection cannot be acquired before this timeout occurs, an exception will be thrown. This timeout is applied after the connectionTimeout period. If the value is zero (0), HikariCP will attempt to obtain and validate a connection. If a connection is obtained, but fails validation, an exception will be thrown and the pool not started. However, if a connection cannot be obtained, the pool will start, but later efforts to obtain a connection may fail. A value less than zero will bypass any initial connection attempt, and the pool will start immediately while trying to obtain connections in the background. Consequently, later efforts to obtain a connection may fail. Default: 1
There is an alternative way to do this, which doesn't rely on a specific Connection Pool library or a specific database. Note that you will need to use spring-retry to achieve the desired behaviour with this approach
First you need to add spring-retry to your dependencies :
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>${spring-retry.version}</version>
</dependency>
Then you can create a decorator over DataSource that will extends AbstractDataSource like bellow :
#Slf4j
#RequiredArgsConstructor
public class RetryableDataSource extends AbstractDataSource {
private final DataSource dataSource;
#Override
#Retryable(maxAttempts = 5, backoff = #Backoff(multiplier = 1.3, maxDelay = 10000))
public Connection getConnection() throws SQLException {
log.info("getting connection ...");
return dataSource.getConnection();
}
#Override
#Retryable(maxAttempts = 5, backoff = #Backoff(multiplier = 2.3, maxDelay = 10000))
public Connection getConnection(String username, String password) throws SQLException {
log.info("getting connection by username and password ...");
return dataSource.getConnection(username, password);
}
}
Then you will need to inject this custom DataSource decorator into Spring context by creating a custom BeanPostProcessor :
#Slf4j
#Order(value = Ordered.HIGHEST_PRECEDENCE)
#Component
public class RetryableDatabasePostProcessor implements BeanPostProcessor {
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
if(bean instanceof DataSource) {
log.info("-----> configuring a retryable datasource for beanName = {}", beanName);
return new RetryableDataSource((DataSource) bean);
}
return bean;
}
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
}
Last but not least you will need to enable Spring retry by adding #EnableRetry annotation to spring main class, example :
#EnableRetry
#SpringBootApplication
public class RetryableDbConnectionApplication {
public static void main(String[] args) {
SpringApplication.run(RetryableDbConnectionApplication.class, args);
}
}
I am using play java 2.5.
I have created a database with following java code.
public OnStartup() throws SQLException {
//demo create database with java code
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/?user=root&password=12345678");
Statement s = con.createStatement();
int Result = s.executeUpdate("CREATE DATABASE recruit3");
}
Module:
public class OnStartupModule extends AbstractModule {
#Override
public void configure() {
bind(OnStartup.class).asEagerSingleton();
}
}
application.conf:
play.modules {
enabled += "be.objectify.deadbolt.java.DeadboltModule"
enabled += modules.CustomDeadboltHook
enabled += modules.OnStartupModule
}
default.driver=com.mysql.jdbc.Driver
default.url="jdbc:mysql://localhost:3306/recruit3"
default.username=root
default.password="12345678"
My question is, why running the web-app creating
error Cannot connect to database [default]
How to fix that, if I don't want to create the database with mysql workbench.
Any suggestion or cannot do this, please tell me.
Thanks for advance.
As well as moving your database keys to the db.default namespace, you should be injecting Database into OnStartup to access the database configured with those properties.
First, add Play's JDBC support to build.sbt.
libraryDependencies += javaJdbc
If you're already running activator, make sure you use the reload command to pick up the changes to the build.
Update your application.conf to place the database configuration into the correct namespace.
db {
default {
driver=com.mysql.jdbc.Driver
url="jdbc:mysql://localhost:3306/recruit3"
username=root
password="12345678"
}
}
Finally, update OnStartup to receive a Database object that will be injected by Play.
import javax.inject.Inject;
import play.db.Database;
public class OnStartup {
#Inject
public OnStartup(final Database db) throws SQLException {
db.withConnection((Connection conn) -> {
final Statement s = con.createStatement();
return s.executeUpdate("CREATE DATABASE recruit3");
});
}
}
This allows you to configure the database one time, in application.conf, instead of hard-coding DB configuration into a class.
You can find more information here.
Your database keys start with default instead of db.default. The correct syntax is something like this:
db {
default {
driver=com.mysql.jdbc.Driver
url="jdbc:mysql://localhost:3306/recruit3"
username=root
password="12345678"
}
}
You already made your class as eager singleton, so it should work
I am new to Spring boot.I want to add some sql while database is creating like seed data.
#Value("classpath:com/foo/sql/db-test-data.sql")
private Resource dataScript;
#Bean
public DataSourceInitializer dataSourceInitializer(final DataSource dataSource) {
final DataSourceInitializer initializer = new DataSourceInitializer();
initializer.setDataSource(dataSource);
initializer.setDatabasePopulator(databasePopulator());
return initializer;
}
private DatabasePopulator databasePopulator() {
final ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
populator.addScript(dataScript);
return populator;
}
props.put("hibernate.query.substitutions", "true 1, false 0");
props.put("hibernate.hbm2ddl.auto", "create-drop");
props.put("hibernate.show_sql", "false");
props.put("hibernate.format_sql", "true");
I have perform this action.But it not working on spring boot.Can any one help me.
Sometimes spring-boot gets more in the way than it helps; IMHO this is especially so with web applications.
What you can do to get around this is to rename the bean that you define.
#Bean("springBootPleaseStopTellingMeHowYouThinkDataSourceInitializer")
public DataSourceInitializer dataSourceInitializer(DataSource dataSource) {
// build it.
}
Now, to turn off the built in bit that looks for data.sql in application.properties
spring.datasource.initialize=false
There, now boot is booted out of the way.
You can take advantage of Spring Boot database initialization capabilities. The simplest way is to place a "data.sql" file in the root of the classpath. So you just need to:
Change your sql file name to "data.sql".
Place it in "src/main/resources".
Spring Boot will automatically pick up the file and use it to initialize the database on startup.
You can check the documentation if you need to customize the file name, location, etc.
I'm trying to add a long list of file into mysql and use spring ACL service with grails to attach permission.
So, in my controller i have:
Files.withTransaction {
Files file = new Files(dataStore:ds,created:new Date(),path:target,name:fileName,contentType:contentType,contentLength:contentLength,isFolder:false).save(flush:true)
file = Files.lock(file.id)
filesService.addPermission(file, username ,BasePermission.ADMINISTRATION)
}
the i have no worries with the Files domain object, it supports huge amount of data ,(specially since i 've disabled versioning in mysql), the problem is on filesService which uses aclUtilService,
#Transactional
#PreAuthorize("hasRole('ROLE_ADMIN') or hasRole('ROLE_USER') or hasRole('ROLE_GROUP_OWNER')")
def addPermission(Files f, String username,Permission permission) {
aclUtilService.addPermission f,username,permission
}
Randomly i have the following deadlock (optmistic?)error :
Deadlock found when trying to get lock; try restarting transaction.
Stacktrace follows:
com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException:
Deadlock found when trying to get lock; try restarting transaction at
com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at
com.mysql.jdbc.Util.getInstance(Util.java:381) at
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1045) at
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at
com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3558) at
com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3490) at
com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959) at
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2109) at
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2648) at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2077)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2362)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2280)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2265)
at
org.grails.plugins.springsecurity.service.acl.AclService.save(AclService.groovy:330)
at
org.grails.plugins.springsecurity.service.acl.AclService.createEntries(AclService.groovy:198)
at
org.grails.plugins.springsecurity.service.acl.AclService.updateAcl(AclService.groovy:176)
at
GrailsMelodyGrailsPlugin$_closure5_closure18_closure19.doCall(GrailsMelodyGrailsPlugin.groovy:170)
at
org.grails.plugins.springsecurity.service.acl.AclUtilService.addPermission(AclUtilService.groovy:90)
at
org.grails.plugins.springsecurity.service.acl.AclUtilService.addPermission(AclUtilService.groovy:67)
at
GrailsMelodyGrailsPlugin$_closure5_closure18_closure19.doCall(GrailsMelodyGrailsPlugin.groovy:170)
at
xxxxxxxxxxxxx.FilesService.addPermission(FilesService.groovy:34)
at
GrailsMelodyGrailsPlugin$_closure5_closure18_closure19.doCall(GrailsMelodyGrailsPlugin.groovy:170)
at
xxxxxxxxxxxxxQuantumController$_uploadToS3_closure1$$ENzPdDAW.doCall(QuantumController.groovy:87)
at
org.grails.datastore.gorm.GormStaticApi.withTransaction(GormStaticApi.groovy:686)
at
xxxxxxxxxxxxx.QuantumController$$ENzPdDAW.uploadToS3(QuantumController.groovy:84)
at
grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
at
grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:149) at
net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:259)
at
net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:202)
at
net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:175)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Any help please ?
just encapsulated the transaction in controller and use #Transactional in the service to solve the issue:
Files.withTransaction {
Files file = new Files(
...
if (file.validate()) {
file.save()
filesService.addPermission(file,username,BasePermission.ADMINISTRATION)
...}
}