jooq connection pooling does not release connection - mysql

we tried to use dbcp and c3p0 as database connection pooling in our jooq environment. Both work fine for SELECT statements but CREATE and UPDATE statements does not release the connection.
We initialized the dbcp like:
public static DataSource setupDataSource(String dbUrl, String dbUserName, String dbPassword) {
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(dbUrl, dbUserName, dbPassword);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(connectionFactory, null);
ObjectPool<PoolableConnection> connectionPool = new GenericObjectPool<>(poolableConnectionFactory);
poolableConnectionFactory.setPool(connectionPool);
PoolingDataSource<PoolableConnection> dataSource = new PoolingDataSource<>(connectionPool);
return dataSource;
}
Then we get connection for each query:
Connection dbConnection = null;
try {
dbConnection = dataSource.getConnection();
} ....
DSLContext dslContext = DSL.using(connection, dialect);
Out create statement looks like:
protected final DSLContext jooq;
public E add(E entity) throws Exception {
E transformedEntity = null;
try {
R persisted;
persisted = jooq.insertInto(transformator.getTable())
.set(transformator.createRecord(entity))
.returning()
.fetchOne();
transformedEntity = transformator.getEntityFromTableRecord(persisted);
} catch (DataAccessException e) {
...
}
return transformedEntity;
}
And in the end we close the connection with:
dbConnection.close();
The problem is that the connections stay open and after the connection pool is full no connection can be created. Do I need to close the statements and resultsets? And if yes, how can I do this with jooq?

The easiest way to do this with jOOQ is to pass the data source directly to jOOQ. Instead of:
Connection dbConnection = null;
try {
dbConnection = dataSource.getConnection();
} ....
DSLContext dslContext = DSL.using(connection, dialect);
... write:
DSLContext dslContext = DSL.using(dataSource, dialect);
That way, jOOQ will manage connection lifecycles for you. Otherwise, I suspect you simply have some situations where your connection still leaks and isn't closed properly

Related

Cordapp- Hikari Connection Pool class not found for MySql ConnectionPoolDataSource

I am building an workflow for Corda. I want to use the Hikari connection pool library for connecting to MySql database. I AM NOT trying to replace the, ledger H2 database. This database is for storing/retrieving some information, which is not needed in the ledger. I am able to connect to MySql WITHOUT Hikari. However when I use Hikari, I get an error.
java.lang.ClassNotFoundException: com.mysql.cj.jdbc.MysqlConnectionPoolDataSource
I have tested the Hikari code, as a standalone jar file. It works fine. It is a combination of the way corda loads and runs the jar files, inside cordapps directory, which is causing the issue. Since the class is part of the jar. This seems it a little off
I have added the MySql dependency, inline with what is mentioned in https://docs.corda.net/cordapp-build-systems.html#setting-your-dependencies
I am also able to connect to the MySql DB, if I am not using Hikari.
I explored the cordapp jar .
And I could see that the requisite jar is present inside the cordapp jar.
Gradle dependencies for the cordapp
dependencies {
testCompile "junit:junit:$junit_version"
// Corda dependencies.
cordaCompile "$corda_release_group:corda-core:$corda_release_version"
cordaRuntime "$corda_release_group:corda:$corda_release_version"
testCompile "$corda_release_group:corda-node-driver:$corda_release_version"
runtime "mysql:mysql-connector-java:8.0.11"
cordaCompile "com.zaxxer:HikariCP:2.5.1"
// CorDapp dependencies.
cordapp project(":contracts")
}
Sample code
public class DataSource {
private static final Logger logger = LoggerFactory.getLogger(DataSource.class);
private static HikariConfig config = new HikariConfig();
private static HikariDataSource ds;
static {
try {
logger.info("Connecting with connection pool datasource");
config.setDataSourceClassName("com.mysql.cj.jdbc.MysqlConnectionPoolDataSource");
config.addDataSourceProperty("useSSL", "false");
config.addDataSourceProperty("user", "username");
config.addDataSourceProperty("password", "password");
config.addDataSourceProperty("serverName", "localhost");
config.addDataSourceProperty("useSSL", "false");
config.addDataSourceProperty("port",Integer.parseInt("3306"));
config.addDataSourceProperty("cachePrepStmts", "true");
config.addDataSourceProperty("prepStmtCacheSize", "250");
config.addDataSourceProperty("prepStmtCacheSqlLimit", "2048");
config.addDataSourceProperty("requireSSL", "false");
config.addDataSourceProperty("serverTimezone", "UTC");
config.addDataSourceProperty("useServerPrepStmts", "true");
config.addDataSourceProperty("allowPublicKeyRetrieval", "true");
config.addDataSourceProperty("databaseName", "database");
config.setPoolName("Hikari-MySql Pool Name");
logger.error("-- Create Hikari Datasource with config {} --", config);
ds = new HikariDataSource(config);
} catch (Throwable t) {
logger.error("Error Occurred during Datasource Initializaiton", t);
throw t;
}
}
private DataSource() {
}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
}
public class MySqlConnection {
static private final Logger logger = LoggerFactory.getLogger(MySqlConnection.class);
public Connection getMySqlConnection() {
Connection conn = null;
try {
conn = DataSource.getConnection();
logger.info("------------> Got conne :: " + conn);
} catch (SQLException e) {
logger.error("SQLException :: " + e);
}
return conn;
}
}
public class DataSourceTest {
static private final Logger logger = LoggerFactory.getLogger(DataSourceTest.class);
public static void main(String[] args) {
MySqlConnection mySqlConnection = new MySqlConnection();
Connection conn = null;
try {
conn = mySqlConnection.getMySqlConnection();
logger.info("------------> Got connection :: " + conn);
Statement statement = conn.createStatement();
ResultSet rs = statement.executeQuery("{some select statement}");
} catch (SQLException e) {
logger.error("SQLException :: " + e);
}
}
}
Exception:
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.MysqlConnectionPoolDataSource
at com.zaxxer.hikari.util.UtilityElf.createInstance(UtilityElf.java:90) ~[HikariCP-2.5.1.jar:?]
at com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:314) ~[HikariCP-2.5.1.jar:?]
at com.zaxxer.hikari.pool.PoolBase.<init>(PoolBase.java:108) ~[HikariCP-2.5.1.jar:?]
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:99) ~[HikariCP-2.5.1.jar:?]
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:71) ~[HikariCP-2.5.1.jar:?]
If I run the code from a main class inside a jar, it works. But it does not work from inside a cordapp
In MySQL for HikariCP use setJdbcUrl instead of setDataSourceClassName
config.setJdbcUrl("jdbc:mysql://localhost:3306/simpsons");
The MySQL DataSource is known to be broken with respect to network timeout support. Use jdbcUrl configuration instead.
Check your dependency tree. I think you have some collision with "mysql-connector-java" dependency, which cause mess in class loader.

Database connectivity using mssql2008 and jdbc

So I have setup my code like so
public static Connection getConnection() {
try {
String dbURL = "jdbc:sqlserver://localhost:1433;databaseName=HRDB;
String user = "sa";
String pass = "r";
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection conn = DriverManager.getConnection(dbURL, user, pass);
return conn;
} catch (ClassNotFoundException c) {
return null;
} catch (SQLException s) {
System.out.println(s.toString());
return null;
}
}
However, when I try to connect to the database I get the following exceptions.
com.microsoft.sqlserver.jdbc.SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "java.lang.RuntimeException: Could not generate DH keypair".

MySQL connection pooling with JERSEY

I'm developping a RESTful API with Jersey and MySQL.
I'm actually using the JDBC driver to connect to the database and I create a new connection everytime I want to acess it. As it clearly is a memory leakage, I started to implement the ServletContextClassclass but I don't know how to call the method when I need to get the result of a SQL query.
Here is how I did it wrong:
DbConnection.java
public class DbConnection {
public Connection getConnection() throws Exception {
try {
String connectionURL = "jdbc:mysql://root:port/path";
Connection connection = null;
Class.forName("com.mysql.jdbc.Driver").newInstance();
connection = DriverManager.getConnection(connectionURL, "root", "password");
return connection;
}
catch (SQLException e) {
throw e;
}
}
}
DbData.java
public ArrayList<Product> getAllProducts(Connection connection) throws Exception {
ArrayList<Product> productList = new ArrayList<Product>();
try {
PreparedStatement ps = connection.prepareStatement("SELECT id, name FROM product");
ResultSet rs = ps.executeQuery();
while (rs.next()) {
Product product = new Product();
product.setId(rs.getInt("id"));
product.setName(rs.getString("name"));
productList.add(product);
}
return productList;
} catch (Exception e) {
throw e;
}
}
Resource.java
#GET
#Path("task/{taskId}")
#Consumes(MediaType.APPLICATION_JSON)
public Response getInfos(#PathParam("taskId") int taskId) throws Exception {
try {
DbConnection database= new DbConnection();
Connection connection = database.getConnection();
Task task = new Task();
DbData dbData = new DbData();
task = dbData.getTask(connection, taskId);
return Response.status(200).entity(task).build();
} catch (Exception e) {
throw e;
}
}
Here is where I ended up trying to implement the new class:
ServletContextClass.java
public class ServletContextClass implements ServletContextListener {
public Connection getConnection() throws Exception {
try {
String connectionURL = "jdbc:mysql://root:port/path";
Connection connection = null;
Class.forName("com.mysql.jdbc.Driver").newInstance();
connection = DriverManager.getConnection(connectionURL, "root", "password");
return connection;
} catch (SQLException e) {
throw e;
}
}
public void contextInitialized(ServletContextEvent arg0) {
System.out.println("ServletContextListener started");
DbConnection database = new DbConnection();
try {
Connection connection = database.getConnection();
} catch (Exception e) {
e.printStackTrace();
}
}
public void contextDestroyed(ServletContextEvent arg0) {
System.out.println("ServletContextListener destroyed");
//con.close ();
}
}
But problem is, I don't know what to do next. Any help? Thanks
You need to set the Connection variable as an attribute of the ServletContext. Also, I would recommend using connection as a static class variable so you can close it in the contextDestroyed method.
You can retrieve the connection attribute in any of your servlets later on for doing your DB operations.
public class ServletContextClass implements ServletContextListener {
public static Connection connection;
public Connection getConnection(){
try {
String connectionURL = "jdbc:mysql://root:port/path";
Class.forName("com.mysql.jdbc.Driver").newInstance();
connection = DriverManager.getConnection(connectionURL, "root", "password");
} catch (SQLException e) {
// Do something
}
}
public void contextInitialized(ServletContextEvent arg0) {
System.out.println("ServletContextListener started");
getConnection();
arg0.getServletContext().setAttribute("connection", connection);
}
public void contextDestroyed(ServletContextEvent arg0) {
System.out.println("ServletContextListener destroyed");
try{
if(connection != null){
connection.close();
}
}catch(SQLException se){
// Do something
}
}
}
Finally access your connection attribute inside your Servlet (Resource). Make sure you pass #Context ServletContext to your Response method so you can access your context attributes.
#GET
#Path("task/{taskId}")
#Consumes(MediaType.APPLICATION_JSON)
public Response getInfos(#PathParam("taskId") int taskId, #Context ServletContext context) throws Exception {
try {
Connection connection = (Connection) context.getAttribute("connection");
Task task = new Task();
DbData dbData = new DbData();
task = dbData.getTask(connection, taskId);
return Response.status(200).entity(task).build();
} catch (Exception e) {
throw e;
}
}
Now that we have solved your current issue, we need to know what can go wrong with this approach.
Firstly, you are only creating one connection object which will be used everywhere. Imagine multiple users simultaneously accessing your API, the single connection will be shared among all of them which will slow down your response time.
Secondly, your connection to DB will die after sitting idle for a while (unless you configure MySql server not to kill idle connections which is not a good idea), and when you try to access it, you will get SQLExceptions thrown all over. This can be solved inside your servlet, you can check if your connection is dead, create it again, and then update the context attribute.
The best way to go about your Mysql Connection Pool will be to use a JNDI resource. You can create a pool of connections which will be managed by your servlet container. You can configure the pool to recreate connections if they go dead after sitting idle. If you are using Tomcat as your Servlet Container, you can check this short tutorial to get started with understanding the JNDI connection pool.

jsp mysql server connection timeout

hi i am doing an jsp project. and i deploy my project on apache tomcat. i use mysql as databese.
when i deploy project on remote server it is run good. but after some hours it gives me sql error. then i go back my apache server and start projecet again it run and after some hours it gives me same sql error again. i dont know the problem. is that caused from my java connection code or it is about mysql server. can some one tell me why it gives me sql error.?
public class ConnectionManager {
private String className = "com.mysql.jdbc.Driver";
private String userName ="username";
private String password = "password";
private String url = "jdbc:mysql://localhost:3306/database?useUnicode=true&characterEncoding=utf-8";
/**
* #uml.property name="connectionInstance"
* #uml.associationEnd
*/
private static ConnectionManager connectionInstance = null;
public ConnectionManager(){
}
public static synchronized ConnectionManager getInstance() {
if(connectionInstance == null) {
connectionInstance = new ConnectionManager();
}
return connectionInstance;
}
public Connection getConnection(){
Connection conn = null;
try {
Class.forName(className);
conn = DriverManager.getConnection (url, userName, password);
System.out.println("Connection Established");
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return conn;
}
MySQL has a default connection timeout of 8 hours. So this means that you've kept a SQL connection open for too long. Your code suggests that you're creating only one connection on application's startup and reusing it application wide. This is very bad. This is not threadsafe.
You need to change your code so that you're not declaring and storing the SQL Connection as a static or instance variable anywhere in your code. Instead, it should be declared, created and closed within the shortest possible scope. Preferably within the very same method block as where you're executing the SQL query.
Here's a minor rewrite of your ConnectionManager which does the job properly:
public class ConnectionManager {
private static final String DRIVER = "com.mysql.jdbc.Driver";
private static final String USERNAME ="username";
private static final String PASSWORD = "password";
private static final String URL = "jdbc:mysql://localhost:3306/database?useUnicode=true&characterEncoding=utf-8";
static {
try {
Class.forName(DRIVER);
}
catch (ClassNotFoundException e) {
throw new ExceptionInInitializerError(DRIVER + " missing in classpath!", e);
}
}
public static Connection getConnection() throws SQLException {
return DriverManager.getConnection(URL, USERNAME, PASSWORD);
}
}
Use it as follows:
public class SomeDAO {
public SomeEntity find(Long id) throws SQLException {
Connection connection = null;
// ...
try {
connection = ConnectionManager.getConnection();
// ...
}
finally {
// ...
if (connection != null) try { connection.close(); } catch(SQLException ignore) {}
}
return someEntity;
}
To improve connecting performance, use a connection pool instead of DriverManager.
See also:
Show JDBC ResultSet in HTML in JSP page using MVC and DAO pattern
Are you closing connections properly after using them.

Am I Using JDBC Connection Pooling?

I am trying to determine if I am actually using JDBC connection pooling. After doing some research, the implementation almost seems too easy. Easier than a regular connection in fact so i'd like to verify.
Here is my connection class:
public class DatabaseConnection {
Connection conn = null;
public Connection getConnection() {
BasicDataSource bds = new BasicDataSource();
bds.setDriverClassName("com.mysql.jdbc.Driver");
bds.setUrl("jdbc:mysql://localhost:3306/data");
bds.setUsername("USERNAME");
bds.setPassword("PASSWORD");
try{
System.out.println("Attempting Database Connection");
conn = bds.getConnection();
System.out.println("Connected Successfully");
}catch(SQLException e){
System.out.println("Caught SQL Exception: " + e);
}
return conn;
}
public void closeConnection() throws SQLException {
conn.close();
}
}
Is this true connection pooling? I am using the connection in another class as so:
//Check data against database.
DatabaseConnection dbConn = new DatabaseConnection();
Connection conn;
ResultSet rs;
PreparedStatement prepStmt;
//Query database and check username/pass against table.
try{
conn = dbConn.getConnection();
String sql = "SELECT * FROM users WHERE username=? AND password=?";
prepStmt = conn.prepareStatement(sql);
prepStmt.setString(1, user.getUsername());
prepStmt.setString(2, user.getPassword());
rs = prepStmt.executeQuery();
if(rs.next()){ //Found Match.
do{
out.println("UserName = " + rs.getObject("username") + " Password = " + rs.getObject("password"));
out.println("<br>");
} while(rs.next());
} else {
out.println("Sorry, you are not in my database."); //No Match.
}
dbConn.closeConnection(); //Close db connection.
}catch(SQLException e){
System.out.println("Caught SQL Exception: " + e);
}
Assuming that it's the BasicDataSource is from DBCP, then yes, you are using a connection pool. However, you're recreating another connection pool on every connection acquirement. You are not really pooling connections from the same pool. You need to create the connection pool only once on application's startup and get every connection from it. You should also not hold the connection as an instance variable. You should also close the connection, statement and resultset to ensure that the resources are properly closed, also in case of exceptions. Java 7's try-with-resources statement is helpful in this, it will auto-close the resources when the try block is finished.
Here's a minor rewrite:
public final class Database {
private static final BasicDataSource dataSource = new BasicDataSource();
static {
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/data");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
}
private Database() {
//
}
public static Connection getConnection() throws SQLException {
return dataSource.getConnection();
}
}
(this can if necessary be refactored as an abstract factory to improve pluggability)
and
private static final String SQL_EXIST = "SELECT * FROM users WHERE username=? AND password=?";
public boolean exist(User user) throws SQLException {
boolean exist = false;
try (
Connection connection = Database.getConnection();
PreparedStatement statement = connection.prepareStatement(SQL_EXIST);
) {
statement.setString(1, user.getUsername());
statement.setString(2, user.getPassword());
try (ResultSet resultSet = preparedStatement.executeQuery()) {
exist = resultSet.next();
}
}
return exist;
}
which is to be used as follows:
try {
if (!userDAO.exist(username, password)) {
request.setAttribute("message", "Unknown login. Try again.");
request.getRequestDispatcher("/WEB-INF/login.jsp").forward(request, response);
} else {
request.getSession().setAttribute("user", username);
response.sendRedirect("userhome");
}
} catch (SQLException e) {
throw new ServletException("DB error", e);
}
In a real Java EE environement you should however delegate the creation of the DataSource to the container / application server and obtain it from JNDI. In case of Tomcat, see also for example this document: http://tomcat.apache.org/tomcat-6.0-doc/jndi-resources-howto.html
Doesn't seem like it's pooled. You should store the DataSource in DatabaseConnection instead of creating a new one with each getConnection() call. getConnection() should return datasource.getConnection().
Looks like a DBCP usage. If so, then yes. It's already pooled. And here is the default pool property value of the DBCP.
/**
* The default cap on the number of "sleeping" instances in the pool.
* #see #getMaxIdle
* #see #setMaxIdle
*/
public static final int DEFAULT_MAX_IDLE = 8;
/**
* The default minimum number of "sleeping" instances in the pool
* before before the evictor thread (if active) spawns new objects.
* #see #getMinIdle
* #see #setMinIdle
*/
public static final int DEFAULT_MIN_IDLE = 0;
/**
* The default cap on the total number of active instances from the pool.
* #see #getMaxActive
*/
public static final int DEFAULT_MAX_ACTIVE = 8;
As a follow up to BalusC's solution, below is an implementation that I can be used within an application that requires more than one connection, or in a common library that would not know the connection properties in advance...
import org.apache.commons.dbcp.BasicDataSource;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.concurrent.ConcurrentHashMap;
public final class Database {
private static final ConcurrentHashMap<String, BasicDataSource> dataSources = new ConcurrentHashMap();
private Database() {
//
}
public static Connection getConnection(String connectionString, String username, String password) throws SQLException {
BasicDataSource dataSource;
if (dataSources.containsKey(connectionString)) {
dataSource = dataSources.get(connectionString);
} else {
dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl(connectionString);
dataSource.setUsername(username);
dataSource.setPassword(password);
dataSources.put(connectionString, dataSource);
}
return dataSource.getConnection();
}
}