Using both Hive and MySql JDBC drivers - mysql

Question

TL;DR:
Is it problematic to use both Hive and MySql JDBC together?
I'm working on an application that performs several SQL queries using the MySql JDBC driver and afterwards it also send another Hive query using Hive JDBC.
Now whats happening is that the MySql queries are working properly, when the code tries to execute the Hive query it throws this exception:
com.mysql.cj.core.exceptions.WrongArgumentException: Connector/J cannot handle a database URL of type 'jdbc:hive2:'.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:54)
at com.mysql.cj.core.conf.url.ConnectionUrl$Type.fromValue(ConnectionUrl.java:149)
at com.mysql.cj.core.conf.url.ConnectionUrl.getConnectionUrlInstance(ConnectionUrl.java:193)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:195)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at company.services.HiveV2Provider.createConnection(HiveProvider.scala:105)
at company.services.HiveProvider$class.loanConnection(HiveProvider.scala:66)
Now after this exception is thrown the query is executed properly.
My guess is that since I'm loading both the MySql and Hive drivers, the MySql driver is trying to run this query first but when it encounters the Hive URL it throws this exception and then the Hive driver sees it and executes the query properly
This is how I execute the MySql code:
val query = ... // query is created here
var mysqlConn: Connection = null
var stmt: Statement = null
try {
Class.forName("com.mysql.jdbc.Driver")
mysqlConn = DriverManager.getConnection(mysqlAddress, username, password)
stmt = mysqlConn.createStatement()
val rs = stmt.executeQuery(query)
val returnVal = someResultSetHandlingFunction(rs)
rs.close()
returnVal
} catch {
case NonFatal(e) =>
logWarning(s"Failed to execute query on: $mysqlAddress", e)
throw e
} finally {
if (mysqlConn != null) {
mysqlConn.close()
}
}
My Hive code looks the same only with a driver name of: org.apache.hive.jdbc.HiveDriver (and it communicates with jdbc:hive2://someurl)
Versions:
Hive is hive-jdbc-1.1.0-cdh5.7.1
MySql is mysql-connector-java 6.0.4
Does anybody know if there's any way to avoid receiving this exception? Is it problematic to load 2 different JDBC drivers? Reading in other somewhat similar questions I get the impression that this should not be a problem
Just a few clarifications:
I know its probably not the best thing to use JDBC directly but I'm checking something and JDBC is fine for this task
I'm using Scala but I don't think it matters for this issue
Thanks in advance

Answer 1

I almost forgot to answer my question
So the issue is probably related to this bug. Back when I was facing this issue I didn't notice it was just a stack trace print and not an actual failure so it was less problematic than I expected.
Anyway, I then saw that in some specific versions this issue was fixed as you can see here so I just changed my mysql version to 5.1.9 (because I didn't need the higher version for anything specific) and the stack trace failure was gone.
I'll be happy if to hear if someone has a more elegant solution to this
Cheers

Answer 2

I was facing the same issue with MS SQL Server JDBC Driver. The same error was logged but everything worked fine.
According to this Microsoft page:
In the JDBC API 4.0, the DriverManager.getConnection method is
enhanced to load JDBC drivers automatically. Therefore, applications
do not need to call the Class.forName method to register or load the
driver when using the sqljdbc4.jar, sqljdbc41.jar, or sqljdbc42.jar
class library.
So I tried removing the Class.forName and just called DriverManager.getConnection. Things are just working and I'm not getting the annoying error anymore.
I believe the Driver itself must include a "META-INF/services/java.sql.Driver" file that registers itself as a valid JDBC driver, so not necessarily it will work for you, but to SQL Server Driver users it's the way to go.
BTW: I noticed that DriverManager.getConnection takes significant more time (6 or 7 seconds) to load the Drive the first time it's called. Subsequent calls are OK. Depending on your application it may be an issue.

Answer 3

Class.forName("com.mysql.jdbc.Driver")
will register your JDBC driver in the DriverManager. Then you put hive connection uri in
DriverManager.getConnection(mysqlAddress, username, password)
Exception is expected in this case.
Why don't you delegate the call to particular JDBC driver after checking uri like:
if (uri.contains("hive")){
//call Hive JDBC
}
else if (uri.contains("mysql")){
//call Mysql JDBC
}