Spring-Session with JDBC configuration: Table 'test.spring_session' doesn't exist - mysql

I try to run this example but without using Redis, instead with my local MySQL server.
I have edited this spring boot app like this:
Gradle:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:$springBootVersion")
}
}
apply plugin: 'spring-boot'
apply from: JAVA_GRADLE
//this 'if' statement is because I was getting error: Execution failed for task ':samples:findbyusername:findMainClass'.
//> Could not find property 'main' on task ':samples:findbyusername:run'.
if (!hasProperty('mainClass')) {
ext.mainClass = 'sample.FindByUsernameApplication'
}
tasks.findByPath("artifactoryPublish")?.enabled = false
group = 'samples'
dependencies {
compile("org.springframework.boot:spring-boot-starter-jdbc:$springBootVersion")
compile group: 'mysql', name: 'mysql-connector-java', version: '6.0.2'
compile group: 'org.springframework.session', name: 'spring-session', version: '1.2.0.RELEASE'
compile project(':spring-session'),
"org.springframework.boot:spring-boot-starter-web",
"org.springframework.boot:spring-boot-starter-thymeleaf",
"nz.net.ultraq.thymeleaf:thymeleaf-layout-dialect",
"org.springframework.security:spring-security-web:$springSecurityVersion",
"org.springframework.security:spring-security-config:$springSecurityVersion",
"com.maxmind.geoip2:geoip2:2.3.1",
"org.apache.httpcomponents:httpclient"
testCompile "org.springframework.boot:spring-boot-starter-test",
"org.assertj:assertj-core:$assertjVersion"
integrationTestCompile gebDependencies,
"org.spockframework:spock-spring:$spockVersion"
}
def reservePort() {
def socket = new ServerSocket(0)
def result = socket.localPort
socket.close()
result
}
application.properties
spring.datasource.url = jdbc:mysql://localhost:3306/TEST?characterEncoding=UTF-8&useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.datasource.driverClassName = com.mysql.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=but
spring.thymeleaf.cache=false
spring.template.cache=false
HttpSessionConfig.java
#EnableJdbcHttpSession // <1>
public class HttpSessionConfig {
}
Application starts on tomcat but when I hit localhost in my browser I get:
Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.
Mon May 23 21:14:31 CEST 2016
There was an unexpected error (type=Internal Server Error, status=500).
PreparedStatementCallback; bad SQL grammar [INSERT INTO SPRING_SESSION(SESSION_ID, CREATION_TIME, LAST_ACCESS_TIME, MAX_INACTIVE_INTERVAL, PRINCIPAL_NAME) VALUES (?, ?, ?, ?, ?)]; nested exception is java.sql.SQLSyntaxErrorException: Table 'test.spring_session' doesn't exist
I don't remember reading anything about manually creating this table so I assumed that spring will handle it for me...
EDIT:
I actually tried to manually create tables and then application runs OK. But I guess I shouldn't be doing this manually.

Spring Session ships with database schema scripts for most major RDBMS's (located in org.springframework.session.jdbc package), but the creation of database tables for Spring Session JDBC supports needs to be taken care of by the users themselves.
The provided scripts can be used untouched, however some users may choose to modify them to fit their specific needs, using the provided scripts as a reference.
An option would be to use a database migration tool, such as Flyway, to handle the creation of database tables.
Since you're using Spring Boot, it might be of your interest that there is a pending PR to add support for automatic initialization of Spring Session JDBC schema: https://github.com/spring-projects/spring-boot/pull/5879
If the documentation misled you into thinking the tables should be created automatically by Spring Session itself, consider reporting the issue so we can update the documentation if necessary: https://github.com/spring-projects/spring-session/issues

At least as of now, you don't have to create tables manually.
In my test, tables were created automatically after adding the following line into the file application.properties when this file appears like shown above.
spring.session.jdbc.initialize-schema=always
I found this beautiful line from the following stackoverflow page.
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'test.spring_session' doesn't exist - Spring Boot
I am sorry that I don't know if it was necessary to create tables manually in 2016 or 2017. I will update this answer when I get to know this or get to have some more fruitful related information. I am just wishing that nobody will be led to an idea that automatic creation of session tables is impossible with the lastest Spring Framework version of 2019 or later.

spring.session.jdbc.initialize-schema=always worked for me

Related

Spring Boot & Hibernate: MySQL TEXT with local h2 and Flyway

I am building a Spring Boot application which has several long texts in it's entities.
To ensure I can handle my database migrations well I included Flyway. In production I'm using a MySQL database, for local testing I want to implement the default h2 database.
An entity might have the following property
#Column(columnDefinition = "TEXT")
val startText: String?
For my MySQL database, this works fine and looks like this in my flyway schema:
start_text TEXT,
When I now start my tests with the default h2 in-memory database in Spring, I receive the following error:
Schema-validation: wrong column type encountered in column [start_text] in table [t_table]; found [clob (Types#CLOB)], but expecting [text (Types#VARCHAR)]
I understand that h2 does not support the MySQL specific type TEXT but actually I have no clue how to fix this.
Any help is appreciated.
Thank you.
I found a workaround for this.
In my application.yaml I have the following:
spring:
flyway:
placeholders:
text-datatype: 'TEXT' #defines a placeholder that is available in flyway
In my application.yaml in my test folder I have the following
spring:
flyway:
placeholders:
text-datatype: 'VARCHAR(255)'
Now I can use the placeholder in my Flyway scripts and it works fine:
start_text ${text-datatype}

For django testing, how do I use keepdb with mariadb

I have a database with a lot of nonmanaged tables which I'm using for a django app. For testing I'm wanting to use the --keepdb option so that I don't have to repopulate these tables every time. I'm using MariaDB for my database. If I don't use the keepdb option everything works fine, the test database gets created and destroyed properly.
But when I try to run the test keeping the database:
$ python manage.py test --keepdb
I get the following error:
Using existing test database for alias 'default'...
Got an error creating the test database: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CREATE DATABASE IF NOT EXISTS test_livedb ;\n SET sql_note' at line 2")
I assume that this is an issue with a different syntax between MariaDB and MySQL. Is there anyway to get the keepdb option to work with MariaDB?
thanks very much!
For what it's worth: This bug was introduced in Django 2.0.0 and fixed in Django 2.1.3 (https://code.djangoproject.com/ticket/29827)
Two things - check out Factory Boy (for creating test data) and I would suggest checking out Pytest as well. With non-managed tables, the issue I think you'll run into is that (at least in my experience) django won't create them in the test environment and you end up running into issues because there is no migration file to create those tables (since they're unmanaged). Django runs the migration files when creating the test environment.
With Pytest you can run with a --nomigrations flag which builds your test database directly off the models (thus creating the tables you need for your unmanaged models).
If you combine Pytest and Factory Boy you should be able to come up with the ability to setup your test data so it works as expected, is repeatable and testable without issue.
I actually approach it like this (slightly hacky, but it works with our complex setup):
On my model:
class Meta(object):
db_table = 'my_custom_table'
managed = getattr(settings, 'UNDER_TEST', False)
I create the UNDER_TEST variable in settings.py like this:
# Create global variable that will tell if our application is under test
UNDER_TEST = (len(sys.argv) > 1 and sys.argv[1] == 'test')
That way - when the application is UNDER_TEST the model is marked as managed (and Pytest will create the appropriate DB table). Then FactoryBoy handles putting all my test data into that table (either in setUp of the test or elsewhere) so I can test against it.
That's my suggestion - others might have something a little more clear or cleaner.

Using both Hive and MySql JDBC drivers

TL;DR:
Is it problematic to use both Hive and MySql JDBC together?
I'm working on an application that performs several SQL queries using the MySql JDBC driver and afterwards it also send another Hive query using Hive JDBC.
Now whats happening is that the MySql queries are working properly, when the code tries to execute the Hive query it throws this exception:
com.mysql.cj.core.exceptions.WrongArgumentException: Connector/J cannot handle a database URL of type 'jdbc:hive2:'.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:54)
at com.mysql.cj.core.conf.url.ConnectionUrl$Type.fromValue(ConnectionUrl.java:149)
at com.mysql.cj.core.conf.url.ConnectionUrl.getConnectionUrlInstance(ConnectionUrl.java:193)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:195)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at company.services.HiveV2Provider.createConnection(HiveProvider.scala:105)
at company.services.HiveProvider$class.loanConnection(HiveProvider.scala:66)
Now after this exception is thrown the query is executed properly.
My guess is that since I'm loading both the MySql and Hive drivers, the MySql driver is trying to run this query first but when it encounters the Hive URL it throws this exception and then the Hive driver sees it and executes the query properly
This is how I execute the MySql code:
val query = ... // query is created here
var mysqlConn: Connection = null
var stmt: Statement = null
try {
Class.forName("com.mysql.jdbc.Driver")
mysqlConn = DriverManager.getConnection(mysqlAddress, username, password)
stmt = mysqlConn.createStatement()
val rs = stmt.executeQuery(query)
val returnVal = someResultSetHandlingFunction(rs)
rs.close()
returnVal
} catch {
case NonFatal(e) =>
logWarning(s"Failed to execute query on: $mysqlAddress", e)
throw e
} finally {
if (mysqlConn != null) {
mysqlConn.close()
}
}
My Hive code looks the same only with a driver name of: org.apache.hive.jdbc.HiveDriver (and it communicates with jdbc:hive2://someurl)
Versions:
Hive is hive-jdbc-1.1.0-cdh5.7.1
MySql is mysql-connector-java 6.0.4
Does anybody know if there's any way to avoid receiving this exception? Is it problematic to load 2 different JDBC drivers? Reading in other somewhat similar questions I get the impression that this should not be a problem
Just a few clarifications:
I know its probably not the best thing to use JDBC directly but I'm checking something and JDBC is fine for this task
I'm using Scala but I don't think it matters for this issue
Thanks in advance
I almost forgot to answer my question
So the issue is probably related to this bug. Back when I was facing this issue I didn't notice it was just a stack trace print and not an actual failure so it was less problematic than I expected.
Anyway, I then saw that in some specific versions this issue was fixed as you can see here so I just changed my mysql version to 5.1.9 (because I didn't need the higher version for anything specific) and the stack trace failure was gone.
I'll be happy if to hear if someone has a more elegant solution to this
Cheers
I was facing the same issue with MS SQL Server JDBC Driver. The same error was logged but everything worked fine.
According to this Microsoft page:
In the JDBC API 4.0, the DriverManager.getConnection method is
enhanced to load JDBC drivers automatically. Therefore, applications
do not need to call the Class.forName method to register or load the
driver when using the sqljdbc4.jar, sqljdbc41.jar, or sqljdbc42.jar
class library.
So I tried removing the Class.forName and just called DriverManager.getConnection. Things are just working and I'm not getting the annoying error anymore.
I believe the Driver itself must include a "META-INF/services/java.sql.Driver" file that registers itself as a valid JDBC driver, so not necessarily it will work for you, but to SQL Server Driver users it's the way to go.
BTW: I noticed that DriverManager.getConnection takes significant more time (6 or 7 seconds) to load the Drive the first time it's called. Subsequent calls are OK. Depending on your application it may be an issue.
Class.forName("com.mysql.jdbc.Driver")
will register your JDBC driver in the DriverManager. Then you put hive connection uri in
DriverManager.getConnection(mysqlAddress, username, password)
Exception is expected in this case.
Why don't you delegate the call to particular JDBC driver after checking uri like:
if (uri.contains("hive")){
//call Hive JDBC
}
else if (uri.contains("mysql")){
//call Mysql JDBC
}

Spring boot datasource is down with exception

I've setup spring boot so under /health to display the datasource status. However, I'm getting this JSON back:
"dataSource" : {
"status" : "DOWN",
"database" : "MySQL",
"error" : "org.springframework.dao.TransientDataAccessResourceException: StatementCallback; SQL [SELECT 1]; Conversion not supported for type java.lang.Object; nested exception is java.sql.SQLException: Conversion not supported for type java.lang.Object"
},
As you can see my database is mysql and I'm running it on Windows8. I did try the 'select 1' query and it did return 1 in the command line of mysql. Any idea where the problem might be?
DataSourceHealthIndicator runs its configured query (SELECT 1 by default) by calling JdbcTemplate:
this.jdbcTemplate.queryForObject(query, Object.class)
On Java 6, this ultimately leads to a call to ResultSet.getObject(index) and things work as expected. On Java 7, the call is to ResultSet.getObject(index, Object.class) (this overload of getObject is new in Java 7). In its default configuration the MySQL JDBC driver is unable to create and return an instance of java.lang.Object so it throws the exception described in the question.
You can change MySQL's behaviour and have it return an instance of whatever is appropriate for the column by enabling auto-deserialization:
spring.datasource.url: jdbc:mysql://localhost/test?autoDeserialize=true
I've also opened Spring boot issue so that we can make a change so that enabling auto-deserialization isn't necessary.

Connecting to different mysql schemas selected at runtime in a webserver using JPA EntityManagerFactory (JSE,EclipseLink,Tomcat), is it possible?

Is there a way to recreate an EntityManagerFactory already used in a webapplication at runtime? What I want is tell the entityManagerFactory to forget the last database connection and connect to a new schema at runtime when the webuser selects an other database (mysql schema). Or not forget the already used one but also use a new connection to a different mysql schema which was not used yet. These schemas have the exact same structure (tables, etc.) but for different users for security and other reasons. Is this possible?
I am using the Vaadin framework, Eclipselink 2.4.1, Mysql 5.5 and Tomcat 7. What I found related to my situation and what I tried already are followings. I am using an Eclipselink composite persistence unit, the second member is always the same, I want to change the first schema for example to "42_candidate" when the user selects the 42nd candidate on the webpage after an other schema was already connected like "41_candidate".
private static EntityManager createEntityManagerForCandidateSchema(String candidateSchemaName) throws javax.persistence.PersistenceException {
System.out.println("createEntityManagerForCandidateSchema called");
// set persistence unit properties
HashMap<String,String> candidatePuProps = new HashMap<String,String>();
candidatePuProps.put("javax.persistence.jdbc.url", "jdbc:mysql://localhost:3306/"+candidateSchemaName+"?useUnicode=true&characterEncoding=UTF-8");
HashMap<String,Map> compositePuProps = new HashMap<String,Map>();
compositePuProps.put("election_model_candidate", candidatePuProps);
Map puProps = new HashMap();
puProps.put("eclipselink.logging.level", "FINEST");
puProps.put("eclipselink.composite-unit.properties", compositePuProps);
// puProps.put(PersistenceUnitProperties.SESSION_CUSTOMIZER, "com.beharbe.ui.ElectionSessionCustomizer");
boolean candidateDatabaseSchemaNotFound = false;
try {
EntityManagerFactory emf = javax.persistence.Persistence.createEntityManagerFactory("election_composite_pu",puProps);
emf.close(); // to forget latest things
emf = javax.persistence.Persistence.createEntityManagerFactory("election_composite_pu",puProps);
EntityManager em = emf.createEntityManager(compositePuProps);
...
public void selectionChanged(int newCandidatePersonId) {
entityManager = Util.createEntityManagerForCandidateSchema(newCandidatePersonId);
...
(Eclipselink Composite PU)
wiki.eclipse.org/EclipseLink/UserGuide/sandbox/gelernter/Composite_Persistence_Units#Persistence_Unit_Properties
(Dynamic Persistence)
dev.eclipse.org/svnroot/rt/org.eclipse.persistence/branches/2.1/trunk/examples/jpa.employee/eclipselink.example.jpa.employee.dynamic/src/example/Main.java
(EclipseLink - How to configure Database Schema name at runtime)
www.rqna.net/qna/kxvmwy-jpa-eclipselink-how-to-configure-database-schema-name-at-runtime.html
(Eclipselink with Tomcat tutorial)
wiki.eclipse.org/EclipseLink/Examples/JPA/Tomcat_Web_Tutorial#Session_Customizer
(Using JPAContainer with Hibernate (Vaadin))
vaadin.com/book/-/page/jpacontainer.hibernate.html
(How can I make a JPA application access different databases?)
stackoverflow.com/questions/9315593/how-can-i-make-a-jpa-application-access-different-databases
(Connect two or more databases dynamically)
stackoverflow.com/questions/9732750/connect-two-or-more-databases-dynamically?lq=1
(JPA - Using Multiple data sources to define access control)
www.rqna.net/qna/kqqihk-jpa-using-multiple-data-sources-to-define-access-control.html
JPA2 run-time database connection
JPA - EclipseLink - How to configure Database Schema name at runtime (Eclipselink SessionCustomizer)
Maybe I should do it somehow with the EclipseLink SessionCustomizer? (see latest link)
thanks for any help in advance
Meanwhile I found something, this can be what I have to use:
http://wiki.eclipse.org/EclipseLink/Examples/JPA/Auditing
I am trying this way but it still connects to the schema which was connected first when the EMF called first time:
....
candidatePuProps.put(PersistenceUnitProperties.JDBC_URL, "jdbc:mysql://localhost:3306/"+candidateSchemaName+"?useUnicode=true&characterEncoding=UTF-8");
candidatePuProps.put(PersistenceUnitProperties.EXCLUSIVE_CONNECTION_MODE, "Always");
candidatePuProps.put(PersistenceUnitProperties.EXCLUSIVE_CONNECTION_IS_LAZY, "false");
...
If you want to access two different databases/schemas, you can just call createEntityManagerFactory() passing a properties map with your new connection.
To set the schema with EclipseLink in code you can set the tableQualifier in a Customizer. To get a new EntityManagerFactory you can pass the "eclipselink.session-name" property.
Your code looks correct. What version of EclipseLink are you using?
Remove the composite persistence unit, I think you seem to just want to connect to a different database. Composite persistence units are for when you have relationships across databases.