I'm having a small problem with my insert statements in Spring-data/jpa/hibernate/mysql/junit application. I created a small unit test to see if i can do a simple INSERT statement.
This is what i have:
#Test
public void testAddAssignment() {
Assignment assignment = new Assignment();
assignment.setCharacter("Z");
assignment.setType(1);
assignmentController.addAssignment(assignment);
assignment = assignmentController.findAssignmentById(16);
assertEquals(16, assignment.getId());
assertEquals(1, assignment.getType());
assertEquals("Z", assignment.getCharacter());
}
When i try to execute this I get the following error:
org.springframework.dao.InvalidDataAccessResourceUsageException: could not insert: [com.byronvoorbach.domain.Assignment]; SQL [insert into Assignment (CHARACTER, TYPE) values (?, ?)]; nested exception is org.hibernate.exception.SQLGrammarException: could not insert: [com.byronvoorbach.domain.Assignment]
And this is my addAssignment(assigment);
#Transactional
public Assignment addAssignment(Assignment assignment) {
return repository.save(assignment);
}
And this is my repository:
#Transactional(readOnly = true)
public interface AssignmentRepository extends JpaRepository<Assignment, Long> {
public List<Assignment> findByType(int type);
}
Since i never wrote an sql query I'm unable to find where the query gets messed up.
Does anyone have an idea?
(Let me know if you need more source code or my .xml files)
EDIT :
Thanks to JB Nizet and fmucar I was able to fix this problem. I changed the column names and now it works!! Apperantly there are a few limitations to column naming which I wasn't aware of.
Try using something other than character and type as column names, because these are probably reserved keywords in SQL.
CHARACTER and TYPE are keywords for sql.
Try not to use keywords as columns names. That may be the problem.
Related
I would like to be able to insert a value into a table into r2dbc using the #Query designation (as opposed to the built in save function. However I don't know what the return type is for the SQL statement. I've tried String, Book, List<Book> Unit, and Void.
I have two types of errors. When I try and use a Book return value (assuming that the SQL statement will return the value of the inserted row) I get -
java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
If I attempt to use a string I get the following. Apparently String return types are not compatible with r2dbc. There is a springboot persistable package (import org.springframework.data.domain.Persistable) which I've attempted to return a Persistable<String>, but that has not worked.
org.springframework.data.mapping.MappingException:
Couldn't find PersistentEntity for type class java.lang.String!
What I think is happening is that what is being returned is attempting to write to an index of some type that is neither a Book value, String, or some nullable type - ie the SQL message that is returned in a standard mysql interface OK, 1 row inserted into Booksor some such. I don't care about the insert value, but I can't call the function without knowing what that is so I can assign a type.
Here is what I have so far. Another question that came up while I've been working on this - is it possible to call a functions return type? I found this abstract class definition here https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.reflect/-k-callable/return-type.html but no pragmatic examples so far. Thanks for the help~
package platypus.bookstore.repos.books
import org.springframework.data.repository.CrudRepository;
import org.springframework.data.repository.kotlin.CoroutineCrudRepository;
import org.springframework.data.r2dbc.repository.Query
import org.springframework.data.repository.query.Param
import platypus.bookstore.classes.db.books.Books
import platypus.bookstore.classes.general.ResultString
import org.springframework.stereotype.Repository
#Repository
interface BookRepository : CoroutineCrudRepository<Books, Long> {
#Query("""
INSERT INTO BOOKS (title, subtitle, publisher, currentcopyright, authorbio, synopsis, isbn)
values (:title, :subtitle, :publisher, :currentcopyright, :authorbio, :synopsis, :isbn);
""")
suspend fun addBook(title: String, subtitle: String, publisher: String,
currentcopyright: String, authorbio: String, synopsis: String, isbn: String
):???RETURNTYPE???
#Query("""
select * from books
""")
suspend fun findBooks():List<Books>
}
EDIT - I have a working implementation with a save function which I've now confirmed, so this isn't a huge deal. But...I think that this should be looked into and either fixed or more documentation available. I don't like having to rely on "magic" in order to make a simple SQL query into a database. Thanks!
SOLVED! - Thanks to Mark over at the spring-data-r2dbc github (https://github.com/spring-projects/spring-data-r2dbc/issues/618#event-4981291976) this issue has been solved. Here is the implementation for anyone that encounters a similar issue.
Basically you just have to use the #Modifying tag to write raw SQL -
Calling from the controller -
suspend fun addBook(book: Books):List<Books>{
println("addBooks")
// bookRepo.save(book);
bookRepo.saveabook(
book.title, book.subtitle, book.publisher,
book.currentcopyright, book.bookedition,
book.uniqueid, book.authorbio, book.synopsis, book.isbn
)
var bookList:List<Books> = findBooks()
return bookList;
}
Inside the repository -
import org.springframework.data.r2dbc.repository.Query
import org.springframework.data.r2dbc.repository.Modifying
#Repository
interface BookRepository : CoroutineCrudRepository<Books, Long> {
// The result of a modifying query can be:
// Void (or Kotlin Unit) to discard update count and await completion.
// Integer or another numeric type emitting the affected rows count.
// Boolean to emit whether at least one row was updated.
#Modifying
#Query(
"""
insert into books (title, subtitle, publisher, currentcopyright, bookedition, uniqueid, authorbio, synopsis, isbn) values (:title, :subtitle, :publisher, :currentcopyright, :bookedition, :uniqueid, :authorbio, :synopsis, :isbn)
""")
suspend fun saveabook(title:String, subtitle:String, publisher:String, currentcopyright:String, bookedition:String, uniqueid:String, authorbio:String, synopsis:String, isbn:String):Void
#Query("""
select * from books
""")
suspend fun findBooks():List<Books>
}
I have one very annoying issue, i have read all existing documentation online and read all stackoverflow questions and answers related to this topic, but simply can not make this to work!
I am really desperate and i do not know what i am missing so i will try to give you all what i have so far. Basically what i am trying to do is to save a lot of data with one query instead of multiple queries for each object. As you can suspect i am using Spring Boot, Hibernate and MySql.
So basic facts that i have learned so far based on what i read related to "batch insert using mysql + hibernate" is next:
Mysql does not support Sequence ID, so i can not use it, like i could use it for PostgreSql
Hibernate does not support batch insert out of the box, there are couple of app properties that needs to be added
And this is what i have so far:
Application properties that i added:
spring.datasource.url=jdbc:mysql://localhost:32803/db?rewriteBatchedStatements=true
spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
spring.jpa.open-in-view=false
spring.jpa.properties.hibernate.show_sql=true
spring.jpa.properties.hibernate.use_sql_comments=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.batch_versioned_data=true
spring.jpa.properties.hibernate.id.new_generator_mappings=false
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.type=trace
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.jpa.properties.hibernate.current_session_context_class=org.springframework.orm.hibernate5.SpringSessionContext
#Entity
data class Person (
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
val id: Long?,
var firstName: String,
var lastName: String,
var country: String,
var org: Int
)
What i want is to save a lot of Persons at once, as you can see i added batch size 50, if i understood correctly that means i will do one database hit per 50 persons while saving. (correct me if i am wrong)
And at the end i have Repository where i execute that batch insert:
#Repository
class PersonRepositoryCustomImpl : PersonRepositoryCustom {
#PersistenceContext
private val entityManager: EntityManager? = null
#Transactional
override fun batchSave2(persons: Set<Person>) {
val session = entityManager!!.unwrap(Session::class.java)
persons.forEachIndexed { index, person ->
if ( index % 50 == 0 ) {
session!!.flush()
session.clear()
}
session!!.save(person)
}
session.close()
}
#Transactional
override fun <T : Person?> batchSave(entities: Collection<T>): Collection<T>? {
val savedEntities: MutableList<T> = ArrayList(entities.size)
var i = 0
for (t in entities) {
savedEntities.add(persistOrMerge(t))
i++
if (i % 50 == 0) { // Flush a batch of inserts and release memory.
entityManager!!.flush()
entityManager.clear()
}
}
return savedEntities
}
private fun <T : Configuration?> persistOrMerge(t: T): T {
return if (t!!.id == null) {
entityManager!!.persist(t)
t
} else {
entityManager!!.merge(t)
}
}
}
So here you can see that i have tried to make this works on 2 almost the same ways, but of course both of them seems not to work.
In order to confirm that i am actually doing batch insert i am looking at this:
https://tableplus.com/blog/2018/10/how-to-show-queries-log-in-mysql.html
so basically that should show me queries that are being executed on DB, and there i can see that for each person object i have one insert statement.
Basically that result of this query:
SELECT
*
FROM
mysql.general_log;
And there i can clearly see that i have multiple insert statements that do one query per object (person).
Edit:
https://blog.arnoldgalovics.com/configuring-a-datasource-proxy-in-spring-boot/
I have also implemented datasource proxy, which proved me that i am not doing batch insert:
Name:, Time:1, Success:True, Type:Prepared, Batch:False, QuerySize:1, BatchSize:0, Query:["insert into person(firstName, lastName, country, org) values (?, ?, ?, ?)"], Params:[(10,John,Johny,USA,ORG)]
i have multiple of records like this one.
Thanks in advance for any kind of help!
Just to give an answer in case someone needs it:
So long story short i was not able to make MySql + hibernate batch processing work, for sake of testing i actually was able to make it work with PostgreSQL.
But anyway if anyone needs this with MySql there is a way using JDBC batch processing, and code more or less is very straight forward:
private String INSERT_SQL_PARAMS = "INSERT INTO item_params(p_key, p_value, item_id) values (?,?,?)"
override fun saveParams(configParams: Set<ItemParam>) {
jdbcTemplate!!.batchUpdate(INSERT_SQL_PARAMS , configParams, 3000) { ps, argument ->
ps.setLong(1, argument.pKey)
ps.setString(2, argument.pValue)
ps.setString(3, argument.itemId)
}
}
mysql doesn't support sequences so you can generate one using this strategy in your entity class:
#Id
#GeneratedValue(generator = "voucher_sequence")
#GenericGenerator(name = "voucher_sequence",strategy = "increment")
and also in your mysql url enable batching:
jdbc:mysql://localhost:3306/db?rewriteBatchedStatements=true
and in your application.properties:
spring.jpa.properties.hibernate.jdbc.batch_size=20
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
then use saveAll() method to persist list of entities
I am trying to do an easy search on a table that can be on any kind of database. The following query is working an the most databases, but I cannot find a solution which works on mysql.
The tables in my database are generated by the active objects framework, so I cannot change the names or config of those instances.
Here is the query that works fine on all databases but MySQL:
select * from "AO_69D057_FILTER" where "SHARED" = true AND "CONTAINS_PROJECT" = true AND UPPER("FILTER_NAME") like UPPER('%pr%').
MySql is not able to use the table name in double quotes for some reason. If I use the unquoted table name it works on MySQL but not on Postgres. Postgres is converting the table name to lowercase because it is unquoted. AO is generating the table names in upper case.
I also tried to use an alias, but that can not work because of the evaluation hierarchy of the statement.
Any suggestions how to get rid of the table name problem?
By default double quotes are used to columns.
You can change it:
SET SQL_MODE=ANSI_QUOTES;
Here is the documentation about it:
http://dev.mysql.com/doc/refman/5.7/en/sql-mode.html
I had the same problem. I select the query according to the exception I get. In the first call of the db search, I try without quotes if it fails then I try with quotes. Then I set useQueryWithQuotes variable accordingly so that in future calls I do not need to check the exception. Below is the code snipped I am using.
private Boolean useQueryWithQuotes=null;
private final String queryWithQuotes = "\"OWNER\"=? or \"PRIVATE\"=?";
private final String queryWithoutQuotes = "OWNER=? or PRIVATE=?";
public Response getReports() {
List<ReportEntity> reports = null;
if(useQueryWithQuotes==null){
synchronized(this){
try {
reports = new ArrayList<ReportEntity>( Arrays.asList(ao.find(ReportEntity.class, Query.select().where(queryWithoutQuotes, getUserKey(), false))) );
useQueryWithQuotes = false;
} catch (net.java.ao.ActiveObjectsException e) {
log("exception:" + e);
log("trying query with quotes");
reports = new ArrayList<ReportEntity>( Arrays.asList(ao.find(ReportEntity.class, queryWithQuotes, getUserKey(), false)));
useQueryWithQuotes = true;
}
}
}else{
String query = useQueryWithQuotes ? queryWithQuotes : queryWithoutQuotes;
reports = new ArrayList<ReportEntity>( Arrays.asList(ao.find(ReportEntity.class, query, getUserKey(), false)));
}
...
}
I'm using Jdbctemplate and I need the inserted id of a query.
I read that I have to build a particular PreparedStatement and use GeneratedKeyHolder object.
The problem is that in my application all inserts method uses this JdbcTemplate update method:
getJdbcTemplate().update(SQL_INSERT,param1,param2,param3,...);
Is there another way to get the inserted id without refactoring all daos?
Looking at the documentation for NamedParameterJdbcTemplate and JdbcTemplate You have two choices:
use NamedParameterJdbcTemplate's update method.
use JdbcTemplate's update method.
There are also some other methods available which will populate the keys to the given GeneratedKeyHolder, it's up to you which one suits your needs.
EDIT
For e.g. using JdbcTemplate:
GeneratedKeyHolder holder = new GeneratedKeyHolder();
jdbcTemplate.update(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection con) throws SQLException {
PreparedStatement statement = con.prepareStatement("INSERT INTO SOME_TABLE(NAME, VALUE) VALUES (?, ?) ", Statement.RETURN_GENERATED_KEYS);
statement.setString(1, "SomeName");
statement.setString(2, "SomeValue");
return statement;
}
}, holder);
long primaryKey = holder.getKey().longValue();
This does not work for PostgreSQL, the holder returns all the attributes. Here is a link to the solution in PostgreSQL Spring: How to use KeyHolder with PostgreSQL
I have the following JPA SqlResultSetMapping:
#SqlResultSetMappings({
#SqlResultSetMapping(name="GroupParticipantDTO",
columns={
#ColumnResult(name="gpId"),
#ColumnResult(name="gpRole"),
// #ColumnResult(name="gpRemarks")
}
)
Which is used like this:
StringBuilder sbQuery = new StringBuilder("Select ");
sbQuery.append(" gpId, ");
sbQuery.append(" gpRole, ");
// sbQuery.append(" gpRemarks ");
sbQuery.append(" FROM v_group_participants_with_details ");
Query query = em.createNativeQuery(sbQuery.toString(), "GroupParticipantDTO");
The view is like this:
DROP VIEW IF EXISTS `v_group_participants_with_details`;
CREATE VIEW `v_group_participants_with_details`
AS
SELECT
gp.id AS gpId,
gp.role AS gpRole,
gp.remarks AS gpRemarks
FROM GroupParticipation gp
;
The GroupParticipation table has the remarks column defined as LONGTEXT (I'm using Mysql 5.x)
Now for the problem:
When the remarks field is commented out from the query everything works perfectly, but if I try to include the remarks field in the query, I get the following error:
javax.persistence.PersistenceException: org.hibernate.MappingException:
No Dialect mapping for JDBC type: -1
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException
(AbstractEntityManagerImpl.java:614)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:76)
What gives? How can I get a LONGTEXT column from a native query?
This problem is reported in HHH-1483 and HHH-3892. In short, Hibernate does not know, how to map a LONGVARCHAR column returned by a native query.
This problem is fixed in Hibernate 3.5.0+. For previous versions, a workaround would be to extend the MysqlDialect to register the correct Hibernate Type for a LONGVARCHAR:
import java.sql.Types;
import org.hibernate.Hibernate;
public class MyMySQL5Dialect extends org.hibernate.dialect.MySQL5Dialect {
public MyMySQL5Dialect() {
super();
// register additional hibernate types for default use in scalar sqlquery type auto detection
registerHibernateType(Types.LONGVARCHAR, Hibernate.TEXT.getName());
}
}