I am using Spring Data JPA (1.7.2-RELEASE) in combination with Hibernate (4.3.8.Final) and MySQL (5.5). I want to manage two entities in a bidirectional assosciation. The save and update of the enties works fine, but the deletion doesn't work.
#Entity
public class Beacon extends AbstractEntity {
#OneToMany(fetch = FetchType.EAGER, mappedBy = "beacon", cascade = ALL)
private Set<Comment> comments;
/**
* #return the comments
*/
public Set<Comment> getComments() {
return comments;
}
/**
* #param comments the comments to set
*/
public void setComments(Set<Comment> comments) {
this.comments = comments;
}
}
and
#Entity
public class Comment extends AbstractEntity {
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "beacon_id")
private Beacon beacon;
public Beacon getBeacon() {
return beacon;
}
public void setBeacon(Beacon beacon) {
this.beacon = beacon;
}
}
Having a beacon with comments stored in the database, I want to delete the comment but it doesn't work. I don't get an exception but the entity is still present in the database.
This is my unit test:
#Test
public void deleteWithStrategyCheck() {
Beacon beacon = this.beaconRepository.save(createBeacon());
Comment comment = this.commentRepository.save(createEntity());
comment.setBeacon(beacon);
comment = this.commentRepository.save(comment);
this.commentRepository.delete(comment.getId());
assertThat(this.commentRepository.exists(comment.getId())).isFalse();
assertThat(this.beaconRepository.exists(beacon.getId())).isTrue();
assertThat(this.beaconRepository.findOne(beacon.getId()).getComments()).doesNotContain(comment);
}
If I delete the comment via a sql statement it works.
You need to add orphanRemoval = true to your #OneToMany mappings, and remove the Comment from the parrent beacon.
If you delete the Comment without removing it from the parrent collection you should actually get the exception (unless you are not using InnoDB storage engine, (and ou should)).
beacon.getComments().remove(comment),
will do the work then. (with orphanRemoval you don't need to call EM.remove(comment). Without it, you need to remove the comment from the collection and call EM.remove(comment).
Related
I wrote an application (Springboot + Data JPA + Data Rest) that keeps throwing OutOfMemoryException at me when the application loads. I can skip that code that runs on application start but then the exception may happen later down the road. It's probably best to show you what happens on application start because it's actually super simple and should not cause any problems imho:
#SpringBootApplication
#EnableAsync
#EnableJpaAuditing
public class ScraperApplication {
public static void main(String[] args) {
SpringApplication.run(ScraperApplication.class, args);
}
}
#Component
#RequiredArgsConstructor(onConstructor = #__(#Autowired))
public class DefaultDataLoader {
private final #NonNull LuceneService luceneService;
#Transactional
#EventListener(ApplicationReadyEvent.class)
public void load() {
luceneService.reindexData();
}
}
#Service
#RequiredArgsConstructor(onConstructor = #__(#Autowired))
public class LuceneService {
private static final Log LOG = LogFactory.getLog(LuceneService.class);
private final #NonNull TrainingRepo trainingRepo;
private final #NonNull EntityManager entityManager;
public void reindexData() {
LOG.info("Reindexing triggered");
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(entityManager);
fullTextEntityManager.purgeAll(Training.class);
LOG.info("Index purged");
int page = 0;
int size = 100;
boolean morePages = true;
Page<Training> pageData;
while (morePages) {
pageData = trainingRepo.findAll(PageRequest.of(page, size));
LOG.info("Loading page " + (page + 1) + "/" + pageData.getTotalPages());
pageData.getContent().stream().forEach(t -> fullTextEntityManager.index(t));
fullTextEntityManager.flushToIndexes(); // flush regularly to keep memory footprint low
morePages = pageData.getTotalPages() > ++page;
}
fullTextEntityManager.flushToIndexes();
LOG.info("Index flushed");
}
}
You can see what I am doing is clear out the index, read all Trainings from the TrainingRepo in a paged way (100 at a time) and write them into the index. Not much going on actually. A few minutes after the "Index purged" message I get this - and only this:
java.lang.OutOfMemoryError: Java heap space
In the logs I get to see "Index purged" but never see any "Loading page ..." message, so it must be stuck on the findAll() call.
I had the JVM write a heap dump and loaded it into Eclipse Memory Analyzer and got a full stack trace: https://gist.github.com/mathias-ewald/2fddb9762427374bb04d332bd0b6b499
I also looked around the report a bit, but I need help interpreting this information which is why I attached some screenshots from Eclipse Memory Analyzer.
EDIT:
I just enabled "show-sql" and saw this before everything hung:
Hibernate: select training0_.id as id1_9_, training0_.created_date as created_2_9_, training0_.description as descript3_9_, training0_.duration_days as duration4_9_, training0_.execution_id as executi14_9_, training0_.level as level5_9_, training0_.modified_date as modified6_9_, training0_.name as name7_9_, training0_.price as price8_9_, training0_.product as product9_9_, training0_.quality as quality10_9_, training0_.raw as raw11_9_, training0_.url as url12_9_, training0_.vendor as vendor13_9_ from training training0_ where not (exists (select 1 from training training1_ where training0_.url=training1_.url and training0_.created_date<training1_.created_date)) limit ?
Hibernate: select execution0_.id as id1_1_0_, execution0_.created_date as created_2_1_0_, execution0_.duration_millis as duration3_1_0_, execution0_.message as message4_1_0_, execution0_.modified_date as modified5_1_0_, execution0_.scraper as scraper6_1_0_, execution0_.stats_id as stats_id8_1_0_, execution0_.status as status7_1_0_, properties1_.execution_id as executio1_2_1_, properties1_.properties as properti2_2_1_, properties1_.properties_key as properti3_1_, stats2_.id as id1_5_2_, stats2_.avg_quality as avg_qual2_5_2_, stats2_.max_quality as max_qual3_5_2_, stats2_.min_quality as min_qual4_5_2_, stats2_.null_products as null_pro5_5_2_, stats2_.null_vendors as null_ven6_5_2_, stats2_.products as products7_5_2_, stats2_.tags as tags8_5_2_, stats2_.trainings as training9_5_2_, stats2_.vendors as vendors10_5_2_, producthis3_.stats_id as stats_id1_6_3_, producthis3_.product_histogram as product_2_6_3_, producthis3_.product_histogram_key as product_3_3_, taghistogr4_.stats_id as stats_id1_7_4_, taghistogr4_.tag_histogram as tag_hist2_7_4_, taghistogr4_.tag_histogram_key as tag_hist3_4_, vendorhist5_.stats_id as stats_id1_8_5_, vendorhist5_.vendor_histogram as vendor_h2_8_5_, vendorhist5_.vendor_histogram_key as vendor_h3_5_ from execution execution0_ left outer join execution_properties properties1_ on execution0_.id=properties1_.execution_id left outer join stats stats2_ on execution0_.stats_id=stats2_.id left outer join stats_product_histogram producthis3_ on stats2_.id=producthis3_.stats_id left outer join stats_tag_histogram taghistogr4_ on stats2_.id=taghistogr4_.stats_id left outer join stats_vendor_histogram vendorhist5_ on stats2_.id=vendorhist5_.stats_id where execution0_.id=?
Apparently, it creates the statement to fetch all the Training entities but the Execution statement is the last it manages to execute.
I changed the relation from Training to Execution from #ManyToOne to #ManyToOne(fetch = FetchType.LAZY) and suddenly I the code was able to load data into the index again. So I am thinking something might be wrong with my Execution entity mapping. Let me share the code with you:
#Entity
#Data
#EntityListeners(AuditingEntityListener.class)
public class Execution {
public enum Status { SCHEDULED, RUNNING, SUCCESS, FAILURE };
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#ToString.Include
private Long id;
#Column(updatable = false)
private String scraper;
#CreatedDate
private LocalDateTime createdDate;
#LastModifiedDate
private LocalDateTime modifiedDate;
#Min(0)
#JsonProperty(access = Access.READ_ONLY)
private Long durationMillis;
#ElementCollection(fetch = FetchType.EAGER)
private Map<String, String> properties;
#NotNull
#Enumerated(EnumType.STRING)
private Status status;
#Column(length = 9999999)
private String message;
#EqualsAndHashCode.Exclude
#OneToOne(cascade = CascadeType.ALL)
private Stats stats;
}
And since it is a relation of Execution, here's the Stats entity, too:
#Entity
#Data
public class Stats {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#ToString.Include
private Long id;
private Long trainings;
private Long vendors;
private Long products;
private Long tags;
private Long nullVendors;
private Long nullProducts;
private Double minQuality;
private Double avgQuality;
private Double maxQuality;
#ElementCollection(fetch = FetchType.EAGER)
private Map<String, Long> vendorHistogram;
#ElementCollection(fetch = FetchType.EAGER)
private Map<String, Long> productHistogram;
#ElementCollection(fetch = FetchType.EAGER)
private Map<String, Long> tagHistogram;
}
All this is running in a single transaction and I can't see a clear here, so the EntityManager loading all this data still references it.
To fix this inject the EntityManager and invoke clear. Or alternatively make the scope of the transaction the processing of one page.
I recommend the TransactionTemplate for this.
I'm not familiar with the FullTextEntityManager but it might have similar problems.
For more background you might want to read up on the JPA entity lifecycle.
I believe it has to do with your FullTextEntityManager not finding enough memory. You have to configure your queryPlanCache.Go through this thread on how to Stackoverflow and this one too.
I have following relationships between three objects
public class ProductEntity {
#Id
private int id;
#OneToMany(mappedBy = "productEntity",
fetch = FetchType.LAZY)
private List<ProductInfoEntity> productInfoEntityList = new ArrayList<>();
#Column(name = "snippet")
private String snippet;
}
public class ProductInfoEntity {
#Id
private int id;
#ManyToOne
#JoinColumn(name = "product_id")
private ProductEntity productEntity;
#ManyToOne
#JoinColumn(name = "support_language_id")
private SupportLanguageEntity supportLanguageEntity;
}
public class SupportLanguageEntity {
#Id
private int id;
#Column("name")
private String name;
}
And this is actual database design
Then, I'd like to make a specification to query as followings:
select * from product_info
where product_id = 1
and support_language_id = 2;
I am also using annotation for the specification which means that I use ProductEntity_, ProductInfoEntity_ and so on.
Can you please give me a full working code for the specification for query mentioned above?
Thank you guys
To use Specification your ProductInfoEntityRepository have to extend JpaSpecificationExecutor
#Repository
public interface ProductInfoEntityRepository
extends JpaRepository<ProductInfoEntity, Integer>, JpaSpecificationExecutor<ProductInfoEntity> {
}
As far as I understand you use JPA metamodel. So then
#Autowired
ProductInfoEntityRepository repository;
public List<ProductInfoEntity> findProductInfoEntities(int productId, int languageId) {
return repository.findAll((root, query, builder) -> {
Predicate productPredicate = builder.equal(
root.get(ProductInfoEntity_.productEntity).get(ProductEntity_.id), // or root.get("productEntity").get("id")
productId);
Predicate languagePredicate = builder.equal(
root.get(ProductInfoEntity_.supportLanguageEntity).get(SupportLanguageEntity_.id), // or root.get("supportLanguageEntity").get("id")
languageId);
return builder.and(productPredicate, languagePredicate);
});
}
If you want to make specifications reusable you should create utility class contains two static methods productIdEquals(int) and languageIdEquals(int).
To combine them use Specifications(Spring Data JPA 1.*) or Specification(since Spring Data JPA 2.0)
select * from product_info where product_id = 1 and support_language_id = 2;
Should work as written. But the only thing useful will be comment.
Perhaps you want the rest of the info in all three tables?
SELECT pi.comment, -- list the columns you need
p.snippet,
sl.name
FROM product AS p -- table, plus convenient "alias"
JOIN product_info AS pi -- another table
ON p.id = pi.product_info -- explain how the tables are related
JOIN support_language AS sl -- and another
ON pi.support_language_id = sl.id -- how related
WHERE p.snippet = 'abc' -- it is more likely that you will start here
-- The query will figure out the rest.
From there, see if you can work out the obfuscation provided by JPA.
I have a problem with column json when it's necessary to switching from PostgreSQL to MariaDB/MySql.
I use Spring Boot + JPA + Hibernate + hibernate-types-52.
The table i want to map is like this:
CREATE TABLE atable(
...
acolumn JSON,
...
);
Ok it works for PostgreSQL and MariaDB/MySql.
The problem is when i want to deploy an application that switch easly from one to another because the correct hibernate-types-52 implementation for PostgreSQL and MySQL/MariaDB are different
This works on MySQL/MariaDB
#Entity
#Table(name = "atable")
#TypeDef(name = "json", typeClass = JsonStringType.class)
public class Atable {
...
#Type(type = "json")
#Column(name = "acolumn", columnDefinition = "json")
private JsonNode acolumn;
...
}
This works on PosgreSQL
#Entity
#Table(name = "atable")
#TypeDef(name = "json", typeClass = JsonBinaryType.class)
public class Atable {
...
#Type(type = "json")
#Column(name = "acolumn", columnDefinition = "json")
private JsonNode acolumn;
...
}
Any kind of solutions to switch from JsonBinaryType to JsonStringType (or any other solution to solve this) is appreciated.
The Hypersistence Utils project, you can just use the JsonType, which works with PostgreSQL, MySQL, Oracle, SQL Server, or H2.
So, use JsonType instead of JsonBinaryType or JsonStringType
#Entity
#Table(name = "atable")
#TypeDef(name = "json", typeClass = JsonType.class)
public class Atable {
#Type(type = "json")
#Column(name = "acolumn", columnDefinition = "json")
private JsonNode acolumn;
}
That's it!
There are some crazy things you can do - with the limitation that this only works for specific types and columns:
First, to replace the static #TypeDef with a dynamic mapping:
You can use a HibernatePropertiesCustomizer to add a TypeContributorList:
#Configuration
public class HibernateConfig implements HibernatePropertiesCustomizer {
#Value("${spring.jpa.database-platform:}")
private Class<? extends Driver> driverClass;
#Override
public void customize(Map<String, Object> hibernateProperties) {
AbstractHibernateType<Object> jsonType;
if (driverClass != null && PostgreSQL92Dialect.class.isAssignableFrom(driverClass)) {
jsonType = new JsonBinaryType(Atable.class);
} else {
jsonType = new JsonStringType(Atable.class);
}
hibernateProperties.put(EntityManagerFactoryBuilderImpl.TYPE_CONTRIBUTORS,
(TypeContributorList) () -> List.of(
(TypeContributor) (TypeContributions typeContributions, ServiceRegistry serviceRegistry) ->
typeContributions.contributeType(jsonType, "myType")));
}
}
So this is limited to the Atable.class now and I have named this custom Json-Type 'myType'. I.e., you annotate your property with #Type(type = 'myType').
I'm using the configured Dialect here, but in my application I'm checking the active profiles for DB-specific profiles.
Also note that TypeContributions .contributeType(BasicType, String...) is deprecated since Hibernate 5.3. I haven't looked into the new mechanism yet.
So that covers the #Type part, but if you want to use Hibernate Schema generation, you'll still need the #Column(columnDefinition = "... bit, so Hibernate knows which column type to use.
This is where it start's feeling a bit yucky. We can register an Integrator to manipulate the Mapping Metadata:
hibernateProperties.put(EntityManagerFactoryBuilderImpl.INTEGRATOR_PROVIDER,
(IntegratorProvider) () -> Collections.singletonList(JsonColumnMappingIntegrator.INSTANCE));
As a demo I'm only checking for PostgreSQL and I'm applying the dynamic columnDefinition only to a specific column in a specific entity:
public class JsonColumnMappingIntegrator implements Integrator {
public static final JsonColumnMappingIntegrator INSTANCE =
new JsonColumnMappingIntegrator();
#Override
public void integrate(
Metadata metadata,
SessionFactoryImplementor sessionFactory,
SessionFactoryServiceRegistry serviceRegistry) {
Database database = metadata.getDatabase();
if (PostgreSQL92Dialect.class.isAssignableFrom(database.getDialect().getClass())) {
Column acolumn=
((Column) metadata.getEntityBinding(Atable.class.getName()).getProperty("acolumn").getColumnIterator().next());
settingsCol.setSqlType("json");
}
}
#Override
public void disintegrate(SessionFactoryImplementor sessionFactory, SessionFactoryServiceRegistry serviceRegistry) {
}
}
metadata.getEntityBindings() would give you all Entity Bindings, over which you can iterate and then iterate over the properties. This seems quite inefficient though.
I'm also not sure whether you can set things like 'IS JSON' constraints etc., so a custom create script would be better.
Project type :- Spring-boot JPA project
Hi,
I have below Rest service which increments a number in database.
#RestController
public class IncrementController {
#Autowired
MyNumberRepository mynumberRepository;
#GetMapping(path="/incrementnumber")
public String incrementNumber(){
Optional<MyNumber> mynumber = mynumberRepository.findById(1);
int i = mynumber.get().getNumber();
System.out.println("value of no is "+i);
i = i+1;
System.out.println("value of no post increment is "+i);
mynumber.get().setNumber(i);
MyNumber entity = new MyNumber();
entity.setId(1);
entity.setNumber(i);
mynumberRepository.save(entity);
return "done";
}
}
Entity is as below :-
#Entity
#Table(name = "my_number")
public class MyNumber {
#Id
private Integer id;
private Integer number;
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public Integer getNumber() {
return number;
}
public void setNumber(Integer number) {
this.number = number;
}
}
Below is the Repository :-
public interface MyNumberRepository extends JpaRepository<MyNumber, Integer>{
}
The service works well when I call increment number sequentially , but when concurrent threads call the incrementservice then i get non consistent results. How can I handle this situation ?
Also have to deploy the app on multiple places and connecting to same DB. i.e Scalability concern.
Thanks,
Rahul
You must use a pessimistic lock. This will issue a SELECT FOR UPDATE and lock the row for the transaction and it's not possible for another transaction to overwrite the row.
public interface MyNumberRepository extends JpaRepository<MyNumber, Integer> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
Optional<MyNumber> findById(Integer id);
}
And then you have to make your REST method transactional by adding #Transactional
#RestController
public class IncrementController {
#Autowired
MyNumberRepository mynumberRepository;
#Transactional
#GetMapping(path="/incrementnumber")
public String incrementNumber(){
Optional<MyNumber> mynumber = mynumberRepository.findById(1);
int i = mynumber.get().getNumber();
System.out.println("value of no is "+i);
i = i+1;
System.out.println("value of no post increment is "+i);
mynumber.get().setNumber(i);
MyNumber entity = new MyNumber();
entity.setId(1);
entity.setNumber(i);
mynumberRepository.save(entity);
return "done";
}
}
Above solution will work , but i feel you are doing over-engineering for very simple problem.
My recommendation would be to use database sequence.I feel your requirement is quite straight forward.In your service u can simply call getnextvalue on the sequence and then set the value in the Id field.This way u don't have to manage locks also as Database will do that for you.
In oracle particularly sequences are managed in a different transactions . So if ur calling code fails with exception , still the value of sequence will be incremented . This will ensure that multi-threads will not see the same value of the sequence in case of exceptions.
Instead of locking transaction, you could also use an Oracle sequence or MySQL "AUTO_INCREMENT" feature which will prevent any ID being returned twice.
https://community.oracle.com/thread/4156674
Thread safety of MySql's Select Last_Insert_ID
public class ClassYear {
private Set<Examination> examination = new HashSet<Examination>();
#Id
#Column(name = "class_id")
#GeneratedValue(generator = "system-uuid")
#GenericGenerator(name = "system-uuid", strategy = "uuid")
#DocumentId
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
#OneToMany(cascade={CascadeType.ALL},fetch=FetchType.EAGER)
#JoinColumn(name="class_id")
public Set<Examination> getExamination() {
return examination;
}
public void setExamination(Set<Examination> examination) {
this.examination = examination;
}
}
public class Examination {
private ClassYear classYear;
#ManyToOne(cascade={CascadeType.ALL},fetch=FetchType.EAGER)
#JoinColumn(name="class_id")
public ClassYear getClassYear() {
return classYear;
}
public void setClassYear(ClassYear classYear) {
this.classYear = classYear;
}
}
So here, Examination table refers the class_id column of ClassYear table. Now I need the class_id column to be used on few other tables. How should I do it..? It's One to many relationship concept but I need that class_id to be used on many tables (say like HomeWork table). I searched the Internet and found explanations for one to many & many to one concepts, but I can't find exactly how to do it for multiple tables with a particular column in a table as One to Many relationship.
The Examination entity owns the relationship, so you should map the examinations in ClassYear entity (which is the non-owning side of the relationship) using mappedBy like this (without join column):
#OneToMany(mappedBy="classYear", cascade={CascadeType.ALL})
public Set<Examination> getExamination() {
return examination;
}
I removed the EAGER fetch type in this example (the default for #OneToMany is LAZY). I don't know your use case but always eagerly fetching mapped collections like this is not the best idea.
If you need to reference ClassYear in other entities like HomeWork, do it the same way. The HomeWork entity will have #ManyToOne mapping to ClassYear and will be the owning side of the relationship (just like in Examination) and all the homeworks for the ClassYear can be mapped using mappedBy in the ClassYear entity.
Also one small detail:
#ManyToOne(cascade={CascadeType.ALL},fetch=FetchType.EAGER)
#ManyToOne has the EAGER fetch type as default, so you don't have to specify it manually.