I have a table called 'otp', I need to delete records that is older than a certain period, let say 5 minutes for now.
I use Spring Framework with hibernate with MySql. I tried all possible ways, writing a '#Query' in the DTO interface, trying the Spring Data query way, I even tried to do a 'Select' on the records. When running in the code it does not select any records nor does it delete any records older than the period asked for. When I copy the 'Query" statement into the MySQL workbench it do work, I only need to put the DB name in front of the table name to get it to work. Below is snippets of the code. I removed the "#Query" statement and kept the Spring Data query.
Thank you in advance for your help.
The Entity or model.
'
#Getter
#Setter
#NoArgsConstructor
#Entity
#Table(name = "otp")
public class Otp {
#Id
#Column(name="phoneNumber")
private String phoneNumber;
#Column(name="otp")
private String otp;
#Column(name="createdOn", nullable = false, updatable = false)
#CreationTimestamp
private Timestamp createdOn;
}
`
Code that must delete the rows.
'
#Repository
public interface OTPdto extends CrudRepository<Otp, String> {
#Modifying
#Transactional
void deleteByCreatedOnLessThan(Object now);
}
'
Code that build the date for the query.
'
Date now = Date.from(Instant.now());
Object param = new java.sql.Timestamp(now.getTime());
otPdto.deleteByCreatedOnLessThan(param);
'
Hard to tell by provided info, but maybe I can help.
First of all, I would make sure that CreatedOn column gets filled in with correct information. Also I would try something like deleteByCreatedOnBefore(java.sql.Timestamp time) instead of deleteByCreatedOnLessThan(Object now). java.sql.Timestamp is not raw timestamp value, so maybe comparasing "LessThan" doesn't work well with that.
I will go another route and just clear the otp field. Seems I need to keep the phone number in the Db. Will use the phone number in future to remove the record.
Related
I have a server running Spring boot + JPA + Hibernate. I am using MySQL database (Using InnoDb engine by default).
The implementation draws inspiration from many articles I had search on Internet.
I have implemented REST API to facilitate building a website dynamically.
I wanted to log all the API requests into a log (audit log). So when the API is called,
I store the request method name and few parameters into auditlog table in MySql.
Just before I return from the API, I store the response as well by updating the same record.
I was reviewing the code logs of Hibernate when I make API requests using the web application client as well as Postman.
I noticed that for every API, it takes on an average 150ms - 200ms for inserts and updates.
This is proving to be costly for APIs which fetch very less information.
So I want to know how I can speed up the inserts so that my inserts/updates take less than 10 -20 ms.
My Auditlog entity is
#Entity
#Table(name="auditlog")
public class AuditLog{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(nullable = false, updatable = false)
#Temporal(TemporalType.TIMESTAMP)
#CreatedDate
private Date created_at;
#Column(nullable = false)
#Temporal(TemporalType.TIMESTAMP)
#LastModifiedDate
private Date updated_at;
#NotBlank
private String methodName;
private String param1;
// Text field with private information like password masked
#Column(length = 65535, columnDefinition = "text")
private String request;
// Text field
#Column(length = 65535, columnDefinition = "text")
private String response;
private Integer result;
... // getters and setters
}
My AuditLogRepository is :
public interface AuditLogRepository extends JpaRepository<AuditLog, Long>{
}
In my REST API controller I am doing the following
...
AuditLog logEntry = new AuditLog();
// set all the values except generated ones like id, created_at and updated_at
logEntry.setMethodName(...);
logEntry.setParam1(...);
logEntry.setRequest(...);
// Save into the table using autowired repoitory
auditLogRepoitory.saveAndFlush(logEntry);
// ... do the operation of the API
// Update the logEntry
logEntry.setResult(...);
logEntry.setResponse(...);
auditLogRepoitory.saveAndFlush(logEntry);
...
Please help me in improving the insert and updates to the table.
Or please help in improving the code so that I can make APIs response faster.
Thanks,
Sri Prad
First tips
if you want to speed up insert/update don't user JpaRepository.save method (notice that saveAndFlush() internally calls save method).
Because JpaRepository.save internal select the entity in order to know if the entity is new or if it exists in database.
Here is the default implementation of jpaRepository.save :
#Transactional
public <S extends T> S save(S entity) {
Assert.notNull(entity, "Entity must not be null.");
if (this.entityInformation.isNew(entity)) {
this.em.persist(entity);
return entity;
} else {
return this.em.merge(entity);
}
}
I think using jdbcTemplate is the best option.
Second tips
when thinking about optimizing the inserts, it is probably useful to think about doing bulk inserts. According to mysql documentation website , The time required for inserting a row is determined by the following factors, where the numbers indicate approximate proportions:
Connecting: (3)
Sending query to server: (2)
Parsing query: (2)
Inserting row: (1 × size of row)
Inserting indexes: (1 × number of indexes)
Closing: (1)
So you can easily see how bulk insert can help you improve insert speed.
Third tips
You probably need to tune your mysql instance settings as explained in this stackeroverflow anwser
Others options
Make sur you have selected the right ID generation strategy as explained here https://dzone.com/articles/spring-boot-boost-jpa-bulk-insert-performance-by-100x
If your framework allows for it, do
START TRANSACTION
at the beginning of building the page and storing the auditing. And
COMMIT
at the end.
I'm trying to make a query to retrieve some data which has been created between two dates (represented as Instant).
Here below an extract from the Entity I'm using:
#Entity
public class HistoricalData {
#Id
#GeneratedValue
private Long id;
#Column
private String name;
#CreationTimestamp
private Instant timestamp;
#Column
private Double price;
}
And the query I've written to retrieve the data between the two Instants;
#Query("select h from HistoricalData h where h.timestamp between :timestampStart and :timestampEnd and upper(name) = upper(:name)")
List<HistoricalData> findHistoricalDataBetween(#NonNull Instant timestampStart, #NonNull Instant timestampEnd, #NonNull String name);
Which produces this SQL query:
select historical0_.id as id1_5_, historical0_.price as price2_5_, historical0_.timestamp as timestam3_5_ from historical_data historical0_ where (historical0_.timestamp between ? and ?) and upper(historical0_.name)=upper(?)
Also I wrote the "hibernate JPA" query just to try but no success:
List<HistoricalData> findHistoricalDataByTimestampAfterAndTimestampBeforeAndName(#NonNull Instant timestampStart, #NonNull Instant timestampEnd, #NonNull String name);
Keep in mind that all the above queries compile correctly and do not throw any exception, they just retrieve nothing from the database
The database I'm using is a latest version of MariaDB and the connector version is the 2.7.2
Also the SpringBoot version I'm using is the 2.5.3
Here is DDL from the table definition (automatically generated from Hibernate):
create table historical_data
(
id bigint not null primary key,
price double null,
timestamp datetime not null,
name varchar not null
);
An this is how the timestamp looks like in the database:
Even though records between those two Instants are present in the database I'm still getting nothing as a result from the query.
Looks like the reason is a time zone.
MySQL driver uses incorrect time zone transformations, using a default local time zone in place of a connection time zone (or vice versa).
Just debug this query inside MySQL driver to have fun and figure out what happens.
You can add parameters to the database URL to see which actual values are passed for the prepare statement
jdbc:mysql://<DATABASE_URL>?logger=com.mysql.cj.log.Slf4JLogger&profileSQL=true
When a run a GET request i get an exception o.h.engine.jdbc.spi.SqlExceptionHelper : Unknown column 'disburseme0_.reason_type' in 'field list' in stack trace even though i have configured the field correctly in the entity class. I have a Spring Boot SOAP interface that is querying a MySql database view. I have assigned one of the unique keys from the parent tables as the view Id in JPA.
Part of my entity class has:
#Entity
#Table(name="disbursement_payload")
public class Disbursement {
#Id
#Column(name="ID")
private long disbursementId;
#Column(name="ReasonType")
private String reasonType;
public long getDisbursementId() {
return disbursementId;
}
public void setDisbursementId(long disbursementId) {
this.disbursementId = disbursementId;
public String getReasonType() {
return reasonType;
}
public void setReasonType(String reasonType) {
this.reasonType = reasonType;
}
I have the view as:
CREATE VIEW disbursement_payload AS (
SELECT
iso_number AS Currency,
trans_desc AS ReasonType,
account_number AS ReceiverParty,
amount AS Amount
FROM m_payment_detail, m_loan_transaction
WHERE m_payment_detail.`id`= m_loan_transaction.`payment_detail_id` AND
m_payment_detail.`payment_type_id`=2
);
Is there something im missing , in the entity or view definition? I have read one of the comments here could not extract ResultSet in hibernate that i might have to explicitly define the parent schemas. Any assistance, greatly appreciated.
do the mapping for db column and class var name based on camelCase conversion basded on underscore _ separated name
you could try using
CREATE VIEW disbursement_payload AS (
SELECT iso_number AS currency
, trans_desc AS reason_type
, account_number AS receiver_rarty
, amount AS amount
FROM m_payment_detail
INNER JOIN m_loan_transaction
ON m_payment_detail.`id`= m_loan_transaction.`payment_detail_id`
AND m_payment_detail.`payment_type_id`=2
);
the view code is SQL code and hibernate see a view as a table, so the conversion of column name is base on the same rules
and a suggestion you should not use (older) implicit join based on where condition you should use (more recent) explici join sintax ..
I'm developing a J2EE 6 Web Application, using a MySql 5.02 DataBase. I'm trying to generate a hash digest of the ID, every time I create a new Entity. This is set to a column on the Table.
But well, I'm stuck on something that is apparently easy, and according with what I found googling, possible. Basically I want to retrieve the ID (Primary Key) of a newly persisted object, but whatever I try, it returns null.
The steps are follow are:
Create the Instance of the Entity -> userCard = new Users();
Setting the corresponding fields with some values.
Calling the persist() method of the EntityManager.
After reading some forums, I understood I had to either call flush() after persist(), or use merge() and use the returned entity to retrieve the id.
public void createAndFlush(Users users) {
em.persist(users);
em.flush();
}
public Integer edit(Users users) {
return ((Users)em.merge(users)).getIdvcards();
}
None of them (among some other combinations) work, well, the Entity is successfully persisted, but the ID field returns null.
These are the correspinding annotations of the id column I want to retrieve:
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Basic(optional = false)
#Column(name = "idvcards")
private Integer idvcards;
And obviously, the column idvcards of my table is set to Auto-Increment and Primary Key.
Maybe I just need to clarify my Peristence-Management concepts, but I'd appreciate if I can get some hint to solve this basic issue.
Maybe is not the most elegant solution ever, but finally I succeded on retrieving the ID of the new Entity:
public Integer create(User user) {
em.persist(users);
em.flush();
return (Integer) em.getEntityManagerFactory().getPersistenceUnitUtil().getIdentifier(users);
}
And well, althought is not related to the functionality, I changed the entity name to the singular form as #Bohemian suggested.
Try this, it works for me:
#Id
#GeneratedValue
#Column(name = "id", unique = true, nullable = false)
Also, I don't use the value returned from merge. I just persist then flush and the entity object magically gets the new key value.
p.s. Tables should never be named in the plural, and especially not the entity class. Call the class User (and the table if you can). Otherwise it's just confusing: Users sounds like a collection of User.
It will work fine
em.persist(usmApproveTransaction);
em.flush();
System.out.println("++++++e++++++++"+usmApproveTransaction.getUatApproveTransIdPk());
return usmApproveTransaction;
I'm getting id using em.flush(); after persist
++++++e++++++++51472
I am using JPA (Hibernate) to store an entity on a MySQL 5.0 server.
Here is the entity in simplified form:
#Entity
#Table(name = "messages")
public class Message implements Serializable
{
#Id
#GeneratedValue
#Column
private long id;
#Column
private String content;
#Column(insertable = false)
private Date read;
#Column(insertable = false)
private Date deleted;
}
The columns "read" and "deleted" in table "messages" are defined so that they can contain NULL values. When I first tried to persist one of these entities I got an exception. Apparently, what Hibernate was doing there was listing the "read" and "deleted" columns in the column list of the insert statement but not in the value list. I got around that problem with the "insertable = false" statement in the #Column annotations you see above.
Now, however, I have a bigger problem. I want to set the read or date fields to non-null values. When I do that, I get a similar exception "You have an error in your SQL syntax". What he is doing now is listing all fields in the where part of the update statement, including "read" and "deleted". And what he does is check like "...and read=NULL". Which, in MySQL, of course should be "...and read IS NULL".
Rummaging around, I already found the "updatable" parameter for the #Column annotation. But if I set that to false, both "read" and "deleted" are never updated at all, so that is not what I am looking for, either.
...Help?
read is a reserved word in mysql and it looks like the driver is not escaping the name with back quotes. I think the best solution is to rename the column.