I would like to set indexes on multiple columns within a single table in MySQL database. After reading this article, I'm not 100% sure which approach to use.
So my (simplified) table looks like this:
#Data
#Entity
#SuperBuilder
#NoArgsConstructor
#AllArgsConstructor
#Table(name = "loan")
public class Loan {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "loan_id", unique = true, nullable = false)
private long id;
#Column(name = "amount", unique = false, nullable = false)
private double amount;
#Column(name = "rate", unique = false, nullable = false)
private double rate;
#Column(name = "payments", unique = false, nullable = false)
private int payments;
#Column(name = "pmt", unique = false, nullable = false)
private double pmt;
}
I will have a lot of search queries, for instance:
SELECT *
FROM Loan loan
WHERE loan.amount =: amount
AND loan.rate =: rate
AND loan.payments =: payments
AND loan.pmt =: pmt
LIMIT 1;
Now, I would like to index fields in WHERE clause. Essentially, I would like to achieve effect of a "composite key" where in table loan there are only unique combinations of mentioned fields. So I cannot have two rows all with some values.
Is there such a configuration?
ou can add a UNIQUE constraint, which would be indexed automatocally
#Table(uniqueConstraints =
{
#UniqueConstraint(name = "UniqueWhereclause", columnNames = { "amount", "rate","payments","pmt" })})
or you can create an index alone
#Entity
#SuperBuilder
#NoArgsConstructor
#AllArgsConstructor
#Table(name = "loan", indexes = {
#Index(columnList = "amount, rate,payments,pmt", name = "name_idx") })
DOUBLE is likely to cause trouble for you. Switch to DECIMAL with a suitable number of decimal places -- so that = can be tested correctly.
Also, that's a strange combination of 4 things to use in the WHERE. Any 3 of those columns should (mathematically) determine the value for the 4th.
Does the table also have the name of the person taking out the loan? Or is this table a list of possible loans? As rates change, so you add more rows to the table? But why have the table if 3 columns can be used to compute the 4th?
Related
How can I create a Single Entity from two other tables/Entity in JPA/hibernate making a INNER JOIN result with a common key. I was using the below code but it gives me a full join instead of an inner join. it give me records from the meal table even if the
"id 1" does not exist in the allergies table, example:
{id=1, name='tacos', description='Mexican food', price ='10',peanuts=null, celery=null, sesameSeeds=null}
How can constrain to don't return any records if the 'id' is missing from the secondary table allergies? to show only records when the primary key is present in both tables.
I want something like this instead:
{id=1, name='tacos', description='Mexican food', price ='10',peanuts='no', celery='no', sesameSeeds='no'}
Please advise.
#Entity
#Table(name = "meal")
#SecondaryTable(name = "allergens", pkJoinColumns = #PrimaryKeyJoinColumn(name = "meal_id"))
class Meal {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
Long id;
#Column(name = "name")
String name;
#Column(name = "description")
String description;
#Column(name = "price")
BigDecimal price;
#Column(name = "peanuts", table = "allergens")
boolean peanuts;
#Column(name = "celery", table = "allergens")
boolean celery;
#Column(name = "sesame_seeds", table = "allergens")
boolean sesameSeeds;
// standard getters and setters
}
I have the following (simplified) entities:
#Table(name = "groups")
public class Group {
#Column(name = "id")
private long id;
#Column(name = "name")
private String name;
#ManyToMany(fetch = FetchType.LAZY)
private Set<User> users;
...
}
#Table(name = "users")
public class StoredUser extends StoredBase {
#Column(name = "id")
private long id;
#Column(nullable = false, unique = true)
private String username;
#ManyToMany(mappedBy = "users")
private Set<Group> groups;
...
}
So I wanted to get something like list of username group by groupId:
group_id group_name username
--- --- ----
1 gr1 1, 2, 3
2 gr2 4, 5
3 gr3 1, 4
At first I was just using groupRepository.findAll() and convert the response based on that. But the amount of unrelated data coming with Group and User is big and is slowing down the response. So I want to fetch just the related values only.
So I wonder what is the best way to achieve this?
Many thanks
group_concat and a 'group by' should get you there.
Something like (untested, but someone will be along to tell me if I'm being fick):
select g.group_id, g.group_name, group_concat(u.username) as all_users
from groups as g join users as u on g.users = u.id
group by u.id;
You can use projections from Spring Data JPA (doc). In your case, create interface:
interface GroupAndUsers{
String getId();
String getName();
Set<Integer> getUsersId();
}
and add following method to your repository:
List<GroupAndUsers> findAllGroupAndUsers();
I have a slow query I'm trying to speed up:
#Query("select max(t.timestamp) from TradeDbo t where t.currencyPair.id = :currencyPairId")
Date findMaxTimestamp(#Param("currencyPairId") Long currencyPairId);
The entity is defined by:
#Entity()
#Table(name = "Trade",
indexes = { #Index(name = "idx_timestamp_currencypairid",
columnList = "timestamp,currency_pair_id")})
public class TradeDbo extends Auditable<String> {
#Id #GeneratedValue
#Getter private Long id;
#Version
private long version;
#JoinColumn(name = "currency_pair_id")
#ManyToOne(fetch=FetchType.EAGER)
#Getter private CurrencyPairDbo currencyPair;
#Column(name = "timestamp")
#Convert(converter = DateConverter.class)
#Getter private Date timestamp;
...
and, as you can see, I've defined an index on the timestamp/currencypairid, (see how to speed up max() query) which I thought would have made max(timestamp) just a read of the last page of the btree, but it's still taking as long as it did before adding the index.
do your query as
"select max(t.timestamp) from TradeDbo t where t.currencyPair_id = :currencyPairId"
Your composite index is not usefult for you performance you should change the column sequence
columnList = "currency_pair_id,timestamp"
using first the column involved in where condition ..
the index content build using the columns is used reading for left to right
This question relates to managing id numbers to ensure that I don't end up with duplicate identities on my person's table. It is a springboot project with MySQL database.
Some background:
I have an HTML form for submitting an "episode". Each episode contains "persons" and has a relationship to "persons" of ManyToMany.
"episodes" are entered and submitted into the database (db1) by field staff. A few hours later the episode is manually entered into a second database (db2) by BackOffice staff.
On my spring attached database (db1) I have a persons table which has a native auto generated id field. db1 also has a id2 field - which records the unique id for the person from db2.
Field staff do not always have access to id2 when they enter a episode, but BackOffice staff do.
When I save a new "episode" I need the save method to check if person id2 exists in the database and perform an update on person (not create new).
Then delete the duplicate person.
Episode entity:
#Entity
public class Episode {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
...
#Column(name="start_date")
#DateTimeFormat (pattern="dd/MM/yyyy HH:mm")
private Date start_date;
#ManyToMany(cascade = {CascadeType.PERSIST, CascadeType.MERGE})
#JoinTable(name = "episode_person", joinColumns = #JoinColumn(name = "episode_id", referencedColumnName = "id"), inverseJoinColumns = #JoinColumn(name = "person_id", referencedColumnName = "id"))
private List<Person> persons;
#OneToOne(cascade=CascadeType.ALL)
//#JoinColumn(name = "id")
private Address address;
Person Entity
#Entity
public class Person {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private Long id2;
private String surname;
private String firstname;
private String phoneHome;
#DateTimeFormat(pattern = "dd/mm/yyyy")
private Date dob;
#ManyToMany(mappedBy = "persons", cascade = {CascadeType.PERSIST, CascadeType.MERGE})
private List<Episode> episodes;
EpisodeServiceImpl
#Override
#Transactional
public Episode saveEpisode(Episode episode) {
List mergedPeople = personService.mergeDetachedWithAttached( episode.getPersons() );
episode.setPersons( mergedPeople );
return episodeRepository.save(episode);
}
PersonServiceImpl
#Transactional
#Override
public List mergeDetachedWithAttached(List<Person> people) {
final List<Person> results = new ArrayList<>();
if ( people != null && !people.isEmpty() ) {
// loop over every person
for (Person person : people) {
// for current person try to retrieve from db via Id2
Person db2Person = personRepository.findById2( person.getId2() );
// if a person was retrieved use them instead of submitted person
if (db2Person != null ) {
System.out.println("A matching person was found in the db using id2 - this is the person that will be added");
results.add(db2Person);
} else {
results.add(person);
}
}
}
return results;
The way this is written at the moment when ever a new episode is submitted I create new person(s) even if I successfully looked them up from db1 using id2 and added them to the episode.
How can I handle this so that:
I can merge duplicate identities based on comparing id2. The joining table that holds episode_id and person_id will also need to be updated where a id is deleted after a merge.
It's much easier if you replace the #ManyToMany association with 2 bidirectional #OneToMany associations, meaning that you map the association table as well.
This way, considering that you have those duplicated Episode or Person entities, you can easily move the joined association by simply adjusting the #ManyToOne associations on the association entity. As for the duplicated Episode or Person you can either use UPSERT as explained below, or do a manual merging after the entries are added to the DB by the batch process.
Typically, when you have multiple nodes that running concurrently, you can use the database UPSERT or MERGE operation.
You could combine the UPSERT with Hibernate #SqlInsert annotation.
To handle the FK updates, you'd need to use the FK ON DELETE CASCADE strategy.
My object model is given below and would like your inputs on the number of indexes to create for faster query responses (on h2, mysql). Assumptions and questions are given below the following model.
#Entity
#Table(name = "user")
public class User {
#Id
#GeneratedValue(strategy = IDENTITY)
#Column(name = "id", unique = true, nullable = false, insertable = false, updatable = false)
private Integer id;
#ManyToOne(fetch = FetchType.LAZY)
#ForeignKey(name = "fk_user_org_id")
#Index(name = "idx_user_org_id")
#JoinColumn(name = "org_id", nullable = false, referencedColumnName = "id")
#NotNull
private Organization organization;
#ManyToOne(fetch = FetchType.LAZY)
#ForeignKey(name = "fk_user_year_id")
#Index(name = "idx_user_year_id")
#JoinColumn(name = "year", nullable = false, referencedColumnName = "id")
#NotNull
private Year year;
#ManyToOne(fetch = FetchType.LAZY)
#ForeignKey(name = "fk_user_created_by")
#Index(name = "idx_user_created_by")
#JoinColumn(name = "created_by", nullable = false, referencedColumnName = "id")
#NotNull
private User createdBy;
#Column(name = "name", nullable = false)
private String name;
#Column(name = "desc")
private String desc;
#Column(name = "is_system", length = LEN_1)
#Type(type = "org.hibernate.type.YesNoType")
private boolean isSystem = false;
#Column(name = "user_type", nullable = false)
private UserType userType;
#Column(name = "status", nullable = false)
#NotNull
private Status status;
}
Our plan is to use multi column indexes instead of a single column index (i.e. create index user_idx based on (organization, year, isSystem, status, userType, createdBy)). Assuming I have this index, will I get optimized responses for my queries listed below.
select * from user where organization=1 and year=2010;
select * from user where organization=1 and year=2010 and isSytem=true or false; (i.e. system users or application defined users)
select * from user where organization=1 and year=2010 and isSytem=false and userType=Manager (i.e. all managers)
select * from user where organization=1 and year=2010 and isSytem=false and userType=Employee (i.e. all employees)
select * from user where organization=1 and year=2010 and isSytem=false and userType=Manager and status=ACTIVE (i.e. Active users)
select * from user where organization=1 and year=2010 and createdBy='Sam' or 'Joe'
Does [6] need a different multi column index, consisting of the above 3 columns?
Since we are creating a multi column index as per my original assumption, can I safely remove the individual indexes (idx_user_org_id, idx_user_year_id, idx_user_created_by) as currently defined in the model?
You should switch the order of the columns in your index:
(organization, year, isSystem, userType, status, createdBy)
This allows it to better serve these two queries:
select * from user where organization=1 and year=2010 and isSystem=false and userType=Manager
select * from user where organization=1 and year=2010 and isSystem=false and userType=Employee
Does [6] need a different multi column index, consisting of the above 3 columns?
It doesn't need a new index - it can use the existing one but in a less efficient way - only the first two columns will be used. Adding a new index for this query looks like a good idea though.
can I safely remove the individual indexes
Yes. You should remove unused indexes otherwise they will just take up disk space and slow down table modifications without providing any benefit.