I am trying to use a metadata JSONB column for a multi-tenant application. Every user for each tenant must have the same metadata, but the tenants have varying metadata fields.
In order to keep all user metadata in sync per tenant, when the tenant admin modifies the metadata fields I need to make sure all users have their metadata JSONB column updated with the following criteria:
If the metadata field/key already exists, the value needs to be retained
If the metadata field/key is new, the key needs to added with a null value
If there are any metadata fields/keys that are not included in the updated list, they should be deleted from the JSON object
For example, all the users for Tenant #1 have the following metadata assigned: { "EmployeeNo" : 123, "HireDate" : "2012-10-10", "Age" : 43 } and somewhere down the line the admin decides they don't care about Age, but they do want to start tracking ParkingSpace.
I need the new metadata record to retain the EmployeeNo and HireDate values, remove the Age key/value, and add the ParkingSpace key with a null value. { "EmployeeNo" : 123, "HireDate" : "2012-10-10", "ParkingSpace" : null }.
I would have thought that I could run an update query similar to the following where it returns a JSONB object where it selects the values if the key exists and a null if it doesn't:
UPDATE users SET metadata = metadata[keys: 'EmployeeNo', 'HireDate', 'ParkingSpace'] WHERE tenant_id = 1;
Obviously that won't work, but hopefully it indicates the issue?
Update: I might have misunderstood your question. Maybe you wanted something like that instead:
UPDATE users
SET metadata = (SELECT json_object_agg(n,metadata->>n) FROM unnest(ARRAY['EmployeeNo','HireDate','ParkingSpace']) AS t(n))
This solution involves creating a brand new jsonb object by extracting only the field that you want from the original metadata. The fields to be copied over are specified as an array that you can easily customize.
Original answer: I think this should do it:
UPDATE users
SET metadata = (metadata - 'Age') || '{"ParkingSpace": null}'::jsonb;
I am using the || operator which merges 2 jsonb objects into one and the - operator which removes a key/value pair.
Related
My problem is simple and straight forward. I want to edit multiple records and save them in database. For editing a single record, I have used following statement in JpaRepository
DatabaseEntity findByAbcId(Integer abcId);
Here, I am trying to fetch a record from Mysql database table DatabaseEntity with respect to its column named abcId which is a foreign key of another table named ABC.
In my service class, I get this record, set its attributes and simply save it back in database like:
//Getting an existing record from database:
//Giving hardcoded value just for understanding
DatabaseEntity databaseEntity = databaseEntityRepository.findByAbcId(106);
//Setting edited fields into Model object. Except for its own id and abcId(foreign key)
databaseEntity.setCol1(value);
databaseEntity.setCol2(value);
databaseEntity.setCol3(value);
databaseEntityRepository.save(databaseEntity);
The above code will get a record and save its edited version into the database.
Now lets take a similar scenario but this time, the database is retrieving multiple records. from the database. Suppose multiple records are present against abcId column in my table. The changes in my code will be:
//Storing the result in the list as there are multiple records stored against 106
List<DatabaseEntity> databaseEntity = databaseEntityRepository.findByAbcId(106);
//What should I code here?
Now I am confused how to set values in multiple fields in a single go from spring boot. The values that I need to edit are also in another list and I tried iterating over both lists and change the database records accordingly but my approach is not a good one
//List of those records which I have edited
if(newRecordsList != null) {
//Using atomic Integer because only that can be changed within lambda expression
AtomicInteger outerLoopCounter = new AtomicInteger(0);
List<DatabaseEntity> databaseEntity = databaseEntityRepository.findByAbcId(Abc.getId());
databaseEntity.forEach(obj -> {
AtomicInteger innerLoopCounter = new AtomicInteger(0);
newRecordsList.forEach(newRecordsListObj -> {
//condition so that first record is updated according to the updated first record and other changes are updated accordingly.
if(outerLoopCounter.get() == innerLoopCounter.get()) {
obj.setName(newRecordsListObj.getName());
obj.setCondition(newRecordsListObj.getCondition());
obj.setValue(newRecordsListObj.getValue());
databaseEntityRepository.save(obj);
}
innerLoopCounter.incrementAndGet();
});
outerLoopCounter.incrementAndGet();
});
}
I know this is a very bad approach and I want to update my logic. So please help me to update these multiple records inside database.
Thanks in advance
I have a jsonb column to which I will add data all at one time, or add data incrementally at various points in time or update existing name-value pairs.
One such case is explained below. Let me know how to app the second user_profile after the first is added. I am not sure if the user_profile format itself needs to be modified. I tried inserting 2 profiles in a single insert and was able to retrieve data according to my needs. So I believe the design format of JSON is ok.
create table myjsontab (jsonb_details jsonb);
insert first profile_type data:
insert into myjsontab(jsonb_details)
values (
'{"user_profile":{
"customer":{
"profile_name":"customer",
"user_status":"NEW","user_state":"pending",
"s3url":["file1","abc/1/clients/51/clients/acmepmwallet1/"]
}
}}'
);
----------------- works.
Insert two profile_type data in one insert
insert into myjsontab(jsonb_details)
values (
'{"user_profile":{
"customer":{
"profile_name":"customer",
"user_status":"NEW","user_state":"pending",
"s3url":["file1","abc/1/clients/51/clients/acmepm1/"]
},
"admin staff":{
"profile_name":"admin staff",
"user_status":"NEW","user_state":"admin upload",
"s3url":"abc/"
}
}}'
);
----------------- also works.
but I am looking to see how I can insert the second profile ("admin staff") set as an "update" or "addition at a later time".
If I try update <table_name> set json_details= json_details|| 'new dataset' there are many issues with formatting the data. I think this approach is wrong.
I am not sure if jsonb_set is a good option to use.
update users set user_details=jsonb_set_lax(user_details->'user_profile','{user_profile,2}','{
"profile_name":"corporate staff",
}',true)
This seems to have worked to solve my issue. Posting this if someone comes here looking for similar answers...
The point to note is the '{user_profile,2}' index. Since I have 2 items in my array, I have to "INSERT" this as the 3rd item which in array parlance is position 2. Since arrays start from 0.
So in your case, you have to find the jsonb_array_length and use that index number as the entry point.
I want to organize some part of my system, but i can't choose convenient form for data representation for interaction with my app.
So i have some local "repository" of data object, descripted as follows:
Object1
{ id = TypeId, field1 = value1, otherObjectSpecifedField = value2 ... }
...
There are many objects (for example 1000) of many types (for example 50). Each type have it own UniqueId and his own description and set of fields.
Next thing is that for each object i have a set of filters, which corresponds that this object is actual right now. It looks like this:
Filter
{ filterName1 = filterValue, filterName2 < filterValue }
Object // filter is applied for this object
{ ... }
The process of using this "repository":
In my app i have application states, which means filters from above.
Example: application localization can be 'en' (my application knows this value and can change it on start) and we have filter, named 'localization' and in our repository we can use it like this:
Filter { localization = 'en'}
Object1 { ... } // this object i should choose when localization is en
When my app decides to check which set of obects is actual right now it cames to repository and asks it: "Here you an TypeId and please, walk through each filter+object pair and say what object is actual by filters. If you need to resolve some filter values (localization from example above) i will resolve them for you".
Then repository walks through each object and compare which is actual by filters now, and which is not and give actual to app. So he check every filter of every object and gives it only if all of them is actual and he did it in runtime.
In current implementation this set of fiters + objects is stored in xml file in very specific xml format, which is comfortably to read from app, but very hard to maintain by human. And i think that there is some place to optimization of all process. I think we can delegate of walking through Objects and comparing it filters to someone else.
Now i think in side of NoSQL document oriented databases. Because each Object has his unique structure and maybe using select routine i can choose what i need.
Maybe someone have any suggestions about that type of database organization? Maybe you know some specific data structure for that type of data?
Maybe I've missed something, because it looks to me like you have a number of different types of objects: one type per TypeId. If that's so, then I think this can be done with a standard SQL database, assuming that the fields within an object have a consistent type. If not, it can still be done with a NoSQL database.
In a SQL database, you would use a separate table for each type (since they each have their own set of fields), and search across the appropriate table with SQL. So, for example, you can create a table with two fields (I'm using SQLite here, which doesn't require types for fields):
create table Object1 (field1, otherObjectSpecifiedField);
This table can then have data inserted into it:
insert into Object1 values ("field1value", "otherfieldValue");
Filtering uses standard SQL:
select * from Object1 where field1 = "field1value";
As I mentioned, this could also be done with a NoSQL database such as MongoDB. That would look something like this in the Mongo CLI:
Create table and insert first object:
db.test.insert({ id: "TypeId", field1: "value1", otherObjectSpecifedField: "value2"});
Select an object from the table:
db.test.find({id: "TypeId", field1: "value1"});
/* { "_id" : ObjectId("57cf97060216d33b891615ba"), "id" : "TypeId", "field1" : "value1", "otherObjectSpecifedField" : "value2" } */
I have models:
class Reference(models.Model):
name = models.CharField(max_length=50)
class Search(models.Model):
reference = models.ForeignKey(Reference)
update_time = models.DateTimeField(auto_now_add=True)
I have an instance of Reference and i need to get all last searches for the reference. Now i am doing it in this way:
record = Search.objects.filter(reference=reference)\
.aggregate(max_date=Max('update_time'))
if record:
update_time = record['max_date']
searches = reference.search_set.filter(update_time=self.update_time)
It is not a big deal to use 2 queries except the one but what if i need to get last searches for each reference on a page? I would have got 2x(count of references) queries and it would not be good.
I was trying to use this solution https://stackoverflow.com/a/9838438/293962 but it didn't work with filter by reference
You probably want to use the latest method.
From the docs, "Returns the latest object in the table, by date, using the field_name provided as the date field."
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#latest
so your query would be
Search.objects.filter(reference=reference).latest('update_time')
I implemented a snippet from someone in gist but I don't remember the user neither have the link.
A bit of context:
I have a model named Medicion that contains the register of mensuration of a machine, machines are created in a model instance of Equipo, Medicion instances have besides of a Foreign key to Equipo, a foreign key to Odometro, this model serves as a kind of clock or metre, that's why when I want to retrieve data (measurements aka instances of Medicion model) for a certain machine, I need to indicate the clock as well, otherwise it would retrieve me a lot of messy and unreadable data.
Here is my implementation:
First I retrieve the last dates:
ult_fechas_reg = Medicion.objects.values('odometro').annotate(max_fecha=Max('fecha')).order_by()
Then I instance an Q object:
mega_statement = Q() # This works as 'AND' Sql Statement
Then looping in every date retrieved in the queryset(annotation) and establishing the Q statement:
for r in ult_fechas_reg:
mega_statement |= (Q(odometro__exact=r['odometro']) & Q(fecha=r['max_fecha']))
Finally passed this mega statement to the queryset that pursues to retrieve the last record of a model filtered by two fields:
resultados = Medicion.objects.filter(mega_query).filter(
equipo=equipo,
odometro__in=lista_odometros).order_by('odometro', 'fecha') # lista_odometros is a python list containing pks of another model, don't worry about it.
[Edit: Apparently, this is only an issue for arrays and FoxyBOA's answer might direct to (or even is) the answer.]
My question relates to these software: Hibernate3+Annotation, Spring MVC, MySQL and in this example also Spring Security.
I was wondering, why collections, which are automatically associated by Hibernate contain null values for each row number of the child table (besides the elements which are correct). My Example:
I have a users and an authorities table, the primary key of the users table is username which serves as foreign key. Right now, there are 13 rows in my authorities table. When I retrieve a user from the database (MySQL InnoDB) and Hibernate automatically retrieves the user's authorities corresponding to this mapping:
#OneToMany
#JoinColumn(name = "username")
#IndexColumn(name="id") // "id" was the primary key and is used to sort the elements
public Authority[] getAuthorities() {
return authorities;
}
public void setAuthorities(Authority[] authorities) {
this.authorities = authorities;
}
... I end up with a collection "authorities" containing 14 (0-13) elements of which only four are not-null (four rows in the database table belong to that specific user, so that is correct). As far as I realize, I am using Hibernate defaults for properties like Fetchmode etc. I am getting the user like this:
Criteria criteria = getSession().createCriteria(User.class);
criteria.add(Restrictions.eq("username",username));
User user = (User) criteria.uniqueResult();
The logging information from org.hibernate.loader.loader correctly "mentions" four rows for the resultset. Still, the user created has the four correct elements plus ten null values in the Array. In my specific example, this results in this exception:
java.lang.IllegalArgumentException: Granted authority element 0 is null - GrantedAuthority[] cannot contain any null elements
The answer lies in the #IndexColumn annotation. It is using the value of id as the array index, thus the number of elements in the Array is basically going to be the value of the highest ID in the Authorities table.
see the hibernate documentation on indexed collections
try removing the annotation.
Also just as a thought; have you considered using a Set for the mapping? it isn't strictly necessary, it just a bit more common form of mapping that's all.
I can recommend you check your data. If you have a missed indexes (id column in your case), then instead of missed id you'll get null in your array.
I.e.
table authorities:
username id
bob 1
bob 3
bob 5
As a result you will have an array:
{0=null, 1=bob, 2=null, 3=bob, 4=null, 5=bob}
UPDATE:
I met the situation in two cases:
Missed key values in indexed column id at authorities table (e.g. 0,1,3,4,5 - missing value 2. Hibernate will automatically add to an array value with key 2 and value null).
Indexed values are in order, but select criteria filter part of them (e.g. your HQL similar to that "from user u join u.authorities a where a.id=2". In that case hibernate load a user, but in authorities array you will have only 3 values: 0 - null, 1 - null, 2 - authority with id 2).