Changing attributes over time in relational database - relational-database

At the core of my app is a table called Entities:
Entities:
-----------
id: number // PK
name: string
position_x: number
position_y: number
These entities have interactions with each other, called Events:
Events:
-----------
id: number // PK
name: string
date: Date
from: number // Entities FK
to: number // Entities FK
With events, we can manage all kinds of events between two entities. By selecting a timestamp / event, we can "time-travel" through each event, and see how changes affect the entities.
I just learned we want to support changes to entity attributes in the timeline as well. (travelling back should still get you the original attributes). So now I want to create a new Event-like table, which would look something like this:
SingleEntityEvents:
-----------
id: number // PK
name: string
date: Date
to: number // Entities FK
type: enum // CHANGE_NAME | CHANGE_POSITION
payload: json
Now my app would get the entities and render the original name if applicable, but render an updated name if there is a CHANGE_NAME event happening before the currently selected date. However, now I am leaning towards getting rid of the Entities table entirely, and make everything (including Entity creation) an event. But then how do I properly refer to these entities in the original (from -> to) Events? Is this even possible?
In other words, I want to be able to model this stream of events, but I am not sure how to:
Create entity through event:
name: 'Create Entity'
date: 01/01/2022
entityID: 123 // Its not to reference but to create an entity with this ID?
type: CREATE_ENTITY
payload: {
"name": "New Entity",
"position_x": 1,
"position_y": 0
}
If I select any time after this first event, the app should render an entity called "New Entity", positioned on coordinate (1, 0).
Update name of entity through event
name: 'Update Entity Name'
date: 01/03/2022
entityID: 123 // Now, since this is an update event, this is an actual reference to the previously created entity
type: UPDATE_ENTITY_NAME
payload: "Foo Bar"
If I select any time after this second event, the app should now render the same entity but it's now called "Foo Bar". Its position is unchanged so its still on (1, 0).
My question is, are there best practices to model such event streams? And how do I go about creating this entity ID?

Related

Any way to build up a dictionary / json object of sensor values in esphome?

I am trying to build up a dictionary / json object of sensor values in esphome. I have a sensor that sends me key / value pairs (e.g. one sensor reading could be { “temperature“: 25.1 }, another one could be { “speed“: 50.1 } and so forth) in very high frequency (milliseconds). What I would like to do is to collect data for these key / value pairs for a certain time span, for simplicity say ten seconds, and only then take the dictionary and post it to a web service. It would also somehow combine the readings for the same key if sent multiple times within the ten seconds time span for example by averaging them out, using a filter or whatever. So the final dictionary to be posted to the web service would then look like
{
“temperature“: 26.3,
“speed“: 52.5,
…
}
How could I achieve this - any idea / proposal?
Thanks and best regards
Dear stackoverflow community,
I found a solution to this issue. I am now using a global variable in esphome. This can be defined as follows:
globals:
- id: "my_dict"
type: std::map<std::string, std::string>
With this, I have a global map which I can use to store the key / value pairs. Adding a new key / value pair via a lambda works as simple as shown in the following (where in this example, the key is stored in the variable key and the value is stored in the variable value:
lambda: |-
id(my_dict)[key] = value;
Every ten seconds, I post the dictionary content to the web service and then clear the dictionary again:
interval:
- interval: 10s
then:
- http_request.post:
url: "https://<URL>"
json: |-
for ( auto item : id(my_dict) ) {
root[item.first] = item.second;
}
- lambda: |-
id(my_dict).clear();

writing script in scala to join two mysql tables and create one object (quill)

I have two mysql tables: Owners & Pets
Owner case class:
Owner(id: Int, name: String, age: Int)
Pet case class:
Pet(id: Int, ownerId: Int, type: String, name: String)
I want to create out of those tables list of OwnerAndPets:
case class OwnerAndPets(ownerId: Int,
name: String,
age: String,
pets: List[Pet])
(its for migrations purposes, I want to move those tables to be a collection of mongodb, which the collection documents would be OwnerAndPets objects)
I have two issues:
when I use join with quill on Owner & Pet I get list of tuples [(Owner, Pet)]
and if I have few pets for an owner I will get:
[(Owner(1, "john", 30), Pet(3,1,"dog","max")),
(Owner(1, "john", 30), Pet(4,1,"cat","snow"))]
I need it as (Owner(1, "john", 30), [Pet(3,1,"dog","max"), Pet(4,1,"cat","snow")])
how can I make it like this?
when I use join with quill on Owner & Pet I will not get owners that dont have pets and its fine cause this is what it supposed to be, but in my script in this case I would want to create object like:
OwnerAndPets(Owner(2, "mark", 30), List[])
Would appreciate any help, thanks
this is my join query:
query[Owner].join(query[Pet]).on((o, p) => o.id == p.o_id)
Your question highlights one of the major differences between FRM (Functional Relational Mapping) systems like Quill and Slick as opposed to ORMs like Hibernate. The purpose of FRM systems is not to build a particular domain-specific object hierarchy e.g. OwnersAndPets, but rather, to be able translate a single database query into some set of objects that can reasonably be pulled out of that single query's result set - this is typically a tuple. This means it is up to you to join the tuples (Owner_N, Pet_1-N) object into a single OwnersAndPets object in memory. Typically this can be done via groupBy and map operators:
run(query[Owner].join(query[Pet]).on((o, p) => o.id == p.o_id))
.groupBy(_._1)
.map({case (owner,ownerPetList) =>
OwnerAndPets(
owner.id,owner.name,owner.age+"", // Not sure why you made 'age' a String in OwnerAndPets
ownerPetList.map(_._2))
})
That said, there are some database vendors (e.g. Postgres) that internally implement array types so in some cases you can do the join on the database-level but this is not the case for MySQL and many others.

Best way to handle data list of REST web service with foreign key(one to many)

I am going to implement the REST base CRUD modal in my my app.I wan to display the list of product data with edit and delete link
Product
id, title, unit_id, product_type_id, currency_id,price
Q1: what should be json response look like?
There are two formats comes in my mind to place the data in Json as a response of REST Get call
[
{
id:1,
title:"T-Shirt",
unit_id:20,
unit_title: "abc"
product_type_id:30,
product_type_title:"xyz"
currency_id: 10,
currency_name: "USD"
min_price:20
},
{...}
]
and the another one is
[
{
id:1,
title:"T-Shirt",
unit: {
id: 20,
title: "abc"
},
product_type: {
id: 30,
title: "xyz"
},
currency_id: {
id:10,
name: "USD"
},
min_price:20
},
{...}
]
what is the better and standard way to handle the above scenario?
Furthermore, let suppose I have 10 more properties in product table which will never display on list page. but i needed it when user going to edit the specific item.
Q2: Should I the load all data once at the time of displaying product list and pass the data to edit component.
or
Load only the needed propeties of product table and pass the id to produt edit component and a new REST GET call with id to get the properties of product.
I am using React + Redux for my front end
Typically, you would create additional methods for API consumers to retrieve the values that populate the lists of currency, product_type and unit when editing in a UI.
I wouldn't return more data than necessary for an individual Product object.

Ember many-to-many Ids are not included in JSON payload on post?

I have a meeting and a sales-rep models, The relation is ManyToMany.
The problem is, When I want to create a New meeting, and assign existing salesReps to it (They are already saved to the store), But The salesReps IDS are not included in The post action caused by model.save() (not even an empty array), To make it more clear, Here is what my code looks like:
meeting.coffee:
Meeting = DS.Model.extend
client: DS.belongsTo('client')
salesReps: DS.hasMany('sales-rep')
memo: DS.attr('string')
startDate: DS.attr('date')
duration: DS.attr()
sales-rep.coffee:
SalesRep = DS.Model.extend
meetings: DS.hasMany('meeting')
firstName: DS.attr('string')
lastName: DS.attr('string')
title: DS.attr('string')
meetings/new.coffee (the save action am using inside new meeting controller):
save: ->
meeting = #get('model')
meeting.set('client', #get('client'))
meeting.get('salesReps').pushObjects(#get('salesReps.content'))
meeting.save().then =>
#transitionToRoute 'meetings'
the JSON payload: ( POST http://localhost:4200/api/meetings)
meeting: {memo: null, start_date: null, duration: "00:15", client_id: null}
client_id: null
duration: "00:15"
memo: null
start_date: null
No matter what, There is no ANY trace of the salesReps ids in the payload!!
What I tried so far:
Setting the hasMany relation in the meeting model only.
Setting {async: true}, and then {async: false}, on both SalesRep, And
then on one of them
spending almost 2 days googling and reading all related posts in here
with no luck
Any Help/hints/Advice, Is highly appreciated
I will write the solution I found after endless reading and researching, I will write the full details, and trial/failure i've been through, Because no one, no one EVER should have to spend more than 3 days trying to fix something like that!!
I am using Ember-cli, So, there are a files/directories structure am following:
First attempt:
Trying all combinations of async: true, embedded true and what not.
Result, No luck
Second attempt:
in app/serializers/ I added the following serializer file:
meeting.coffee
`import DS from "ember-data"`
`import Ember from "ember"`
`import config from '../config/environment'`
get = Ember.get
serializer = DS.RESTSerializer.extend
serializeHasMany: (record, json, relationship) ->
rel_ids = get(record, relationship.key).map (rel) -> get(rel, 'id') || []
json["#{relationship.key.underscore().singularize()}_ids"] = rel_ids
json
`export default serializer`
result:
Adding this serializer, And I finally was able to send sales_rep_ids:[] array to the controller! and I could confirm that the server is saving the accociations as required.
But, When listing meetings, I was not able to list the associated salesReps, So, I checked the JSON am getting from the server, and it was correct (salesReps Ids were included!) But still not listed in Ember
Third Attempt:
After more reading and endless head-banging-against-the-wall, Changing ONE line fixed the problem!:
in app/serializers/meeting.coffee
change serializer = DS.RESTSerializer.extend to
serializer = DS.ActiveModelSerializer.extend
And Voila! Saved to the back-end, And listed correctley as association in ember!
This solution is a result of 3+ of constant headache, Am posting it here hopefully it might be helpful to someone facing the same problem, I can't claim that it's my own solution, but, It's the result of reading many people's code.
am not sure if it's the Ember way to do so, So, Any suggestions, Improvements Ideas and thought are welcome.

Recursive as_array for One-to-many relationships with Kohana 3 ORM

To begin with I'd like to say I know how to create ugly solutions for my problem. I am searching good solutions and best practices :)
How do I create deep hierarchial arrays (to json_encode later) from Kohana 3 ORM objects including related objects where the relation type is one-to-many?
The problem is that the ORM->as_array() method does work recursively for the "has one" and "belongs to" relationships but will stop and force you to use ->find_all() manually when you encounter a "has many" relationship.
Say I am creating a JSON API REST server using Kohana 3 and the built in ORM.
When someone looks at this URL: www.example.com/api/user?id=5
They will be served a JSON object for the user where id=5.
These are the orm-models and the relations:
User belongs to a Country
User has many Messages.
Message belongs to a Category
I would like this to work:
echo json_encode(
ORM::factory('user', 5)
->with('country')
->with('messages')
->with('messages:category')
->find()
->as_array()
);
and give me output like this:
{
name: "John"
age: 54,
country_id: 5,
country: {
name: 'Sweden',
code: 'SE'
},
messages: {
{
content: 'Lorem ipsum dolor...',
category_id: 1,
category: {...}
},
{
content: 'Sit amet elit...',
category_id: 2,
category: {...}
},
{
content: 'Consectetur ipsum dolor...',
category_id: 3,
category: {...}
}
}
}
BUT THAT WONT WORK.
This is all you would get:
{
name: "John"
age: 54,
country_id: 5,
country: {
name: 'Sweden',
code: 'SE'
}
}
Has someone forked or extended the Kohana 3 ORM to support this kind functionality?
Does someone know of any good Kohana 3 api module that takes care of this issue for you somehow?
As far as I know, there is no way to do this using only the ORM. This is the case for two reasons:
If a message has a relation for category, that usually means that category has a corresponding relation for the messages in the category. If you want to get the message "in full" - i.e. including its category - let's say that you also want to get the category "in full" - i.e. including its messages. This is obviously a very bad idea, as you could very easily design yourself into an infinite loop. In other words, if there was a magic "recurse all relations" capability, how would it know when to stop recursing?
All that the ORM is doing behind the scenes when you call find() is building a SQL query that returns a row of data from the database. However, what you are trying to do is too complicated for a single query that returns a single row. (There is a way to retrieve multiple rows as one row with comma-separated values in each field using MySQL's GROUP_CONCAT function, but I guarantee you that it is not worth the trouble.)
For both of these reasons, the ORM's with() method only works on belongs_to (in your case, the user's country and the message's category).
One way to do this query is to break it up into three steps, like this:
// Step 1: Get the user
$user = ORM::factory('user', 5)
->with('country')
->find();
// Step 2: Get the messages
$user->messages
->with('category')
->find_all();
// Step 3: Make user and the messages into arrays.
// User is easy, but messages are a little harder
// because they need to turn
// from "an object containing an array of objects"
// into "an array of arrays."
$user_arr = $user->as_array();
$fixer = function($obj)
{
return $obj->as_array();
};
$user_arr['messages'] = array_map($fixer, $user->messages->as_array());
// Now you can output it
echo json_encode($user_arr);