Right now I have a relatively complex piece of code (it's not that complex) that I'd like to simplify through Sequel associations
MyApp::Model::Node.where { last_heartbeat > Time.now - HEARTBEAT_TIMEOUT.seconds }.join(:system_status, node_id: Sequel.expr(:node__id))
Pretty nasty, this statement.
So I'd like to accomplish one thing, pretty much: simplify the above statement by replacing the join with an association in my SystemStatus Model.
Let me explain the above join:
I have two tables Node and SystemStatus. The comparison last_heartbeat > Time.now - HEARTBEAT_TIMEOUT.seconds essentially gives me all the "live nodes" in my application (my application is clustered, meaning the user can spawn multiple instances of it at the same time).
So that mess of a statement gives me "all the statuses of my live nodes" which is joined by system_status.node_id to node.id.
So I started this out by establishing my one to one relation in SystemStatus - this sets the association to the node table
one_to_one :node
Next, I needed to join it by a specific key node.id because it's not a straightforward join
one_to_one :node, key: :id
But my question is:
How do I do the rest of this association
Should I perform the association in Node as opposed to SystemStatus? It seems I could simply do one_to_one :system_status and not have to specify a key.. The above complex join resides in the node Model anyways.
If you have the association in SystemStatus, you need to use many_to_one, as the foreign key is in the that model's table, not the associated model's table. You can just transfer your where condition to the association using a block:
SystemStatus.many_to_one :live_node, class: :Node, key: :node_id do |ds|
ds.where{last_heartbeat > Time.now - HEARTBEAT_TIMEOUT.seconds}
end
Related
I have Student model and Class model, and they have many-to-many relationship.
(A student can register for many classes, and a class can include many students)
I have Enrollment table as a join table.
(you can get the picture in the following website)
https://fmhelp.filemaker.com/help/18/fmp/en/index.html#page/FMP_Help/many-to-many-relationships.html
■ Student table
attributes:
・name
・age
■ Class table
attributes:
・className
・desrciption
■ Enrollment table
attributes:
・studentId
・classId
I think this is typical many-to-many relationship and I'm working this with MySQL and rails.
I would like to know if I could implement this relationships on Elasticsearch.
I read some articles which say that Elasticsearch does not support it, but is there any hacks or best practice for this?
Your use case is better suited for relational database.
Store and query data separately in elastic search and join data in API (business side).
Elasticsearch does not have concept of joins. It is based on concept of denormalization. Denormalization is used to improve the response time of a query at the expense of adding redundant data. Data from different tables can be combined and stored in single place , avoiding the need of joins, which results in faster retrieval at cost of storage(duplicity of data).
Your document which is equivalent to row in a table can be modeled as below
{
studentName:"",
"age":"",
....
classes:[
{
className:"class1",
...
},
{
className:"class2",
...
}
]
}
For each student store all the classes associated with him/her. This will cause duplication of class data across students. It will lead to faster search but slower update as any change in class data will need to be updated across students.
You can also model your data other way around with class as parent and array of students under it. Choose based on your use case.
For your sub model you can use different data types.
Object -- data is flattened.
Nested relations is maintained between properties of model.
Child/Parent sub model becomes different document. It is used when sub model data changes frequently. It does not case re-indexing of parent document.
Suppose I have a table that acts as an inventory of my house - inventory_items if you will. inventory_items contains everything I own, but only the most general information (i.e fields that will apply to everything I own, like a name, purchase date).
I then wish to have a separate table for electronics_data which is an inventory item, but has special information to store (lets say serial_number, wattage) and another for furniture_data which contains furniture specific information (number_of_legs, material).
In all instances, items in electronics_data will have a matching item in inventory_items linked by an id field. The same is true of furniture_data.
If I now wish to show a list of my inventory items, but include specific information from the child tables, logically I think to load the inventory_data, find out what type of item this is, and load the right information from the right table. I can think of two better ways:
1) Create a foreign key relationship between inventory_items and electronics_data - thus loading all items will get me all of my child data too. But, not all items in inventory_items will have a matching item in electronics_data so does this mean a foreign key can't work?
2) Create a view which loads the extra tables if a matching item exists in them, and load the view in my application. If I have lots of different 'types' of data, will this make my view unnecessarily slow (checking everything) and actually defeat the object of the view in the first place?
These are general questions - particularly 2) I would imagine is very data dependent.
Thanks!
1) Foreign keys will work, since the specialised tables are the child tables, so you need to make sure that each record in the child table has a corresponding record in the overall inventory_items table. The reverse is not necessarily true.
2) The view can left join the child tables on the inventory_items table. If the fields used in the join are indexed in all tables, then the operation is not that resource intensive. The biggest catch could be how you build the view, if you have lots of specialised child tables. But this is probably a wider application design question anyway (if you are looking at your electronic devices, then you probably do not want to see the fields from the furniture items table - in these specialised views I would use inner join, not left join).
well it will make your life easier if you could join the tables when extracting data. There are a lot of ways to join tables, in your case if all your tables have an I.D column then you could use an 'Equijoin' This is how you could do so
SELECT inventory_items.name, electronics_data.wattage, furniture_data.material
FROM inventory_items, electronics_data, furniture_data
WHERE inventory_items.i.d=electronics_data.i.d=furniture_data.id;
so with a join like this you can add as many columns as you wish but make sure to highlight the table they are from and in the 'WHERE' clause show where they are equal otherwise it wont return any data
I have posted an fairly detailed response to a similar question here, even how to define the views you mention. Note that the code shown in the view definition is for illustration only. It will not show the most efficient way to write it. Better ways should be fairly straight-forward, however.
A word about view performance. Take a view that joins very large tables in such a way that the query
select * from <view>
takes a long time, say 30 minutes. The query
select * from <view> where <criteria>
could take fractions of a second. In most modern DBMSs, the where criteria is merged with the existing query in the view definition to execute the query. It does not execute the view definition and then do the filtering. So test view performance with actual queries not "data dump" queries.
I have a Ruby 2.2.2 app that uses ActiveRecord as the ORM (similar to Rails). I have two tables, "users" and "accounts", where "accounts" have belongs_to :user. For a given user, I simply want to eager load the contents of the "accounts" table into a Ruby object (named user_accounts) so I can perform operations like:
user_accounts.find_by_product_name("nameofproduct")
...without the find_by_product_name method performing a SQL query. I simply want to preload all entries from the "accounts" table (that belong to a given user) into a Ruby object so I can minimize the number of SQL queries performed.
No matter how much documentation I read, I cannot figure out how to do this properly. Is this possible? If so, how?
If you don't want the ORM to re-query the database, then I think you are better of using the select method added by the Enumerable mixin. Because if you try to use the find_by_* methods, I think it will always send another query.
Here is an example of how it could be achieved.
# The accounts association will be loaded and cached the first time
#user.accounts.select { |account| account.name == "nameofproduct" }
# The next time, it is already loaded and will select from the previously loaded records
#user.accounts.select { |account| account.name == "nameofanotherproduct" }
I would store that accounts in a hash instead of an array, because lookups in a hash are much faster (in O(1)) than in an array which only allows O(n).
group_by helps you building that hash.
# assuming that #user.accounts returns all accounts
user_accounts = #user.accounts.group_by(&:name)
# querying the hash. use `first` to receive the first matching
# account (even if there is only one)
user_accounts['nameofproduct'].first
I have a schema where most of the tables have associated users_*_meta tables which store per-user data like starred/unstarred, rating, and the like.
For example, stories -< users_stories_meta >- users.
However, I'm having trouble figuring out how to perform a joined load of a row and the related metadata row for the current user without either writing my own ORM on top of SQL Alchemy's expression builder or using a closure to generate a new session and schema for each request. (relationship() doesn't seem to support resolving components of primaryjoin lazily)
What would the simplest, least wheel-reinventing way be to treat the appropriate subset of the many-to-many relationship as a one-to-many relationship (one user, many stories/authors/elements/etc.) specific to each request?
There's a recipe at http://www.sqlalchemy.org/trac/wiki/UsageRecipes/GlobalFilter which illustrates how the criterion represented by a relationship() can be influenced at the per-query level.
I have
OrgUnits,
OrgUnitJobs,
Employees,
OrgUnitJobEmployee (this one may seem redundant in this context, its here fyi)
tables.
org units have jobs defined in OrgUnitJobs (1..n)
employee (can also have many jobs) has jobs defined in OrgUnitJobEmployee table.
and employee has a PrimaryJob (1..1) in OrgUnitJobs table, so how do I get a list of all the employees who's primary job is in that org unit.
I am using LINQ2SQL ORM, so I have an OrgUnit partial class, I would like to be able to get this employee list within that partial ORM class, without writing another query, or a linq select.
I just dont want to go to my OCRepository class and write a GetEmployeeListForOrgUnit(orgUnitId) there, it does not sound right.
Check DataContext.LoadOptions to tweak what's included in the first query. This means it'll execute a join, instead of N subqueries.
In your case, something like:
DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Job>(j => j.PrimaryEmployees);
dbContext.LoadOptions = dlo;