I currently have a tree structure containing data elements; on each of these, it is possible to perform basic CRUD operations. From here, I need to implement per-user permissions for each of these four actions. So a given user could be given Create and Read, but no Update or Delete permission. These permissions would then cascade down the tree to any children of the permitted object; therefore, that given user would have Create and Read for any child objects of the root object.
What's the best way of storing these permissions using an SQL (MySQL with PHP specifically) database? Currently I'm thinking the ideal solution may be to create another database table which tracks a User ID, an object ID, and then a list of booleans tracking each possible permission, and then checking the user ID and object ID against the permission table and traveling up the tree until a permission object is found (or not found, as the case may be).
My main issue with this is twofold. Firstly, it makes it impossible to give permission to one object, but not its children. Secondly, it seems like it might cause a performance hit on particularly deep objects. So, what seems like a good way of going about this?
Recursive data structures are often hard to "map" to SQL queries. Some databases have special support for it (Oracle, for example) but MySQL has no built-in support (= you can work around that but it's clumsy).
We needed something similar in our application. Our solution was to store normalized data (i.e. "user X has permission Y on node Z" -> three columns with FK relations) in a simple table.
A DAO/manager object reads this table and builds a cache where it can quickly look up permissions as we need them.
To summarize: Keep the database simple and write special helper code in your application to transform the database into the structure which you need.
Related
I'm looking on advice for serializing information for an audit trail in a MySQL database.
I store events that have multiple relations, 6 to be exact. In an Events table. Therefore each record has 6 foreign keys in it. I'm wondering what the most scalable approach for serializing the related entities is, and storing it in this same Event record. This is because the data should persist even if the underlying records have been deleted or changed.
The API is Typescript and we interface the DB with Typeorm. My initial approach was going to be adding an #BeforeInsert event the loads all the related entities as json (some may or may not be present), and storing all of the entities in that format. Either using the "json" field, text, or doing some sort of blob conversion.
This will work for the foreseeable future, but I'm wondering what the most scalable approach would be. Thank you!
I don't know what is the best solution, and maybe it will depend on your features / entities.
Here are some other options that you can use :
Concerning the deleted record : use softRemove / softDelete. It permits to never delete record in DB but set it as deleted for typeorm, using #DeleteDateColumn decorator.
Check the two following properties of typeorm : softRemove / softDelete and recover / restore.
Concerning the updated record : a workaround for the JSON stringify is to never update an entity and always make a copy and then update that copy (not clearly the best solution). Typeorm also use #VersionColumn decorator to persist versioning of the entity. You can change your code to take the older version of that entity.
Depending on your entity the table can grow very fast !
With these two options there is always a persistance of your data.
The advantage can be that it will be resilient for model migration as all your data will be stored as data and not JSON stringify text in a column.
The disadvantage is obviously the size of the database, the maintainability, and the development of it.
Otherwise you can use a subscriber that handles all modifications on the Event table. In this subscriber you can insert your modifications in a table name EventAudit or EvenLogWhatever something like "Name was updated from X to Y" or even more complex with an another database storing all version. In fact, you can do whatever you like inside this global subscriber as you can access the datasource, the entity, and the manager.
I hope it will help
I have been thinking about this title for a long time.
If we want to randomly pair two users and don't consider any conditions, What should I do in database structure and code?
Also, if we have many conditions to query the user, is it not suitable for using the Realtime database and I should use MySQL or something else?
I don't have the experiment in this field and like to know how most people would do.
Thank you.
You should have a "pairs" node, which lists the pair of each user.
When a user wants to find a pair:
Add a key-value node to "pairs", where the key is the UID, and the value is an empty string.
Add a listener to your new node.
Search in "pairs" for another user that has an empty string as a value.
If found, change the values of both nodes to the other user's UID.
When the listener callback will be called, it means some other user just paired with you, so you can use the value to know the UID of the other user. Also, don't forget to cancel the listener.
The reads and writes to the database should be atomic, in order to prevent bugs in the pairing process (like overriding an existing pair). Therefore, you should use
firebase transactions.
If there are certain conditions for a pair, you can save the conditions' data in the node of the user, inside the "pairs" node (for temporary data), or inside the "users" node that you probably have already (for long term data).
Important
However!, this method leaks data about the existing pairs, and about the users that are waiting for a pair. I recommend moving the code to your server or to a cloud function. Security is really critical here! You should also write some strict security rules for the database.
Hope I managed to help! 😀
I need to store organisation ownership hierarchy in a laravel backend. Each node in the hierarchy can be one of a number of types, and each relationship needs to carry the amount of ownership (and potentially more meta data relating to the relationship between nodes). The structure can be arbitrarily deep, and it must be possible to attach a subtree an arbitrary number of times (see C1 below, which appears twice). Below is a sketch of kind of hierarchy I need....
I am using mySQL 8 so I have access to CTE for recursion. I have looked into the adjacency-list package (staudenmeir/laravel-adjacency-list) which uses CTE and looks good, but it uses self referencing tables. I think this means that I cannot store relationship data, and the I don't think I can get the repeated sub tree structure you see above.
I am currently exploring many to many relationships, with a custom pivot table to store the "relationship weighting". But I am unsure if this is a sensible approach and perhaps I'm missing some useful design pattern or this.
I am aware that this is a nebulous question, but while I'm trying to crack this myself using eloquent relationships, I thought I might get a discussion going about design pattens for this type of work.
I'm trying to figure out how Salesforce's metadata architecture works behind the scenes. There's a video they've released ( https://www.youtube.com/watch?v=jrKA3cJmoms ) where he goes through many of the important tables that drive it along (about 18m in).
I've figured out the structure for the basic representation / storage / retrieval of simple stuff, but where i'm hazy is how the relationship pivot table works. I'll be happy when:
a) I know exactly how the pivot table relates to things (RelationId column he mentions is not clear to me)
b) I can construct a query for it.
Screenshot from the video
I've not had any luck finding any resources describing it at this level in the detail I need, or managed to find any packages that emulate it that I can learn from.
Does anyone have any low-level experience with this part of Salesforce that could help?
EDIT: Thank you, David Reed for further details in your edit. So presumably you agree that things aren't exactly as explained?
In the 'value' column, the GUID of the related record is stored
This allows ease of fetching -to-one related records and, with a little bit of simple SQL switching, resolve a group of records in the reverse direction.
I believe Salesforce don't have many-to-many relationships, as opposed to using a 'junction', so the above is still relevant
I guess now though I wonder what the point of the pivot table is at all, as there's a very simple relationship going on here now. Unless the lack of index on the value columns dictates the need for one...
Or, could it be more likely/useful if:
The record's value column stores a GUID to the relationship record and not directly to the related record?
This relationship record holds all necessary information required to put together a decent query and ALSO includes the GUID of the related record?
Neither option clear up the ambiguity for me, unless I'm missing something.
You cannot see, query, or otherwise access the internal tables that underlie Salesforce's on-platform schema. When you build an application on the platform, you query relationships using SOQL relationship queries; there are no pivot tables involved in the work you can see and do on the platform.
While some presentations and documentation discuss at some level the underlying implementation, the precise details of the SQL tables, schemas, query optimizers, and so on are not public.
As a Salesforce developer or developer who interacts with Salesforce via the API, you do not need to worry about the underlying SQL implementation used on Salesforce's servers at almost any time. The main point at which that knowledge can become helpful is when you are working with massive data volumes (multiple millions of records). The most helpful documentation for that use case is Best Practices for Deployments with Large Data Volumes. The underlying schema is briefly discussed under Underlying Concepts. But bear in mind
As a customer, you also cannot optimize the SQL underlying many application operations because it is generated by the system, not written by each tenant.
The implementation details are also subject to change.
Metadata Tables and Data Tables
When an organisation declares an object’s field with a relationship type, Force.com maps the field to a Value field in MT_Data, and then uses this field to store the ObjID of a related object.
I believe the documentation you mentioned is using the identifier ObjId ambiguously, and here actually means what it refers to earlier in the document as GUID - the Salesforce Id. Another paragraph states
The MT_Name_Denorm table is a lean data table that stores the ObjID and Name of each record in MT_Data. When an application needs to provide a list of records involved in a parent/child relationship, Force.com uses the MT_Name_Denorm table to execute a relatively simple query that retrieves the Name of each referenced record for display in the app, say, as part of a hyperlink.
This also doesn't make sense unless ObjId is being used to mean what is called GUID in the visual depiction of the table above in the document - the Salesforce Id of the record.
I am currently working on a Wikipedia API which means that we have a
database for each language we want to use. The structure of each
database is identical, they only differ in their language. The only
place where this information is stored is in the name of the database.
When starting with one language the straight forward approach to use a
mapping between the tables to needed classes (e.g. Page) looked fine.
We defined an engine and corresponding metadata. When we added a
second
database with its own setup for engine and metadata we ran into the
following error:
ArgumentError:
Class '<class 'wp.orm.types.pages.Page'>' already has a primary mapper defined.
Use non_primary=True to create a non primary Mapper.clear_mappers() will remove
*all* current mappers from all classes.
I found an email saying that there must be at least one primary
mapper, so using this option for all databases doesn't seem feasible.
The next idea is to use sharding. For that we need a way to
distinguish
between the databases from the perspective of an instance, as noted in
the docs:
"You need a function which can return
a single shard id, given an instance
to be saved; this is called
"shard_chooser"
I am stuck here. Is there a way to get the database name given an
Object
it is loaded from? Or a possibility to add a static attribute based on
the engine? The alternative would be to add a language column to every
table which is just ugly.
Am I overseeing other possibilities? Any ideas how to define multiple
mappers for the same class, that map against tables in different
databases?
I asked this question on a mailing list and got this answer by Michael Bayer:
if you'd like distinct classes to
indicate that they "belong" in a
different database, and you have very
clear lines as to how this is
performed, use the "entity_name"
concept described at
http://www.sqlalchemy.org/trac/wiki/UsageRecipes/EntityName
. this sounds very much like your use
case.
The next idea is to use sharding. For that we need a way to
distinguish
between the databases from the perspective of an instance, as noted
in
the docs:
"You need a function which can return a single shard id, given an
instance to be saved; this is called "shard_chooser"
horizontal sharding is a method of
storing many homogeneous instances
across multiple databases, with the
implication that you're creating one
big "virtual" database among
partitions - the main concept is
that an individual instance gets
placed in different partitions based
on some ruleset. This is a little
like your use case as well but since
you have a very simple delineation i
think the "entity name" approach is
easier.
So the basic idea is to generate anonymous subclasses for each desired mapping which are distinguished by the Entity_Name. The details can be found in Michaels Link