So, it may be best to just try it out and see through some trial and error, but I'm trying to figure out the best way to migrate a pretty simple structure from mysql to mongodb. Let's say that I have a main table in mysql called 'articles' and I have two other tables, one called 'categories' and the other 'category_linkage'. The categories all have an ID and a name. The articles all have an ID and other data. The linkage table relates articles to categories, so that you can have unlimited categories related to each article.
From a MongoDB approach, would it make sense to just store the article data and category ID's that belong to that article in the same collection, thus having just 2 data collections (one for the articles and one for the categories)? My thinking is that to add/remove categories from an article, you would just update($pull/$push) on that particular article document, no?
In my opinion, a good model would look like this:
{"article_name": "name",
"category": ["category1_name", "category2_name", ...],
"other_data": "other data value"
}
So, to embed the category names directly to the article document. Updating article categories is easy, but removing a category altogether requires modifying all articles belonging to the category. If removing categories is frequent, then keeping them separate might be a good idea performance-wise.
This approach makes it also easy to make queries on the category name (no need to map name to id with a separate query).
Thus, the "correct" way to model the data depends on the assumed use case, as is typically the case with mongodb and other nosql databases.
If you have access to a Mac computer, you could give the MongoHub GUI a try. It has an "Import from MySQL" feature.
Related
i wanted to get your expert opinion about this dilema chosing bewteen JSON or Pivot Table
Let just say we have 2 tables here
people
jobs
A person may have multiple jobs, alas, a jobs might have multiple person subscirbed to it.
What is the best approach to it?
Method 1: JSON
I would have jobs column in people table, that contain json array of that person's jobs id, example : [1,2,4]
Method 2: Pivot
I would create pivot table job_person with job_id and person_id column, well, you know Laravel Eloquent style many to many pivot table
I have done some searching, and i found articels favouring each method, some say JSON better because it musch simpler, others would say Pivot is better due to that is how relationship database should work, etc etc.
But i want to know, which one should i use in what scenario? Like if it is just simple case like above scenario, JSON would be better?
What if there are other variables included like additional pivot columns
(Maybe each pivot also contain status column that can be set to active or past_job)
Or what if in the future we want to be able to get all peoples whom have a specific jobs, in which case Pivot would be preferable i think.
What if instead of jobs, the other table would be books and a person can have an extensive of books making we might have tens, or even hundreed pivot records just for one person? And there will be another hundreed persons?
What if instead of books, the other table were stocks in which case, a person might subscribed / unsubscribed multiple stock multiple times?
And maybe to the basic principle, what is each one's advantages/disadvantages?
Thank you very much
I would rather not choose JSON, as there's no benefit from choosing it, you will sacrifice many of the database features and make querying the data difficult and slow.
What if there are other variables included like additional pivot
columns (Maybe each pivot also contain status column that can be set
to active or past_job)
Job and Person are not dependent on each others, so you need to create an associative table between them something like "PersonJob" and add necessary information to it, this is easy to traverse in Laravel.
Or what if in the future we want to be able to get all peoples whom
have a specific jobs, in which case Pivot would be preferable i think.
You could easily query this using the associative table.
And maybe to the basic principle, what is each one's
advantages/disadvantages?
it just that relational databases are made for this kind of stuff and JSON offer no value just hardship.
Being new to relational database design, I am trying to clarify one piece of information to properly design this database. Although I am using Filemaker as the platform, I believe this is a universal question.
Using the logic of ideally having all one to many relationships, and using separate tables or join tables to solve these.
I have a database with multiple products, made by multiple brands, in multiple product categories. I also want this to be as scale-able as possible when it comes to reporting, being able to slice and dice the data in as many ways as possible since the needs of the users are constantly changing.
So when I ask the question "Does each Brand have multiple products" I get a yes, and "Does each product have multiple brands" the answer is no. So this is a one to many relationship, but it also seems that a self-join table might give me everything that I need.
This methodology also seems to go down a rabbit hole for other "product related" information such as product category, each product is tied to one product category, but only one product category is related to a product.
So I see 2 possibilities, make three tables and join them with primary and foreign keys, one for Brand, one for Product Category, and one for Products.
Or the second possibility is to create one table that has the brand and product category and product info all in one table (since they are all product related) and simply do self-joins and other query based tables to give me the future reporting requirements that will be changing over time.
I am looking for input from experiences that might point me in the right direction.
Thanks in advance!
Could you ever want to store additional information about a brand (company URL, phone number, etc.) or about a product category (description, etc.)?
If the answer is yes, you definitely want to use three tables. If you don't, you'll be repeating all that information for every single item that belongs to the same brand or same category.
If the answer is no, there is still an advantage to using three tables - it will prevent typos or other spelling inconsistencies from getting into your database. For example, it would prevent you from writing a brand as "Coca Cola" for some items and as "Coca-Cola" for other items. These inconsistencies get harder and harder to find and correct as your database grows. By having each brand only listed once in it's own table, it will always be written the same way.
The disadvantage of multiple tables is the SQL for your queries is more complicated. There's definitely a tradeoff, but when in doubt, normalize into multiple tables. You'll learn when it's better to de-normalize with more experience.
I am not sure where do you see a room for a self-join here. It seems to me you are saying: I have a table of products; each product has one brand and one (?) category. If that's the case then you need either three tables:
Brands -< Products >- Categories
or - in Filemaker only - you can replace either or both the Brands and the Categories tables with a value list (assuming you won't be renaming brands/categories and at the expense of some reporting capabilities). So really it depends on what type of information you want to get out in the end.
If you truly want your solution to be scalable you need to parse and partition your data now. Otherwise you will be faced with the re-structuring of the solution down the road when the solution grows in size. You will also be faced with parsing and relocating the data to new tables. Since you've also included the SQL and MySQL tags if you plan on connecting Filemaker to an external data source then you will definitely need to up your game structurally.
Building everything in one table is essentially using Filemaker to do Excel work and it won't cut it if you are connecting to SQL, MySQL, etc.
Self join tables are a great tool. However, they should really only be used for calculating small data points and should not be used as pivot points or foundations for your reporting features. It can grow out of control as time goes on and you need to keep your backend clean.
Use summary and sub-summary reporting features to slice product based data.
For retail and general product management solutions, whether it's Filemaker/SQL/or whatever the "Brand" or "Vendor" is it's own table. Then you would have a "Products" table (the match key being the "Brand ID").
The "Product Category" field should be a field in the "Products" table. You can manage the category values by building a standard value list or building a value list based on a "Product Category" table. The second scenario is better for long term administration.
I'm a software developer. I love to code, but I hate databases... Currently, I'm creating a website on which a user will be allowed to mark an entity as liked (like in FB), tag it and comment.
I get stuck on database tables design for handling this functionality. Solution is trivial, if we can do this only for one type of thing (eg. photos). But I need to enable this for 5 different things (for now, but I also assume that this number can grow, as the whole service grows).
I found some similar questions here, but none of them have a satisfying answer, so I'm asking this question again.
The question is, how to properly, efficiently and elastically design the database, so that it can store comments for different tables, likes for different tables and tags for them. Some design pattern as answer will be best ;)
Detailed description:
I have a table User with some user data, and 3 more tables: Photo with photographs, Articles with articles, Places with places. I want to enable any logged user to:
comment on any of those 3 tables
mark any of them as liked
tag any of them with some tag
I also want to count the number of likes for every element and the number of times that particular tag was used.
1st approach:
a) For tags, I will create a table Tag [TagId, tagName, tagCounter], then I will create many-to-many relationships tables for: Photo_has_tags, Place_has_tag, Article_has_tag.
b) The same counts for comments.
c) I will create a table LikedPhotos [idUser, idPhoto], LikedArticles[idUser, idArticle], LikedPlace [idUser, idPlace]. Number of likes will be calculated by queries (which, I assume is bad). And...
I really don't like this design for the last part, it smells badly for me ;)
2nd approach:
I will create a table ElementType [idType, TypeName == some table name] which will be populated by the administrator (me) with the names of tables that can be liked, commented or tagged. Then I will create tables:
a) LikedElement [idLike, idUser, idElementType, idLikedElement] and the same for Comments and Tags with the proper columns for each. Now, when I want to make a photo liked I will insert:
typeId = SELECT id FROM ElementType WHERE TypeName == 'Photo'
INSERT (user id, typeId, photoId)
and for places:
typeId = SELECT id FROM ElementType WHERE TypeName == 'Place'
INSERT (user id, typeId, placeId)
and so on... I think that the second approach is better, but I also feel like something is missing in this design as well...
At last, I also wonder which the best place to store counter for how many times the element was liked is. I can think of only two ways:
in element (Photo/Article/Place) table
by select count().
I hope that my explanation of the issue is more thorough now.
The most extensible solution is to have just one "base" table (connected to "likes", tags and comments), and "inherit" all other tables from it. Adding a new kind of entity involves just adding a new "inherited" table - it then automatically plugs into the whole like/tag/comment machinery.
Entity-relationship term for this is "category" (see the ERwin Methods Guide, section: "Subtype Relationships"). The category symbol is:
Assuming a user can like multiple entities, a same tag can be used for more than one entity but a comment is entity-specific, your model could look like this:
BTW, there are roughly 3 ways to implement the "ER category":
All types in one table.
All concrete types in separate tables.
All concrete and abstract types in separate tables.
Unless you have very stringent performance requirements, the third approach is probably the best (meaning the physical tables match 1:1 the entities in the diagram above).
Since you "hate" databases, why are you trying to implement one? Instead, solicit help from someone who loves and breathes this stuff.
Otherwise, learn to love your database. A well designed database simplifies programming, engineering the site, and smooths its continuing operation. Even an experienced d/b designer will not have complete and perfect foresight: some schema changes down the road will be needed as usage patterns emerge or requirements change.
If this is a one man project, program the database interface into simple operations using stored procedures: add_user, update_user, add_comment, add_like, upload_photo, list_comments, etc. Do not embed the schema into even one line of code. In this manner, the database schema can be changed without affecting any code: only the stored procedures should know about the schema.
You may have to refactor the schema several times. This is normal. Don't worry about getting it perfect the first time. Just make it functional enough to prototype an initial design. If you have the luxury of time, use it some, and then delete the schema and do it again. It is always better the second time.
This is a general idea
please don´t pay much attention to the field names styling, but more to the relation and structure
This pseudocode will get all the comments of photo with ID 5
SELECT * FROM actions
WHERE actions.id_Stuff = 5
AND actions.typeStuff="photo"
AND actions.typeAction = "comment"
This pseudocode will get all the likes or users who liked photo with ID 5
(you may use count() to just get the amount of likes)
SELECT * FROM actions
WHERE actions.id_Stuff = 5
AND actions.typeStuff="photo"
AND actions.typeAction = "like"
as far as i understand. several tables are required. There is a many to many relation between them.
Table which stores the user data such as name, surname, birth date with a identity field.
Table which stores data types. these types may be photos, shares, links. each type must has a unique table. therefore, there is a relation between their individual tables and this table.
each different data type has its table. for example, status updates, photos, links.
the last table is for many to many relation storing an id, user id, data type and data id.
Look at the access patterns you are going to need. Do any of them seem to made particularly difficult or inefficient my one design choice or the other?
If not favour the one that requires the fewer tables
In this case:
Add Comment: you either pick a particular many/many table or insert into a common table with a known specific identifier for what is being liked, I think client code will be slightly simpler in your second case.
Find comments for item: here it seems using a common table is slightly easier - we just have a single query parameterised by type of entity
Find comments by a person about one kind of thing: simple query in either case
Find all comments by a person about all things: this seems little gnarly either way.
I think your "discriminated" approach, option 2, yields simpler queries in some cases and doesn't seem much worse in the others so I'd go with it.
Consider using table per entity for comments and etc. More tables - better sharding and scaling. It's not a problem to control many similar tables for all frameworks I know.
One day you'll need to optimize reads from such structure. You can easily create agragating tables over base ones and lose a bit on writes.
One big table with dictionary may become uncontrollable one day.
Definitely go with the second approach where you have one table and store the element type for each row, it will give you a lot more flexibility. Basically when something can logically be done with fewer tables it is almost always better to go with fewer tables. One advantage that comes to my mind right now about your particular case, consider you want to delete all liked elements of a certain user, with your first approach you need to issue one query for each element type but with the second approach it can be done with only one query or consider when you want to add a new element type, with the first approach it involves creating a new table for each new type but with the second approach you shouldn't do anything...
I am trying to understand the concept of document storage and fail to see how it would apply to some situations. For example, in the case of a CMS/blog engine there may be data in the form of:
Posts
Categories
Users
Comments
In something such as MySQL one might have a table for each, then a join table for each set of related data. i.e. posts_table, categories_table, categories_posts_table
In this case, posts_table would contain the post data, categories_table would contain the categories data and categories_posts_table would contain 2 foreign keys used to associate a specific category to a specific post.
How does this translate into something like mongodb?
The only way i can see this setup being structured in mongo is something like:
posts_collection
The output of a single bson document might look similar to:
{
"title" : "title",
"body" : "blah body",
"categories" : [
"category1",
"category2"
]
}
That makes sense, but it seems like categories are going to be duplicated all over the place. With out some sort of relation, you could never be able to simply change the category name and have it reflected across all of your blog posts (?).
Additionally what if these were like binary documents that took up a lot of space? Instead of duplicating the same image over and over it seems like a relationship would work better?
I guess this is a pretty open question, but i was looking for anyone's input on how i should mentally take apart a problem to tell if it should fit in a db like mongo or not. And equally important is how does one structure data correctly?
I have not touched on Users but it seems like EVERYTHING in this would ultimately end up as an embedded document inside of a User's collection since the User kind of starts everything.
thanks a lot.
What's interesting about document databases is that you really need to think about how your data is going to be used. Storing the same information in multiple places (denormalization) is fine in a document database. So you're correct when you say you could have a root User document with everything else embedded in it.
From my limited experience, there's not a "right" way to model a particular set of data, it's more about how that data is going to be used in the future.
It IS possible to reference another documents. For example if you want a Posts collection and have each Post reference a User document in the Users collection. Take a look at this article about Embed vs. Reference.
I'm using the Adjacency List Model to create categories, and it works perfectly.
When retrieving articles in a certain category (for example electronics), I would like to also retrieve the articles in the sub categories (for example electronics->cameras, or even electronics->cameras->camera lenses).
The way I am doing it now is pulling from the DB all the category id's of the sub categories of electronics, and finding all articles with a category_id in this list.
This seems to me very inefficient and time-consuming, since this could result in many queries to retrieve these sub categories.
Another way I thought of doing this is having every article associated with the whole category tree (for example an article about camera lenses will also be associated with the camera and electronics categories in MANY_MANY table), and when I retrieve all articles in electronics it will also appear.
This would add a lot of redundant data to the database though, as I might have to store 3 or 4 categories for each article. Also, it would complicate actions like moving an article to another category.
Is this the right way to go? Or is there a better/simpler way that I have not thought of?
Any help appreciated!
Have a read of this article about Nested Set Modelling: Managing Hierarchical Data in MySQL.
Using the suggested technique, you can get entire trees or subtrees in one single SELECT. It's a bit more complicated than the "normal" approach, but it's totally worth it if you're going to be doing lots of reads from the table.