I have a mysql table that stores user ratings for different items. It has the following fields:
id (int, pk)
userId (int)
itemId (int)
rating (float)
timestamp (int)
and following indices:
(userId, rating): for queries about all items a particular user has rated
(itemId, rating): for queries about all users that have rated a particular item
This table has over 10 million rows. To make it more scalable, I would like to perform a horizontal partitioning. In particular, I plan to partition the table into 20 tables:
tbl_rating_by_item_0: store ratings whose itemId ending with 0
tbl_rating_by_item_1: store ratings whose itemId ending with 1
......
tbl_rating_by_item_9: store ratings whose itemId ending with 9
and
tbl_rating_by_user_0: store ratings whose userId ending with 0
tbl_rating_by_user_1: store ratings whose userId ending with 1
......
tbl_rating_by_user_9: store ratings whose userId ending with 9
The idea is when querying by itemId we read from tbl_rating_by_item_itemId and when querying by userId we read from tbl_rating_by_user_userId. The drawback is whenever I want to insert or delete a rating, I need to insert into or delete from two tables.
Is there any other solutions?
Have you tried indexing? Creating two composite indexes
INDEX name1 (rating,userid)
INDEX name2 (rating,itemId)
may help increase in performance.
Also consider table partitioning. Have a look at Mysql able partitioning
This is better than physically creating two separate tables.
Related
How my system works is that a user may be assigned to several things.
A column is called group and they may be assigned to several groups, e.g 1,5,8 etc... Is there a way to store them in the same column?
Don't store relations in single column. Normalize your data and introduce new table which will hold references for user group relations like
Table user_groups
user_id group_id
1 1
1 5
1 8
Also have a look at Is storing a delimited list in a database column really that bad?
I'm totally new to MySQL.
I'm working on a university courses system where each course ("event") has many students("users") that are signed in to that course.
I want a to build a MySQL table that for each course ID will store all the students IDs that are signed in to it. I found two approaches:
Serialize students IDs (using json for example)
Create a new row for every new student with course ID and student ID
The first approach has a performance issue because of serialization/de-serialization of students IDs, and the second approach will cause the table to explode very quickly.
What do you think I should do? Is there a better solution?
Thanks!
Your case is a Many-to-Many mapping.
You would have 3 tables.
User
Events
User_Events
The third table will store the mappings, it should consist of two columns, user_id and event_id
user_id would be the foreign key for the User table, similarly, event_id would be the foreign key for the Event table. You can then retrieve data by Joins in queries.
so if you have users with ID 1, 2, and 3
and similarly courses with ID 1, 2, 3, 4
Here's how you will represent students that are signed in a course.
user_id, event_id
1 2
1 3
2 1
3 4
This means, user 1 is signed in course 2 and 3, user 2 in course 1 and user with ID 3 in course with ID 4.
Each student/user can be in several courses/events.
Each course/event can contain several students/users.
So you need a bridge table containing all relations:
user_event (userid, eventid)
with a unique constraint on userid + eventid.
(You can add a technical ID (e.g. user_event_id) to the table, but that's not necessary.)
I am creating a site that is sort of ecommerce-ish. I want to give my users a perfect search ability using specific attributes that differ from product to product. I plan to create 1 products table storing the basic information that is shared among products i.e Name, Description, Price and a few others. Then I plan to create several "details" table say categories_computers with columns Processor, HDD, RAM, etc and another table say table_shoes with columns MATERIAL, SIZE, GENDER, etc.
I am new to Mysql but not to the concept of Databases. I don't think I will have a problem storing this data to each table. My issue comes about from reads. It won't be hard to query a product id but I think it would be extremely wasteful to query all details tables to get the details of the product since 1 product can only have 1 details.
So my question is how can I store a reference to a table in a column so that a product has say ID, Name, Description, Price, Details_Table_ID or something similar to save on queries. Do tables have unique ids in Mysql? Or how does the Stackoverflow community suggest I go about this? Thanks.
EDIT
Silly me, I have just remembered that every table name is uniques so I can just use that, so my question changes to how I can write a query that contains one cell in a table A to be used as a reference to a Table name.
Don't use separate details tables for each category, use a generic details table that can store any attribute. Its columns would be:
Product_ID INT (FK to Products)
Attribute VARCHAR
Value VARCHAR
The unique key of this table would be (Product_ID, Attribute).
So if Product_ID = 1 is a computer, you would have rows like:
1 Processor Xeon
1 RAM 4GB
1 HDD 1TB
And if Product_ID = 2 is shoes:
2 Material Leather
2 Size 6
2 Gender F
If you're worried about the space used for all those attribute strings, you can add a level of indirection to reduce it. Create another table Attributes that contains all the attribute names. Then use AttributeID in the Details table. This will slow down some queries because you'll need to do an additional join, but could save lots of space
Think about just having a single ProductDetails table like this:
ProductDetailID (PK)
ProductID (foreign key to your Products table)
DetailType
DetailValue
this way you do not have to create new columns every time you add a new product detail type. and you'll have many ProductDetail rows for each productid, which is fine and will query ok. Just be sure to put an index on ProductDetails.ProductID !
Since this is an application so you must be generating the queries. So lets generate it in 2 steps. I assume you can add a column product_type_id in your Product table that will tell you which child table to user. Next create another table Product_type which contains columns product_type_id and query. This query can be used as the base query for creating the final query e.g.
Product_type_id | Query
1 | SELECT COMPUTERS.* FROM COMPUTERS JOIN PRODUCT ON COMPUTERS.PRODUCT_ID = PRODUCT.PRODUCT_ID
2 | SELECT SHOES.* FROM SHOES JOIN PRODUCT ON COMPUTERS.PRODUCT_ID = PRODUCT.PRODUCT_ID
Based on the product_id entered by the user lookup this table to build the base query. Next append your where clause to the query returned.
I am trying to figure out the best method to design a database that allows users to rank a number of items.
The items are ranked by all users.
Every item will have a rank assigned to it automatically in the beginning. So when I create a user, the rankings table will be auto populated.
So something like this:
users (id, name) - 100,000+ entries
items (id, name) - never more than 1,000 entries
The only thing I can currently think of to house the rankings is this:
rankings (id, user_id, item_id, ranking)
But that feels wrong because I'll have 100 million entries in the rankings table. I don't know if that's ok? What other option could I use?
Can each user assign either zero or one ranking to each item? Or can she assign more than one ranking to a given item?
If it's zero-or-one, your ranking table should have these columns
user_id INT
item_id INT
ranking INT
The primary key should be the composite key (user_id, item_id). This will disallow multiple rankings for the same item for the same user, and will be decently efficient. Putting a separate id on this table is not the right thing to do.
For the sake of query efficiency I suggest you also create the covering indexes (user_id, item_id,ranking) and (item_id, user_id, ranking).
If you can get your hundreds of thousands of users to rank all 1000 items, that will be great. That's a problem any web app developer would love to have.
This table's rows are reasonably small, and with the indexes I mentioned it should perform decently well both at a smaller scale and as you add users.
We re-factored our user specific configuration from
create table user (id, name, ..., config1, config2, config3, ..) to
create table user (id, name, ...);
create table user_config (id, user_id, config_val);
Our MySQL database size increased by a factor of 2 after making this change and migrating the users from the older table to the newer table. We made this so that user configuration can be made extensible, but why does the space requirement go up because of this. What could be the reason.
If you had an oritinal table with 20 fields, and 1,000,000 users, that would be 20 * 1,000,000 = 20,000,000 items of data.
Say, for example, you now have the same number of users, but decrease the table to 10 fields, and had 10 config rows with three fields each (as per your code). This would be 10 * 1,000,000 + 10 * 3 * 1,000,000 = 50,000,000. This would be a factor of 2.5.
So, basically, for each configuration variable, you are now adding an id (Primary Key), and a user (Foreign Key) field. Added to that, there is now more indexing data that has to be generated.
SO, it could very well be the case that your data requirements have dramatically increased.