Every page on my application will display 1000 records and they are only needed on that page.
Is it more efficient to have one huge table with an additional 'page id' column or to have a new table for each page?
I imagine accessing data from a single small table would be easier than a huge one but it will make the database a bit of a mess.
Any thoughts?
In no way would a single table for each page be a good idea. Very, very silly idea.
Databases are intentionally designed for storing large amounts of information. Just use a single table with a Page_id column. Just make to set up the proper indexes for data retrieval.
1000 columns is nothing, one table with a PK and index or two on columns mostly searched by is good. Simply use indexes on columns which appear in WHERE clause.
Do the pages have similar data? I.E. would the columns be the same for each page? If so, put it all in one table. Having too many tables makes things overly complicated when a modern database is capable of efficiently storing and accessing millions of records.
Related
I am currently working on the redesign of a mysql bdd.
This database is actually a huge table of 200 fields, several million lines, almost no index... In short, disastrous performances and huge ram consumption!
I first managed to reduce the number of fields by setting up 1:n relationships.
I have a question on a specific point:
A number of fields on this database are optional and rarely filled in. (sometimes almost never)
What is the best thing to do in this case?
leave the field in the table even if it is very often of null value
set up a n:n relationship knowing that these relationships, if they exist, will only return one line
...or another solution I haven't thought of
Thank you in advance for your wise advice;)
Dimitri
My suggestion:
First of all, make sure to normalize your db. At least to 3rd Normal Form. This will probably reduce some of your original columns and split them over several tables.
Once that is done, in case you still have lots of 'optional and rarely filled' columns in some of your tables, consider how to proceed depending on your needs: What is most important to you? Disk space or Performance?
Take a look at MySQL Optimazing Data Size for extra tips/improvements for the re-design of your database ...depending on your needs.
Regarding 'set up a n:n relationship...' (I figure you meant 1:1 relationship) can be an interesting option in some cases:
In some circumstances, it can be beneficial to split into two a table that is scanned very often. This is especially true if it is a dynamic-format table and it is possible to use a smaller static format table that can be used to find the relevant rows when scanning the table (extracted from here)
I already saw a few forums with this question but they do not answer one thing I want to know. I'll explain first my topic:
I have a system where each log of multiple users are entered to the database (ex. User1 logged in, User2 logged in, User1 entered User management, User2 changed password, etc). So I would be expecting 100 to 200 entries per user per day. Right now, I'm doing it in a single table and to view it, I just have to filter out using UserID.
My question is, which is more efficient? Should I use one single table or create a table per user?
I am worried that if I use a single table, the system might have some difficulty filtering thousands of entries. I've read some pros and cons using multiple tables and a single table especially concerning updating the table(s).
I also want to know which one saves more space? multiple table or single table?
As long as you use indexes on the fields you're selecting from, you shouldn't have any speed problems (although indexes slow writes, so too many are a bad thing). A table with a few thousand entries is nothing to mySQL (or any other database engine).
The overhead of creating thousands of tables is much worse -- say you want to make a change to the fields in your user table -- now you'd have to change thousands of tables.
A table we regularly search against for a single record # work has about 150,000 rows, and because the field we search for is indexed, the search time is in very small fractions of a second.
If you're selecting those records without using the primary key, create an index on the field you use to select like this:
CREATE INDEX my_column_name ON my_table(my_column_name);
Thats the most basic form. To learn more about it, check here
I would go with a single table. With an index on userId, you should be able to scale easily to millions of rows with little issue.
A table per user might be more efficient, but it's generally poor design. The problem with a table per user is it makes it difficult to answer other kinds of questions like "who was in user management yesterday?" or "how many people have changed their passwords?"
As for storage space used - I would say a table per user would probably use a little more space, but the difference between the two options should be quite small.
I would go with just 1 table. I certainly wouldn't want to create a new table every time a user is added to the system. The number of entries you mention for each day really is really not that much data.
Also, create an index on the user column of your table to improve query times.
Definitely a single table. Having tables created dynamically for entities that are created by the application does not scale. Also, you would need to create your queries with variable tables names, something which makes things difficult to debug and maintain.
If you have an index on the user id you use for filtering it's not a big deal for a db to work through millions of lines.
Any database worth its salt will handle a single table containing all that user information without breaking a sweat. A single table is definitely the right way to do it.
If you used multiple tables, you'd need to create a new table every time a new user registered. You'd need to create a new statement object for each user you queried. It would be a complete mess.
I would go for the single table as well. You might want to go for multiple tables, when you want to server multiple customers with different set of users (multi tenancy).
Otherwise if you go for multiple tables, take a look at this refactoring tool: http://www.liquibase.org/. You can do schema modifications on the fly.
I guess, if you are using i.e. proper indexing, then the single table solution can perform well enough (and the maintenance will be much more simple).
Single table brings efficiency in $_POST and $_GET prepared statements of PHP. I think, for small to medium platforms, single table will be fine. Summary, few tables to many tables will be ideal.
However, multiple tables will not cause any much havoc as well. But, the best is on a single table.
Ok, I am creating a game, I have one table where I save a lot of information about a member, so I have many field in it. How many fields is normal to have in one table? Does it matter? Maybe I should split that info into two-three-four tables? What do you think?
Normalize the Database
If you feel you have too many columns, you probably have repeating groups, which suggests you should normalize the database. See an example here: Description of the database normalization basics
Hard MySQL Limits
MySQL 5.5 Column Count Limit
Every table has a maximum row size of 65,535 bytes.
There is a hard limit of 4096 columns
per table
Splitting of data into tables should generally not be dictated by the number of columns, but by the nature of the data. The process of splitting a large table into smaller ones is called normalization.
The only other reason I can think of to split a table is, if you may need data in clusters, i.e. you often need columns A-D together or columns E-L, but never all columns or columns D-F, then you can split the table into two tables, one containing columns A-D and the primary key, the other one containing columns E-L and the primary key.
Speaking about limits, MySQL says it's 4096 (source).
Yet I haven't seen so big tables yet, even those huge data mining tables don't come close.
You shouldn't be concerned about it as long as your database is normalized. As soon as you can spot same data being stored twice (for example, player table might have player_type column storing some fixed values), it's worth moving such data to separate table, instead od duplicating information in other tables and hence reducing columns count (but that's only "side effect" of normalization, not something that drives it).
I've never personally encountered one with more than 500 columns in it, but short of the maximum sizes there's no reason to have any fewer than the design demands. Beware of doing SELECT * FROM it though.
"information about a member" - umm always difficult, but I always separate identifiable information into another table and use a salt key to link the 2 together. That way it is not as easy to "hijack" usernames and passwords etc. And you can always use the SALT as a session variable rather than username/password/userId or whatever.
Typically I only store a ID, salt and joining date in 1 table. As I said, the rest I try to "hide" so that they cannot be "linked/hijacked".
Hope helps
I am designing a database for a project. I have a table that has 10 columns, most of them are used whenever the table is accessed, and I need to add 3 more columns;
View Count
Thumbs Up (count)
Thumbs Down (Count)
which will be used on %90 of the queries when the table is accessed. So, my question is that whether it is better to break the table up and create new table which will have these 3 columns + Foreign ID, or just make it 13 columns and use no joins?
Since these columns will be used frequently, I guess adding 3 more columns is better, but if I need to create 10 more columns which will be used %90 of the time, should I add them as well, or create a new table and use joins?
I am not sure when to break the table if the columns are used very frequently. Do you have any suggestions?
since it's such a high number of usage cases (90%) and the fields are only numbers (not text) then i would certainly be inclined to just add the fields to the existing table.
edit: only break tables apart if the information is large and/or infrequently accessed. there's no fixed rule, you might just have to run tests if you're unsure as to the benefits.
Space is not a big deal these days - I'd say that the decision to add columns to a table should be based on "are the columns directly related to the table", not "how often will the columns be used".
So basically, yes, add them to the table. For further considerations on mainstream database design, see 3NF.
The frequency of usage should be of no concern for your table layout, at least not until you start with huge tables (in number of rows or columns)
The question to answer is: Is it normalized with the additional columns. Google it, there a plenty of resources about it (with varying quality though)
Ditto some earlier posters. 95% of the time, you should design your tables based on logical entities. If you have 13 data elements that all describe the same "thing", than they all belong in one table. Don't break them into multiple tables based on how often you expect them to be used or to be used together. This usually creates more problems than it solves.
If you end up with a table that has some huge number of very large fields, and only a few of them are normally used, and it's causing a performance problem, then you might consider breaking it up. But you should only do that when you see that it really is causing a performance problem. Pre-emptive strikes in this area are almost always a mistake.
In my experience, the only time breaking up a table for performance reasons has shown any value is when there are some rarely-used, very large text fields. Like when there's a "Miscellaneous Extra Comments" field or "Text of the novel this customer is writing".
My advice is the same as cedo's:go with the 13 columns.
Adding another table to the DB, with another Index might just eat up the space you saved but will result in slower and more complicated queries.
Try looking into Database Normalization for some clearly outlined guidelines for planning database structures.
I am working on a game that I am going to open to the public to have on their game.
The game stores lots of information (about 300 rows) per website and spends a lot of time updating values within this MySQL database.
Is it better (faster/efficient) to add a new table for every website or to just have 1000's of rows in one table and add a column "website_id" or similar?
It is better to add more rows in the same table and create an index on the table. An index on the website_id would probably help a lot.
Definitely go for rows with a website_id. Definitely index the website_id and it should be much more efficient -- I have about 50-60 sites running on a similar concept and it is blazing fast.
Could you perhaps give us some more information pertaining to the kind of information you're storing and how you are going to manipulate this data?
If a single table is going to have 300+ entries for what should be a single entity, you may have a flaw in your physical database design. Ideally, you want to keep your tables from having too many entries, but this is unavoidable at times.
You might also want to look into partitioning as a means of keeping your tables smaller, if you are going to have as much data as you think.
At a glance though, it appears that using a website_id would make sense. Create a table indexing just these, and all of your additional tables can use a field referencing the website_id where required.