Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I need to know if it is more or less efficient to have multiple databases with an index of databases relative to each dataset.
I do not know to what extent multicache can adversely affect performance.
Suppose 10 bases in 2GB data each rather than a single 20GB.
For example: the data of userid 293484 are in third database.
Thanks.
Yes, this is a common technique known as sharding.
http://en.wikipedia.org/wiki/Shard_%28database_architecture%29
Altimately the code you will have to write to maintain such a structure will kill you.
Keep it simple, keep it in one database, and use proper design patterns and indexing.
Database engines are design to deal with large amounts of data, so if your hadrware is sufficient, your queries well structured and the design good, you should not have to many performance problems.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I am trying to decide on what will be more efficient between two different systems for MySQL to handle my data. I can either,
1: Create around 200 tables, each having around 30 rows & 5 columns.
2: Create 1 table, having around 6000 rows & 5 columns.
I am using Laravel for this project and Eloquent will be handling this. Does anybody have any opinions on this matter? I appreciate any/all responses.
Option 2.
For such low row counts the overhead both in terms of programming effort and computation of joining 200(!) tables far outweighs the "flat file" approach. Additionally, MySQL will attempt to cache the entire 6000-row table in RAM, assuming you're not storing massive BLOBs.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I've been trying to get my head round some very tricky SQL queries in MySQL (can range from nested queries, correlated sub queries, group concatenation, temporary tables and self joins). These are often very large and very complicated.
Recently I've been thinking of ways to try and improve the way I do this. Sometimes I try to think how a single record would be included in a dataset and follow how the keys bring together tables. Other times I think of the entire join table and mentally strip away rows according to the WHERE constraints.
Is it worthwhile looking at relational algebra to understand what is going on?
In summary, what strategies do you use for analysing large, complicated SQL queries?
For me, it was just experience. The more I had to interact with such large, complicated codes and the more questions I asked from professors, friends, coworkers, the better I came at being able to understand everything that is going on in a code.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
For a website having 2000000 users, out of which each user shares thousands of pictures and on each picture there will be thousands of comments, in such scenario there will be more than 2000 million comments, so how can I manage this much of big data using MySql. How can following methods improve performance of my database server
Use of table partationing
Use MySQL clusters
Use MySQL with memcached
Please explain other methods and best practices to handle such big database tables
On top of the mentioned optimization, choosing the right indexes on the right fields is crucial for your query performance, make sure your tables are indexed on everything you group, order or search based on.
Also make sure to check out Chapter 8 of the MySql reference which discusses optimization
What you really should be focusing on is optimizing the structure, queries and indexes before getting into memcached and MySql clusters.
As your database grows you monitor the performance and optimize accordingly.
In this case i dont thinl traditional RDBMS is what you need :) , more like NoSQl is what would serve you best
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I have a copy of the PAF (UK Postcode) database it is currently stored in a MySql Database, and i use it on my site to pre-fill address details, however the Database huge 28,000,000+ records and it is very slow to search.
Any ideas how I could slit the DB to improve performance?
Thanks for the help guys!
that is not a large database, not even a large table. you must set appropiate indexes over the table and you will get good performance
There could be several ideas:
create indexes, meaningful ofcourse
review your schema. Avoid using huge datatypes like INT, BIGINT, TEXT etc unless absolutely required
optimize your queries so they use indexes, EXPLAIN statement might help
split your table into multiple smaller tables, say for example based on zones - north, east, west, south etc.
If your table doesn't require many INSERTs or UPDATEs, which I assume it might not being a postcode table, query cache can be a big help for faster queries
You will need to play around and see what option works best for you. But I think the first 2 should just be enough.
Hope it helps!
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have 9 tables (total of 43 fields). I have access to many 1GB MySQL databases. Should I split my 9 tables over multiple databases or just pile them all into one database?
The answer depends on how much data is going to be in each table.
A table itself takes up almost no space - it's the rows that make the database size grow.
What you need to do is estimate how large each table is going to get within the foreseeable future - erring on the side of keeping the tables together.
That said, nine tables with 43 fields (assuming reasonably sized rows) would need to have hundreds of thousands of rows each to approach 1GB. I have a multi-million-row SQLite file which is only 100MB.
It depends.
How much data are you expecting?
How much more complicated is it if you have to manage multiple databases?
How much slower will it be to query multiple databases and aggregate the results?
How important is performance?
Putting everything in a single database will give you better performance (usually) and is easier to develop. You should do that until your data gets big enough that you outgrow the database.