Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
Is the basic technique behind querying across logical shards just querying them all at the same time and consolidating the results?
There doesn't seem to be any built-in features of MySQL or Postgres that allows you to query across logical shards, so I assume you must query each shard or get some sort of software to sit in front of that database that indexes or queries for you.
MySQL is working on a new technology called MySQL Fabric to do this. It's still in early development (as of this writing). But they apparently intend it to be a built-in feature in MySQL 5.7.
You can also use Shard-Query today. This acts as a proxy to query across all your shards transparently. That is, you can write simple SQL queries as if you didn't have a sharded architecture. Shard-Query rewrites SQL and runs queries against each shard in parallel, then combines the results.
I don't know what, if any, solutions exist for PostgreSQL to automatically query across shards.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
Currently, I'm developing one project and there are lots of MySQL query operations with billions of records and also mathematic operation included and it takes more time to perform the query.
so I need your help in choosing technology for big data and DB operation
Currently, I'm using nodejs and MySQL DB
Thanks for giving me the right way to develop this.
It depends on your data. If your data is homogeneous (most of the rows has the same number of columns) and you need to perform complex queries with tons of joins, using a relational database as MySQL is a good option. You can also try other relational databases like Oracle DB, MariaDB and others. It shouldn't be difficult to export your current database and try if the performance improves.
On the other way, if your data is heterogenous and you don't need to perform complex join queries, a NoSQL database can be your option. There are a lot of them but one of the most famous ones is MongoDB. Moreover, Mongo has very good integration with NodeJS. Your main problem would be to convert your actual relational database to a non-relational database.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
i work on a project with node.js and a mysql database.
i have a connection between them with the npm modul mysql.
Now my question:
is it better to send a SQL command and sort the data in node.js or multiple detailed sql commands?
what is faster / more performant?
Thanks.
Without knowing the exact SQL queries, I would say that database operations are faster compared to your own implementation. Many smart people have worked to assure performance, accuracy, atomicity, concurrency etc. of the Mysql engine.
Even if you can gain marginal improvements in some aspect with your own code, it is unlikely that you will be able to justify the investment.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am working in an environment that is under extreme load. It is a DB used by about a thousand users with one application. This application does thousands of queries against the DB. We have noticed significant performance degradation over time and are looking for a long-term solution to this problem. Of course, query optimization is one of the tasks we are working on, and we are also optimizing indexes; however, this will not be enough to see the performance gains we need.
I have worked in SQL Server for several years but my MySQL knowledge is limited. To start scaling MySQL, I've researched Sharding, but as we are using MySQL community edition, I'm nervous that this will cause more headaches than it's worth. The only other possibility is to re-design the application, specifically how it pulls data from the DB, but I'd rather not do that.
So my question is, is sharing worthwhile to pursue? Is it feasible without an enterprise edition of MySQL? Is there another possibility you could recommend?
Turn on the slowlog with long_query_time=1. Wait a day. Use pt-query-digest to identify the 'worst' could of queries. Then let's discuss them. Sometimes it involves the trivial addition of a 'composite' index.
That is, Slow queries is almost always the cause for scaling problems.
If we eliminate that as a problem, then we can discuss sharding and other non-trivial approaches.
We must see SHOW CREATE TABLE and other clues of what is going on.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I am curios why some prefer MYSQL over SQLITE, i am interested on learning MYSQL
SQLite is an embedded library, it requires no server process, and everything it saves is contained within a single, portable file. MySQL is an RDBMS server that's a lot more work to set up, but is multi-user, more scalable, and far more featured.
For example, SQLite is used for mobile applications as well as "development" instances of code where it's only lightly loaded. It can be used at scale but generally isn't, its simple design has limitations.
If you're writing a mobile application and need a local database, SQLite is not a bad call. Spinning up a huge, cumbersome MySQL process to do the same thing on a mobile device is a bad plan.
Although they're both "SQL databases" and have a lot of functional overlap, they're engineered to solve some very different problems. In some trivial use cases it might be an arbitrary choice as to which is best, but in most cases it's pretty obvious which of the two you need.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I will be writing a program in Delphi that will be reading and writing to a MySQL database tables on a regular basis like every 5 seconds. Is this going to be CPU intensive? or get to a point where computer will freeze completely? I know reading and writing to and from a hardrive nonstop can freeze everything on your computer. I am not really sure about MySQL database.
Databases are designed to handle many transactions frequently, but it really depends on what the queries you are using. A simple SELECT on a couple rows is unlikely to cause an issue, but large scale updates targeting many tables or multiple joins can slow performance. It all depends on what your queries are.
This all depends on the computer and the complexity of the query.
As David has said, it really does depend on the hardware and queries you are processing.
I would suggest measuring the processing time of each query to determine whether the writing processes will be stacking over the other 5 second interval queries.
You can find information on how to measure your MySQL processes here.