I have the question exactly as this one:
Nodejs Cluster with MySQL connections
I want to know what is the recommended approach to create database connections for nodejs clusters - either create one database connection to be shared across all workers, or to create one connection per worker.
The above thread doesn't answer this, rather it only answers what is 'working without issues'.
What I would like to know, is whether there are any scaling issues or concurrent-request issues that I would run into if I were to use any one of the 2 approaches when using MySQL database?
What is the approach that people are using at scale? Am I better off creating one connection per worker?
Related
Oracle's database link allows user to query on multiple physical databases.
Is there any MySQL equivalent ? Workaround ?
I want to run a join query on two tables , which are in two physical databases. Is it possible in MySQL ?
I can think of four possible workarounds for your scenario:
use fully-qualified-table-names when querying for the external table. MySQL supports the dbname.tablename-syntax to access tables outside the current database scope. This requires that the currently connected user has the appropriate rights to read from the requested table in another physical db.
if your external database is running on a different MySQL server (either on the same machine or via a network connection) you could use replication to constantly update a read-only copy of the remote table. Replication is only possible if you're running two separate MySQL instances.
use the FEDERATED MySQL storage engine to virtually import the table into your current database. This lifts the requirement of giving the current user access rights into the second database as the credentials are given with the CREATE TABLE-statement when using the FEDERATED storage engine. This also works with the databases running on different physical servers or different MySQL instances. I think that this will be the poorest performing option and does have some limitations - more or less important depending on your usage scenario and your requirements.
This is an extension to method 1. Instead of having to specify the fully-qualified-table-names every time you request information from your external table, you simply can create a view inside your current database based on a simple SELECT <<columns>> FROM <<database>>.<<table>>. This resemble the way, the FEDERATED-method works, but is limited to tables on the same MySQL instance.
Personally I'd consider method (4) as the most useful - but the others could also be possible workarounds depending on your requirements.
There's no MySQL equavilent method at the moment, see this post. However as the poster suggest you can do a work-around if the databases are on the same machine, by just adding the database-name in front of the table-name.
Also see this, it's 6 years old, but still not resolved. It's closed and probably not on their todo-list anymore.
I have multiple shops (a few hundred) and they all need to write an online Mysql database, and at about the same time. The program I have has been written in VB6 and it is currently updating the database successfully (via ODBC), although it is only updating three stores at the moment. The plan for the near future is to have all the other stores update to the same Mysql database.
Will Mysql be able to handle all these stores updating it at the same time using ODBC or is there a better way to do it?
In terms of running into problems caused by too many concurrent connections it depends very much on the platform your database is running on.
https://dev.mysql.com/doc/refman/5.5/en/too-many-connections.html
Creating a test environment to stress test your database would be a sensible way for you find out if it is going to be up to it.
If your database cannot cope with the load, there are many scalable cloud based resources around now which can easily be expanded to cope with higher load. Google Cloud, Windows Azure or Amazon may be worth a look.
Question 1:
I am using MySQL Connector /J to connect to MySQL. I am creating connection for every request. I need to use connection pool. Whether i need to choose c3p0 or i could use MysqlConnectionPool class provided by the connector library.
Question 2:
I may need to load balace / failover between two MySQL database servers. I could use jdbc:mysql://host,host2/dbname to do the failover automatically. I want to use connection pool and failover in combination. How should i acheive it.
I'd recommend using C3PO or something else. It'll integrate into a Java EE app server better, and it's database agnostic.
Your second question is a good deal more complicated. Load balancing is usually done with an appliance of some kind, like an F5 or ACE, that stands between the client and the load balanced instances. Is that how you're doing it? How do you plan to keep the data in synch if you load balance between the two? If the connections aren't "sticky", you'll expect to find INSERTed data in both instances.
Maybe this reference can help you get started:
http://www.howtoforge.com/loadbalanced_mysql_cluster_debian
I have several Rails apps running on a single MySQL server. All of them run the same app, and all of the databases have the same schema, but each database belongs to a different customer.
Conceptually, here's what I want to do:
Customer.all.each do |customer|
connection.execute("use #{customer.database}")
customer.do_some_complex_stuff_with_multiple_models
end
This approach does not work because, when this is run in a web request, the underlying model classes cache different database connections from the A/R connection pool. So the connection on which I execute the "use" statement, may not be the connection the model uses, in which case it queries the wrong database.
I read through the Rails A/R code (version 3.0.3), and came up with this code to execute in the loop, instead of the "use" statement:
ActiveRecord::Base.clear_active_connections!
ActiveRecord::Base.establish_connection(each_customer_database_config)
I believe that the connection pool is per-thread, so it seems like this would clobber the connection pool and re-establish it only for the one thread the web request is on. But if the connections are shared in some way I'm not seeing, I would not want that code to wreak havoc with other active web requests in the same app.
Is this safe to do in a running web app? Is there any other way to do this?
IMO switching to a new database connection for different requests is a very expensive operation. AR maintains a limited pool of connections.
I guess you should move to PostgreSQL, where you have concept of schemas.
In an ideal SQL world this is the structure of a database
database --> schemas --> tables
In MYSQL, database and schemas are the same thing. Postgres has separate schemas, which can hold tables for different customers. You can switch schema on the fly without changing the AR connection by setting
ActiveRecord::Base.connection.set_schema_search_path("CUSTOMER's SCHEMA")
Developing it require a bit of hacking though.
Switching database by connecting/disconnecting is really slow, and is not going to work due to AR connection pools an internal caches. Try using ActiveRecord::Base.table_name_prefix = "customer_" and keep the database constant.
Right now you have connections in ActiveRecord can be per class level. Its looks per thread basis because is in before 1.9 ruby threads sucked so implementations were using process instead of thread, but It may not be true for long.
But since AR uses one thread per Model. You can create different mock models for each database you have. So using answer given in this question.
Code will look something like this. (I have not tested it)
Customer.all.each do |customer|
c_class = Class.new(ActiveRecord::Base)
c_class.establish_connection(each_customer_database_config)
c_class.table_name = customor.table_name()
c_class.do_something_on_diff_models_using_cutomer_from_diff_conn(customer.id)
c_class.clear_active_connections!
end
Why not keep the same db and tables and just have each of your models belong_to a customer? Then you can find all the models for that customer with:
Customer.all.each do |customer|
customer.widgets
customer.wodgets
# etc
end
Oracle's database link allows user to query on multiple physical databases.
Is there any MySQL equivalent ? Workaround ?
I want to run a join query on two tables , which are in two physical databases. Is it possible in MySQL ?
I can think of four possible workarounds for your scenario:
use fully-qualified-table-names when querying for the external table. MySQL supports the dbname.tablename-syntax to access tables outside the current database scope. This requires that the currently connected user has the appropriate rights to read from the requested table in another physical db.
if your external database is running on a different MySQL server (either on the same machine or via a network connection) you could use replication to constantly update a read-only copy of the remote table. Replication is only possible if you're running two separate MySQL instances.
use the FEDERATED MySQL storage engine to virtually import the table into your current database. This lifts the requirement of giving the current user access rights into the second database as the credentials are given with the CREATE TABLE-statement when using the FEDERATED storage engine. This also works with the databases running on different physical servers or different MySQL instances. I think that this will be the poorest performing option and does have some limitations - more or less important depending on your usage scenario and your requirements.
This is an extension to method 1. Instead of having to specify the fully-qualified-table-names every time you request information from your external table, you simply can create a view inside your current database based on a simple SELECT <<columns>> FROM <<database>>.<<table>>. This resemble the way, the FEDERATED-method works, but is limited to tables on the same MySQL instance.
Personally I'd consider method (4) as the most useful - but the others could also be possible workarounds depending on your requirements.
There's no MySQL equavilent method at the moment, see this post. However as the poster suggest you can do a work-around if the databases are on the same machine, by just adding the database-name in front of the table-name.
Also see this, it's 6 years old, but still not resolved. It's closed and probably not on their todo-list anymore.