My MYSQL servers are not configured properly with failover. I am thinking of using Redhat cluster or heartbeat. Also, I need to achieve all tasks w/ one floating IP since application does not know about multiple databases. Can someone suggest what route would be the very for best?
If you decide on using regular replication I would recommend you using this:
MHA tool for MySQL replication high availability
I didn't have a good experience with the ways you want to use, they are not as good as a cluster and not so much better than normal replication with the tool I posted. I do believe you should take a look at this tool and maybe stick with regular replication or jump to serious HA with MySQL cluster. Configuring the cluster might be hard but using Several Nines Solutions things might be a bit easier.
Related
Let's say I have several different containers and each one of them uses it's own database. What is the best practice in this case regarding performance? Run one container, say a MySQL server, with all the databases in there or run one database server container per database?
Any other comment besides the performance would be welcome.
Since Docker container overhead is not significant and negligible here, the question is more about architecture in a microservices paradigm.
Performance is indeed a complex question and there is no general advice, but maybe the following will help you:
Personally, I doubt that at the beginning of the project one should try to solve all possible performance problems in advance (#MVP, #agile)
However, correct me, but it looks like you have not many resources (one host?) and want to be thrifty with these resources in advance.
Ok, what is your biggest concern now?
RAM is a concern
Then having two concurrent MySQL instances on the same host is maybe not that good (but not a problem for different setups)
For one host I would propose to start using one Database container but create different schemas.
It might involve additional work with standard container (https://forums.docker.com/t/multiple-databases-in-official-mysql-container/8324)
Other concerns
I would not care too much now and start with separate databases from the beginning.
Being able to separate your services horizontal to the databases is a huge value! I would not want to weaken this design decision because of very theoretical future performance issues.
You'd want to use a single database server, preferably running a shell you can attach to for administration, sharing either a Unix socket, a port or both to linked containers. This means you'll have an easier time managing the database container as a service, tweaking performances, monitoring usage, backing up volumes, etc.
Granted, there might be non-standard situations where you might want to have independent servers, for instance running servers with isolated host resources, users, databases, though I'm certain this shouldn't apply to developer environments.
I'm running MySQL servers load balanced via HAProxy. But I need to perform some action whenever a node (MySQL) goes down or comes up, like we do in PgPool-II.
Example:
When ever MySQL-1 goes down, I also want to shutdown my WebServer-1. (STONITH like)
In PgPool-II for PostgreSQL, I can do this. I'm unaware, if there are similar tools for MySQL.
Is this possible in HAProxy itself:
Say, when my DB-1 goes down, route the web server traffic, to WebServer-2.
MySQL has quite an extensive support for replication, high availability and sharding. Your question is not very clear to me but here are a few things you could try reading:
For failover, MySQL provides a python script called mysqlfailover which is shipped as part of MySQL utilities.
For sharding and high availability you could try out MySQL Fabric. MySQL Fabric is a relatively new product so expect it to not provide all kinds of fancy sophisticated sharding schemes in theory but it is stable enough for you to trust it with what it provides.
By the way both of them are open source so you could tweak it to suit your need!
What are some generally accepted strategies for restarting a MySQL server on a busy website without interrupting current users? I am using a LAMP setup. I don't mind taking down the site for a time if need be, but if certain user activities are interrupted I could wind up with corrupted data. I do have the ability to bring up a second server if that helps in the transition. I need a solution that results in no corrupted data / data loss.
I suspect this could be a common problem without an easy solution, but not sure what the best approach would be. Any guidance would be appreciated.
Thanks, Brian
Any solution for high availability depends on redundancy.
The most popular strategy today is to run two MySQL servers. Configure the two servers to replicate bidirectionally. This comes with its own challenges; you must manage your applications carefully to write to only one server at a time, to avoid creating update conflicts. When you need to restart one MySQL server, switch your apps to use the other MySQL server.
Even with this configuration, you can't make the switchover without interrupting connections, even if the interruption is brief.
Another solution is MySQL Cluster, in which both MySQL Servers and storage are redundant, but this is also complex to set up and manage, requires high-end hardware resources, and shards your data in ways that make it hard to optimize for general SQL queries.
We are currently evaluating failover support in different databases.
We were earlier using HSQLDB but it seems that it does not have clustering/replication support.
Our requirement is simply to have two database servers, one being only for synchronous backup but if the primary server is down, then the secondary should automatically start acting as the primary server.
Has anyone evaluated MySQL, PostgreSQL or any other DB server for such a use case?
Edit: We had thought of using MySQL cluster but it now seems that it is under GPL license which we won't be able to work with. Could anyone please suggest a synchronous replication/clustering solution which can be used? We are currently using HSQL, so a solution with HSQL used in clustered mode will be ideal for us but we are open for change.
Stackoverflow resources
MySQL supports replication out of the box: see this question for MySQL: Scaling solutions for MySQL (Replication, Clustering)
PostgreSQL also support replication, see this question for that: PostgreSQL replication strategies
If your requirements are simple MySQL will work
I've used MySQL is a simple master-master failover scenario using the setup I read in High Performance MySQL. I highly recommend the book if you're keen on using MySQL.
It has worked well for me, because I just wanted a simple fail-over.
If your use case is just as simple. It will work well.
Just for completeness, the H2 database has some clustering support, but compared to the MySQL and PostgreSQL features it is very limited, it's really only failover. I would first look at HA-JDBC.
for a simple failover where servers are on the same location. you can use DRBD and Heartbeat.
In a nutshell: DRBD stores the data on 2 servers on the same time. fully transparent to the system. with heartbeat the standby checks against the main server, if its not reachable, it takes over the resource, mounts it and starts the database daemon. (works with mysql, postgres and most probably with most other daemons out there)
There is a third-party product that works with HSQLDB:
http://ha-jdbc.sourceforge.net/
Not sure this is within the desired price range of most FOSS-type people :-) but we use DB2 9.7 for exactly this purpose (actually, we mostly use DB2/z on the mainframe for it, but some customers like the DB2/LUW (Linux/UNIX/Windows) option for smaller systems).
DB2 comes with high availability (HA) features built in and you can use db2haicu, the DB2 High Availability Instance Configuration Utility (gotta love those acronym generators employed by Big Blue) to configure things relatively painlessly.
It's active/passive as you desired, although DB2 is certainly capable of active/active setups for load balancing.
The particular setups we're most familiar with at the low end (everything other than a mainframe) are actually shared disk ones, with the HA applying to only DBMS resources and not data, but you can separate the data with DB2 replication features as well.
We've had one client (at least) using Q replication, which is a very low latency replication method, close to synchronous but not quite. DB2 does actually provide real synchronous replication as well.
DeveloperWorks has an interesting article on how this all hangs together, along with the various options.
The main bottle neck of a web server locates usually in the database,in my case,MYSQL.
More specifically,fulltext search and master-slave replication.
And sphinx is a probable solution for fulltext-search,so master-slave replication is the
final pain in ass.
Is it possible to boost the performance significantly with the technology of Cloud Coumputing,
for instance,by services offered by Amazon?
Just a wild guess!
EDIT:what about MySQL and Google App Engine?
Of course. MySQL Enterprise for Amazon EC2 is one MySQL package for Amazon EC2. See also Setting Up MySQL on an EC2 AMI and this tutorial/blog post.
EDIT: App Engine is higher-level than EC2 and is really designed for BigTable/GQL only. However, look at approcket, which allows replicating between AppEngine and MySQL.
You may want to be careful with just switching your web app to use an external data base (ie amazon, et.al.), you want to understand where exactly is your bottleneck or you may end up introducing more performance problems... Remember that by going to an external DB, you're introducing more latency into each query compared to a local (box or net) query.
If your problem is performance, try to find out exactly where the problem lies first, and then you may want to explore other options like query optimization, caching, etc.
Possible - for sure. See for example, xeround, rightscale, Amazon and phpfog. There are probably at least a few more with more to come. They come in varying degrees of "freeness" (How's that for a made up word?) too.
The question, it seems to me, will be performance and reliability.
Who knows, localhost may become a thing of the past for development.