I have linux machine ( ubuntu 14.04 32GB RAM 8 core ...)
I want to run on this machine several Slaves ( currently 5 slave replications but I will need more )
I use master-slave mysql replication
From my point of view there are 2 options to do that
1. use mysqld_multi - set 5 instance ( done that in the past with 2 instance)
2. use Docker container - each one with mysql slave
what is the Best solution ?
which one will be more easy to maintain ( and add ) ?
Tnx for help
The issues I'm trying to solve are:
I have problems with Performance and with the architecture that we use I’m unable to use cluster - so I want to use load balancing and split the read/write
On one of my machines I need to split data from 1 master to different slaves by some column value - and i want all slaves will be on same machine
I would suggest reading the book High Perfromance MySQL.
If you're having performance problems but your server has enough resources to add multiple slaves then you should tweak the configuration of your master MySql instance to better utilize those resources. If you're attempting to split the writes to slaves those changes will never be propagated back to the master. If you want to utilize slaves to increase read performance then you can do that, but I would only suggest this if you've maxed out the box that the master instance is running on.
I would need more information on why you think you need to do this: 'On one of my machines I need to split data from 1 master to different slaves by some column value' to be able to comment about it. At the surface it feels like this is a bad idea, but there could be a reason for it.
Related
I'm looking for a best solution that suits my requirements. I would like to use MySQL with a lot of instances, so I need to be able to add as much master servers with slaves servers as might be needed in the future. There also will be sharding. Currently I've found out that GCP doesn't allow you to add more than one master server to a running instance. If so, what can I do then? I need to create 3 or more master servers and add slave servers to them. And if there is a new row in one of the master servers, the 3 slaves will receive that row and everything will by synchronized, so I'll be able to do a simple SELECT query in one of these slaves to get the actual data. I'm sorry for my english, I'm not a native speaker :)
What you are looking for it's called read replica. Using Google Cloud Cloud SQL for MySQL will let you implement a setup like the one you are describing, deploying multiples read replicas really fast
For the sharding part, you just need to deploy multiple masters with its own read replicas and on your application logic implement the needed code to find the data in the right instance.
I'm launching a big project of mine which took me more than a year.
if this work as I want, this would need a solid database architecture, but I'm a developer and kind of noob for database configuration.
Application is made with Spring/Mysql served with tomcat 8.
So after some research i've concluded this :
I will use mysql cluser with InnoDB engine
To start I need 3 server minimum for mysql clustering
To handle this correctly I'll use ProxySQL to route correctly queries on right node
Left some question:
How does replication works for migrations, if I want to add a column or a table does this replicates on all of nodes ?
I want to take 3 more server on cloud to do this, how much ram is the best for this kind of use ?
Is there better options ?
So what's the idea behind a cluster?
You have multiple machines with the same copy of the DB where you spread the read/write? Is this correct?
How does this idea work? When I make a select query the cluster analyzes which server has less read/writes and points my query to that server?
When you should start using a cluster, I know this is a tricky question, but mabe someone can give me an example like, 1 million visits and a 100 million rows DB.
1) Correct. Every data node does not hold a full copy of the cluster data, but every single bit of data is stored on at least two nodes.
2) Essentially correct. MySQL Cluster supports distributed transactions.
3) When vertical scaling is not possible anymore, and replication becomes impractical :)
As promised, some recommended readings:
Setting Up Multi-Master Circular Replication with MySQL (simple tutorial)
Circular Replication in MySQL (higher-level warnings about conflicts)
MySQL Cluster Multi-Computer How-To (step-by-step tutorial, it assumes multiple physical machines, but you can run your test with all processes running on the same machine by following these instructions)
The MySQL Performance Blog is a reference in this field
1->your 1st point is correct in a way.But i think if multiple machines would share the same data it would be replication instead of clustering.
In clustering the data is divided among the various machines and there is horizontal partitioning means the dividing of the data is based on the rows,the records are divided by using some algorithm among those machines.
the dividing of data is done in such a way that each record will get a unique key just as in case of a key-value pair and each machine also has a unique machine_id related which is used to define which key value pair would go to which machine.
we call each machine a cluster and each cluster consists of an individual mysql-server, individual data and a cluster manager.and also there is a data sharing between all the cluster nodes so that all the data is available to the every node at any time.
the retrieval of data is done through memcached devices/servers for fast retrieval and
there is also a replication server for a particular cluster to save the data.
2->yes, there is a possibility because there is a sharing of all the data among all the cluster nodes. and also you can use a load balancer to balance the load.But the idea of load balancer is quiet common because they are being used by most of the servers. but if you are trying you just for your knowledge then there is no need because you will not get to notice the type of load that creates the requirement of a load balancer the cluster manager itself can do the whole thing.
3->RandomSeed is right. you do feel the need of a cluster when your replication becomes impractical means if you are using the master server for writes and slave for reads then at some time when the traffic becomes huge such that the sever would not be able to work smoothly then you will feel the need of clustering. simply to speed up the whole process.
this is not the only case, this is just one of the scenario this is only just a case.
hope this is helpful for you!!
I have never used a master/slave setup for my mysql databases so please forgive if I make no sense here.
I am curious, let's say I want to have a master DB and 3 slave DB's. Would I need to write my database classes to connect and add/update/delete entries to the master DB or is this automated somehow?
Also for my SELECT queries, would I need to code it to randomly select a random DB server?
What you want to use (and research) is MySQL Replication. This is handled completely independent of your code. You work with the database the same as if there were 1 or 100 servers.
you sound like you are wanting to improve performance/balance load
yes you need to do any destructive changes to the master database. the slaves can only be used for readonly. you would also need to be careful that you don't write to the master and read from the slave instantaneously, otherwise the data may not have been replicated to the slave yet. so any instantaneous reads would still need to come from the master.
i wouldn't suggest just randomly selecting a slave. you could do this by geographical region if they are spread out, or if you are running in a cluster you can use a proxy to do the load balancing for you..
here is some more info that may help
http://agiletesting.blogspot.com/2009/04/mysql-load-balancing-and-read-write.html
You should consider using mysqlnd_ms - PHP's replication and load balancing plugin.
I think this is better solution, especially for a production environment, since it's native to PHP and MySQL Proxy is still in Alpha release.
Useful links:
https://blog.engineyard.com/2014/easy-read-write-splitting-php-mysqlnd
http://pecl.php.net/package/mysqlnd_ms
The master/slave set up should be handled automatically by the MySQL server, so you should not need any special code for this configuration.
I'm relatively new to MySQL replication. In a nutshell I have a MySQL 5.1 server instance on 3 Ubuntu Lucid Lynx servers.
ATM I have Server A (MASTER) which replicates a single database to Server C (SLAVE).
This has been running for a couple months without a problem.
I now want to have Server B (as MASTER) replicate a different single database to Server C (SLAVE).
I was looking into implementing this, but my initial reading seems to indicate that a replication slave server can't have two masters.
My question is, is this the case even when only single or select database are being replicated?
Keep in mind I do not wish to replicate the same database from different masters. I simply wish to replicate multiple separate database on a single server, from separate masters.
You can set a different master per database in MySQL.
The book high performance MySQL has a full chapter on this issue. I recommend getting hold of a copy and using the info in that.
http://oreilly.com/catalog/9780596101718/
You can read a copy of the relevant chapter online here:
http://oreilly.com/catalog/hpmysql/chapter/ch07.pdf
This is from the first edition, the second edition is more up to date, but chapter 8, which deals with replication is not freely available online.
UPDATE
The solution I'm talking about is only in the 2nd edition of the book, see this answer to a simular question: Is it possible to do N-master => 1-slave replication with MySQL?
In general you can't do this. You can't replicate from many-to-one (you can from one-to-many).
Also you can't really just replicate one database - cross-database updates in statement-based mode then become unreplicatable, which means your slave will fail (or become out of sync) as soon as someone does one on the master.
The standard solution is to install multiple mysql instances, which is far from ideal, but works.