I have question about data backup.
We are developing backed for mobile application.
So we have a few EC2 servers, one for api sub-domain and one for admin sub-domain. One RDS Mysql server for the database, also with 2 databases.
But I'm worried about one thing, RDS snapshots is good for database structure. If we will have some errors in application, or will need to revert some changes in structure.
I will just restore from yesterday snapshot. And how about content, because its adding every minute.
Maybe some one can describe mechanism or tools to prevent data our lost. Replications or something like that.
I think I've found the answer - bin log
https://dev.mysql.com/doc/refman/5.5/en/binary-log.html
Related
I am looking at creating a high availability Wordpress installation across two servers with a load balancer to distribute the traffic between these servers, I intend to setup a LAMP stack on these two web servers and configure MySQL master master replication to ensure both servers have the same content similar to the setup at https://www.linode.com/docs/websites/cms/high-availability-wordpress
For anyone created a similar setup before, is there anything to watch out for? Will this work without there being differences in the database assuming both nodes are online? If one node goes down will this synchronise with the other master node successfully?
This is a pretty old post, but I've set up exactly this recently.
I do not recommend setting up WordPress with master-master replication. The nodes went out of sync multiple times and resulted in split-brain. We've reverted back to master-slave replication because of the instability of the setup.
I recently tried this and regret it. I realize your post is now almost 2 years old, but thought I should put in my 2 cents for anyone considering this approach. I recommend you go with MariaDB with MaxScale to separate the reads and writes, and go with master-slave configuration. I'm running into all sorts of deadlocks, and issues with this configuration. Currently using MariaDB with Galera, with HAProxy up front.
This may be a stupid question, but after hours of googleing i cant find a suitable answer to this..
We have a buisness critical application running on cloudbees. The sourcecode is backed up properly and we want the same for our db. Cloudbees doc says:
"CloudBees MySQL databases are backed by EBS volumes on Amazon EC2 which provides a first layer of storage redundancy. EBS volumes are backed up to S3 every 24 hours for disaster recovery and are not generally available for customer use on multi-tenant MySQL clusters. Customers using Dedicated MySQL instances can request rollbacks to previous backup snapshots by filing a support ticket."
So basicly we are protected out of the box in case of emergencies, but not if an employee accidentally deletes something he should not.
So my question is: How can we automaticly do a backup of a cloudbees mysql db every night? We have amazon S3 storage where it could be put.
Any ideas?
You can use a command line script that backups your Databases to your S3 account quite easily, and run it as often as you like. I had exactly the same problem a while back, and wrote up this handy tutorial. It should be perfect for what you want to do.
I develop websites for several small clients and I would love to be able to keep my local databases up to date with each client's production servers. (I'm thinking nightly updates) Many of their databases are in the hundreds of megabytes, so I feel like creating and transferring complete dumps every night is excessive.
Here are the harebrained ideas I have come up with so far:
Create a dump on the server and rsync with the previous night's dump
on my local machine. That should only transfer the changed bits of
the file right?
Create a dump on the server, and locally diff it from last night's
dump. Transfer only that diff. Maybe it could also send to md5 of the
original dump so I could be sure I was applying the diff to the same
base file.
Getting ssh access to (most) of these servers is possible, but requires getting my clients to call their hosting providers with that rather technical request, which is something I would rather not have to ask them to do. Bonus points if you can suggest a solution that I could implement via ftp/php.
Edit:
#Jacob's answer prompted some further Googling which lead to this thread: Compare two MySQL databases
This question (in several forms) seems pretty common. Most of the need, and therefore most of the tools, seems to focus around keeping the schema up to date rather than the data. Also, most of the tools seem to be commercial and GUI.
So far the best looking option seems to be pt-table-sync from the Percona Toolkit, although it looks like it might be a pain to setup on OSX.
I am downloading Navicat to test right now. I runs on OSX but is a commercial GUI program.
I think the most efficient way to set this up is to use rsync to backup the files is by using rsync to run each night , doing a incremental backup by syncing yesterday's back to todays backup then running the rsync for that day. In terms of backing-up the different websites i would look at having the sites as slaves which back up to your server (master) using the replicator function this will allow you to keep all the databases upto date and backed up
http://dev.mysql.com/doc/refman/5.0/en/replication.html
I am using AWS RDS for MySQL. When it comes to backup, I understand that Amazon provides two types of backup - automated backup and database (DB) snapshot. The difference is explained here. However, I am still confused: should I stick to automated backup only or both automated and manual (db snapshots)?
What do you think guys? What's the setup of your own? I heard from others that automated backup is not reliable due to some unrecoverable database when the DB instance is crashed so the DB snapshots are the way to rescue you. If I am to do daily DB snapshots as similar settings to automated backup, I am gonna pay much bunch of bucks.
Hope anyone could enlighten me or advise me the right set up.
From personal experience, I recommend doing both. I have the automated backup set to 8 days, and then I also have a script that will take a snapshot once per day and delete snapshots older than 7 days. The reason is because from what I understand, there are certain situations where you could not restore from the automated backup. For example, if you accidentally deleted your RDS instance and did not take a final snapshot, you would not be able to access the automated backups that were done. But it is also good to have the automated backups turned on because that will provide you the point-in-time restore.
Hope this helps.
EDIT
To answer your comment, I use a certain naming convention when my script creates the snapshots. Something like:
autosnap-instancename-2012-03-23
When it goes to do the cleanup, it retrieves all the snapshots, looks for that naming convention, parses the date, and deletes any older than a certain date.
I think you could also look at the snapshot creation date, but this is just how I ended up doing it.
Just from personal experience, yesterday I accidentally deleted a table and had to restore from an RDS snapshot. The latest snapshot was only 10 minutes old, which was perfect. However, Amazon RDS took about 3 hours to get the snapshot online, during which time, the affected section of our site was completely offline.
So if you need to make a very quick recovery, do NOT depend on RDS backups.
Keep in mind, you can't download your snapshot so that you could view a database dump. Your only option is to wait for it to load in to a new database instance. So if you're only looking to restore a single table, RDS backups can make it a very painful process.
No blame to Amazon on this- they are awesome. But just something to keep in mind when planning, because it was a learning experience for us.
There are some situations where an automated backup does not recover the specific table you want to recover even though it has a point-in-time recovery feature. I am suggesting you enable the Backtracking feature for this kind of recovery and You can use "AWS-Backup" service to manage backups of Amazon RDS DB instances. Backups managed by AWS Backup are considered manual DB snapshots.
Also, you will be required to keep automated backup enabled for creating read-replica for DB-instance in order to improve read performance. The retention period for automated backup should be between 1 and 35 so you can keep it a minimum of 1 day.
I have a MySQL install on a shared server and have access through phpMyAdmin. I want to make a continuous, real time clone of that database to a cloud mySQL database (we have created an Nginx-ready MySQL server specially for this database) I want to create a real time clone of the old one, then update code to point to the new database...
I think you will have difficulty doing real-time replication of a MySQL in a shared server environment. Since you appear to be moving db servers, I would be inclined to do a hot copy of your data, and install that on the new db server. At the same time as taking that copy, you should switch on query logging on your application.
Your switch over would then consist of running logged queries against the new database (faster than they were logged!) and finally, at a point that all logged queries have been run, switching the configuration of the app so that the new db is used.
Edit: the problem with a hot copy is that data is being written to the db at the same time as it is being copied. That means that the 'last updated' time will be different for each table. On that basis, is it possible in your application to set up a 'last_updated' column for each row? If so you will be able to tell for each table which logged queries still need to be copied.
What you're looking for is replication. It has far to many options to cover here in a single post.
http://dev.mysql.com/doc/refman/5.5/en/replication.html
If your going to do replication over the internet you'll want to secure it.Your host might allow a virtual local area network So this doesn't use up your bandwidth resources.
A great set of tools from percona you should look at are maatkit
https://launchpad.net/percona-toolkit
Documentation and usage examples
http://www.maatkit.org/doc/
It's good for other tasks but it also allows you to replicate a live database quickly.
When your working with live databases make sure your backups are upto date.