I am planning to migrate a large MySQL database from a Windows Machine to a Linux machine. The databases uses the InnoDB storage engine on the tables. Because mysqldump is so slow with large databases, i want to use raw data (files) to perform the task. I understand that when doing it this way, the hardware on both machines and the MySQL versions have to be identical, otherwise its complex to accomplish. Now my question is - Is there perhaps a tool that i can use to help accomplish this...maybe a tool that requires to pay or even free it does not matter? Please advise.
Related
What are the advantages/disadvantages of using database snapshots vs. backing up SQL dumps (from mysqldump)?
I find SQL dump files easy to backup and restore. It seems to be much faster than using database snapshots as well so I am not sure what's the point of using snapshots when they take longer to use.
For additional context, I use MySQL 5.6 and more specifically AWS RDS Aurora (MySQL edition).
Probably the most significant reason people use snapshots is that it can take days to import a large dump file.
Snapshots have several advantages:
It's faster to create a snapshot or restore a snapshot after your database gets very large. You might not yet have a database that is large enough to show this advantage.
You don't need an EC2 instance to run mysqldump on, nor do you need to provision storage for the dump file. The snapshot infrastructure takes care of both of those for you, as you invoke a snapshot in a "serverless" manner via GUI or API.
Snapshot lifecycle management is included in the AWS console. You can auto-expire snapshots, or save them to Glacier, etc. You can do the same with dump files, but you're kind of on your own for managing them.
Dump files have advantages too:
Dumps are in human-readable files. You can even use grep to search for data in a dump file.
Dumps are more-or-less portable between platforms and MySQL versions.
Dumps are a good way to transfer data between RDS and your on-prem data center (although there's a recent announcement that RDS now supports import using Percona XtraBackups).
I am designing a data replication solution across timezones and have run into the issue where I can only run and old version of MySQL (5.6) in one location, whereas the other two have MariaDB 10.2.
Now, I have read the information about Replication Compatibility over at MariaDB. Clearly I can't use MariaDB as a master and MySQL as a slave.
Intermediate solution based on Bash scripts
Yet, I want to use my EU server as a master and that is running MariaDB. So I'm now contemplating a way around the limitation. So far I have come up with an intermediate data storage solution in the overseas server, where data is shuffled periodically using my own Bash data migration scripts over a low bandwidth link.
MariaDB is required in the primary location
I have to use MariaDB in my primary location because I'm using the ColumnStore database there. That is unconditional as part of the application design.
Does this situation ring a bell?
Do you have experience with similar situations and would you mind sharing some inspiration as for how you did it?
My best solution so far is with Bash scripts that are cronned, where MySQL data is dumped (mysqldump) and transferred over a low bandwidth link, then merged with the master (ColumnStore storage engine). I'm looking at a T+1 data lag between my primary location and the secondary location that is running MySQL.
Any high level design thoughts or shared experience is highly appreciated.
Best regards
I have a huge MySQL InnoDB database (about 15 Go, and 100M rows), on a Debian server.
I have to save my database every two hours in another server, but without affect performances.
I looked at the MySQL replication, but it does not correspond to the fact that I look for, because I also want to protect of problems which the application could possibly cause.
What would be the best way of dealing with it?
Thank you very much!
I think you need incremental backups.
You can use Percona XtraBackup to make fast incremental backups. This works only if your database uses only InnoDB tables.
Refer to the documentation about how to create incremental backups:
http://www.percona.com/doc/percona-xtrabackup/howtos/recipes_ibkx_inc.html
Have you looked at writing a script that uses mysqldump to dump the contents of the DB, transfers it over to the backup DB (piping it to SSH would work), and inserts it via the command line?
There are options so that mysqldump won't lock the tables and so won't degrade performance too much.
I have been doing some research on best backup procedures for largish (27.678gb) MYSQL database tables.
Currently we are using a program called Rapidsync (which is a offsite backup tool) but it is slow and it locks the tables it's currently backing up therefore causing downtime/slowness of sql.
Our current server is running Windows 2008 r2 with SQL server 2008 (on the same box) also.
Hardware specs for the dedicated server are:
16gb Ram
CPU intel xeon E3-1230 V2 # 3.30GHz
1 TB hard drive
In terms of databases we have 58 in total varying in size in which some need to backed up weekly ideally or even daily.
Through a program we use called Navicat you can tunnel to a database using SSH and copy databases manually, is this a reliable and feasible option if we were to install it on our local machine and copy them across? Or would it be more secure/efficient to use SQL Dump maybe?
I hope I have given all of the necessary info but please do ask if you need to know more.
P.S Only free options at this moment as we are on a tight budget! :)
Thanks in advance
You mentioned SSH, so I suppose the backups can be done also on other server than the database server itself. For Unix, you can use great tool Percona xtrabackup, which is free and supports online (without locking) backup of InnoDB database files and also incremental backups. Maybe it is possible to compile it also on Windows (part of it is written in C, part in Perl).
So you can setup weekly full backup and daily incremental backups. The tool will keep track of which pages of the InnoDB datafile has been changed and will copy only those.
I am looking for a hot backup system for mysql databases? I like to ask you for experiences and expertise.
So the place you want to look for good solutions to MySQL issues is Mysql Performance Blog/Percona. Specifically, what you want to do is use their XtraBackup, which fixes a number of issues that occur with InnoDBs hot backup. You'd be well served to use their patched server with their InnoDB replacement XtraDB.
They integrate a number of third party patches for MySQL, including their own and Google's, and have enhanced the InnoDB storage engine with fixes, enhancements and better logging. Additionally, Percona sponsors a fantastic toolbox for MySQL called Maatkit, which gives some components to help you with backups and integrity checks.
I know InnoDB hot backup but works just for InnoDB
http://www.innodb.com/products/hot-backup/
I use the MySQL GUI Administrator Tool. It has a backup tool, which you can use to schedule daily backups through windows. You can create different backup projects, schedule the backups, and control how the backup runs and the type of output file it creates.
I've always found it reliable, and the other tools bundled with it are also useful for basic database administration.