Let's say your taking daily snapshots of your server as a whole (public_html, sql files, etc) for later restoration in case of system failure.
Is it possible that you can restore damaged a MySQL InnoDB if you took the snapshot while an uncommitted transaction was taking place? Or will InnoDB do just fine and discard "incomplete" transactions on restoration?
From the databases viewpoint we are dealing with an unclean shutdown (i.e. power went off) and a lost connection so it will discard all transactions that are not committed.
If you are taking a snapshot of the server that's just like freezing everything in a cryogenic sleep, then after a restore the database would just awake expecting to talk to a non existing application.
The only issue that i can see is not from a transaction itself but from the fact that the database itself resides inside files. What if you freeze a file half-written to disk. I can see how that might be a problem. On the other hand there's probably some architectural design in place to prevent this as the same is true for a power outage and a database should live through that too.
To best of my knowledge, during a transaction, nothing gets saved into the database until commit has occurred. That's the ACID compliance right there. So no database files get written during a transaction, only after.
Also, in my opinion, database 'snapshots' should be done via dump. I'm not a server administrator so I wouldn't know for a fact, but it'd be a lot safer to restore the data that way.
Although, MySQL is a gray area to me, I'm more confident with SQL Server, so don't take this as a fact, as SQL Server has its own ways of getting backups made.
Related
After a recent DB crash with no RCA from our vendor. I'm left want to expand my knowledge on what sort of things can cause a database to crash.
In our specific case the logs show record index mis-matches just before the crash. We believe this was due to copying .frm and .idb files to another db rather than using mysqldump. There are logged warnings about that for about a week prior to the crash, starting when they were copied. But would it really take so long for the DB to crash?
In my experience, most crashes are due to hardware errors, i.e. your disk is failing. Second most common is user error, like moving InnoDB tablespaces around like they're normal files (you already know, don't do that). Third is bugs in MySQL, because all software has bugs.
It's certainly possible to take an indefinite amount of time before user activity accesses the code path that causes a crash. Can't make any conclusion from that.
Ultimately, you have to create redundancy to protect against crashes. This is especially important for databases. Examples of redundancy:
Use RAID 1 or RAID 10 to do disk mirroring
Use replication to copy data another MySQL instance continuously
Host the other MySQL instance on another physical computer, best if it's on a separate rack in your data center
You might even have another replica in another region of the country
We have a SQL Server 2008 R2 database that backs up transaction logs every now and then. Today there was a big error in the database caused at around 12am... I have transaction logs up to 8am and then 12am - 16pm - etc.
My question is: can I sort of reverse-merge those transaction logs into database, so that I return to the database state at 8am?
Or is my best chance to recover an older full backup and restore all transaction logs up to 8am?
The first option is preferable since full backup has been performed a bit of a while ago and I am afraid to f*ck things up restoring from there and applying trn logs. Am I falsely alarmed about that? Is it actually possible for anything bad to happen if going by that scenario (restoring the full backup and applying trn logs)?
The fact that you don’t create regular transaction log backups doesn’t affect the success of the recovery process. As long as your database is in the Full recovery model, the transactions are stored in the online transaction log and kept in it until a transaction log backup is made. If you make a transaction log backup later than usual, it only means that the online transaction log may grow and that the backup might be bigger. It will not cause any transaction history to be lost.
With a complete chain of transaction log backups back to 8 AM, you can successfully roll back the whole database to a point in time.
As for restoring the full backup and applying trn logs – nothing should go wrong, but it’s always recommended to test the scenario on a test server first, and not directly in production
To restore to a point in time:
In SSMS expand Databases
Right-click the database, select Tasks | Restore| Database
In the General tab, in the Backup sets the available backups will be listed. Click Timeline
Select Specific date and time, change the Time interval to show a wider time range, and move the slider to the time you want to roll back to
You can find more detailed instructions here: How to: Restore to a Point in Time (SQL Server Management Studio)
Keep in mind that this process will roll back all changes made to the database. If you want to roll back only specific changes (e.g. only recover some deleted data, or reverse wrong updates), I suggest a third party tool, such as ApexSQL Log
Reverting your SQL Server database back to a specific point in time
Restore a database to a point in time
Disclaimer: I work for ApexSQL as a Support Engineer
Without not reason, I lose all my data in my database. Fortunately this was just test data, but this made me to think what will happen if this was done with a production db.
Ultimately, every developer got a db problem and want to rollback the db. We don't do things to protect the db, as we think its a DBA work, but then we got into trouble...
What are your backup best practices?
Since all the developers are also the DBAs where I work, we're collectively responsible for our backup strategy as well - if you care about the data, make sure you're at least informed about how the backups work, even if you're not part of the actual decisions.
The VERY first thing I do (before I even have any databases set up) is set up nightly maintenance plans that include a full backup, and direct those backups to a central network share on a different computer (our NAS). At the very least, for the love of your job, don't put the backups on the same physical storage that your database files sit on. What good are backups if you lose them at the same time you lose the disk?
We don't do point-in-time restores, so we don't do log backups (all our databases are set to Simple recovery mode), but if you want logs backed up, make sure you include those as well, as an acceptable interval as well.
On a side note, SQL 2008 supports compressed backups, which speeds up backup time considerably and makes the files much, much smaller - I can't think of an instance where you wouldn't want to use this option. I'd love to hear one, though, and I'm willing to reconsider!
Here are some points from my own experience:
Store your backups and SQL Server database files on the different physical storages. Otherwise, if your physical storage failed you will lose both backups and database files.
Create your own SQL Server database backup schedule.
Test your SQL Server backup. If you never tested your backups I have a doubt that you will be able to restore your database if the failure occurs. Time to time you need to have the practice to restore your backup on a test server.
Test your recovery strategy - here is another tip. If the failure occurs how much time do you need to restore your database to the working state?
Backup SQL Server's system databases.
And this isn't a whole list, you can find more tips in my article https://sqlbak.com/blog/backup-and-recovery-best-practices/
Choosing the right backup strategy is one of the most important factors a DBA should consider at the point of developing the DB.
However the backup strategy you choose depends on a number of factors:
How frequent is transaction carried out on the DB: are there thousands of transactions going on every minute or maybe few transactions per day? For very busy database, i would say, take full nightly backups and transaction log backups every 10 mins or even less.
How critical the data content is: may it be employee payroll data? Then you'll have no acceptable excuse or you may have a few angry faces around your car when you want to drive home! For very critical database, take nightly backups possibly in two locations and transaction log backups every 5 mins. (Also think of implementing mirroring).
Location of your backup location: if your backup location is close to the DB location, then you can afford to take more frequent backup than when it is located several hops away with not excellent bandwidth between.
But in all, i would say, schedule a nighly backup every night, and then transaction log backups at intervals.
I'm setting up a SQL Server 2008 server on a production server, which way is the best to backup this data? Should I use replication and then backup that server? Should I just use a simple command-line script and export the data? Which replication method should i use?
The server is going to be pretty loaded so I need an efficent method.
I have access to multiple computers that I can use.
A very simple yet good solution is to run a full backup using sqlcmd (formerly osql) locally, then copy the BAK file over the network to a NAS or other store. It's sub-optimal in terms of network/disk usage, but it's very safe because every backup is independent and given that the process is very simple it is also very robust.
Moreover, this even works in Express editions.
The "best" backup solutions depends upon your recovery criteria.
If you need immediate access to the data in the event of a failure, a three server database mirroring scenario (live, mirror and witness) would seem to fit - although your application may need to be adapted to make use of automatic failover. "Log shipping" may produce similar results (although without automatic failover, or need for a witness).
If, however, there's some wiggle room in the recovery time, regular scheduled backups of the database (e.g., via SQL Agent) and it's transaction logs will allow you to do point-in-time restores. The frequency of backups would be determined by database size, how frequently the data is updated, and how far you are willing to rollback the database in the event of complete failure (unless you can extract a transaction log backup out of a failed server, you can only recover to the latest backup)
If you're looking to simply rollback to known-good states after, say, user error, you can make use of database snapshots as a lightweight "backup" scenario - but these are useless in the event of server failure. They're near instantaneous to create, and only take up room when the data changed - but incur a slight performance overhead.
Of course, these aren't the only backup solutions, nor are they mutually exclusive - just the ones that came to mind.
Currently I have two Linux servers running MySQL, one sitting on a rack right next to me under a 10 Mbit/s upload pipe (main server) and another some couple of miles away on a 3 Mbit/s upload pipe (mirror).
I want to be able to replicate data on both servers continuously, but have run into several roadblocks. One of them being, under MySQL master/slave configurations, every now and then, some statements drop (!), meaning; some people logging on to the mirror URL don't see data that I know is on the main server and vice versa. Let's say this happens on a meaningful block of data once every month, so I can live with it and assume it's a "lost packet" issue (i.e., god knows, but we'll compensate).
The other most important (and annoying) recurring issue is that, when for some reason we do a major upload or update (or reboot) on one end and have to sever the link, then LOAD DATA FROM MASTER doesn't work and I have to manually dump on one end and upload on the other, quite a task nowadays moving some .5 TB worth of data.
Is there software for this? I know MySQL (the "corporation") offers this as a VERY expensive service (full database replication). What do people out there do? The way it's structured, we run an automatic failover where if one server is not up, then the main URL just resolves to the other server.
We at Percona offer free tools to detect discrepancies between master and server, and to get them back in sync by re-applying minimal changes.
pt-table-checksum
pt-table-sync
GoldenGate is a very good solution, but probably as expensive as the MySQL replicator.
It basically tails the journal, and applies changes based on what's committed. They support bi-directional replication (a hard task), and replication between heterogenous systems.
Since they work by processing the journal file, they can do large-scale distributed replication without affecting performance on the source machine(s).
I have never seen dropped statements but there is a bug where network problems could cause relay log corruption. Make sure you dont run mysql without this fix.
Documented in the 5.0.56, 5.1.24, and 6.0.5 changelogs as follows:
Network timeouts between the master and the slave could result
in corruption of the relay log.
http://bugs.mysql.com/bug.php?id=26489