Every night the MySQL database is saved by a cronjob which uses mysqldump.
During the day, when the CakePHP application is running, I would like to have a logfile working, that could be used as backup in case of a damage happening during the day.
For recovery, it would be necessary to first run the recovery from the mysqldump that was established at night. And secondly, run the recovery from the logfile to get the database changes from the current day.
Does there exist such a logfile possibility and where or how could I get it?
Or are there any other ways to get a proper backup?
It's built into MySQL, you wouldn't do this in your applictation typically.
enter link description here
The binlogs do what you say - it is a binary file that contains every transaction that hits the database. So, if you had a nightly back up and the server crashed, you would recover the bin logs, export the transaction logs from the datetime of the last transaction in the database and essentially re-run or re-play every transaction from that point.
And having used it >10 times before ... it is awesome... but you have to turn in on in your my.conf file (google that) and manage the binlog files as they do get big on busy servers.
Related
I'm trying to create a DataBase for a very little program that just I and 2 more friends use, so the database will be little.
What I want to do is to share this database with these 2 friends, so I thought about storing the data in a OneDrive shared folder.
At the moment, I'm using a txt file as a "database". Which is placed in a shared OneDrive folder, so when my friends execute the program, it can read the data from there and make it sort of real time "online".
The thing is that I can't find the my.ini file in my
C:\Program Files\MySQL\MySQL Server 8.0\
directory, so I can't change the data folder.
Another trouble I have is that I also don't have the Data folder in that directory, instead I've found it in this one:
C:\Program Files\MySQL\MySQL Workbench 8.0 CE
Is it possible to do what I want to do? And how should I proceed?
Do you think I should be using MySQL 5.x version?
Thanks
If you want to share a database with a couple of other users, putting the datadir in OneDrive isn't going to work.
MySQL stores data in the files under the datadir, that's true. They're stored in files called tablespaces which have the .ibd extension.
But as you do INSERT/UPDATE/DELETE operations, data is temporarily stored in memory and in the transaction logs (ib_logfile*. MySQL has to do a delicate coordination between these, to save data in a durable way, while ensuring good performance. It works well, but only if the MySQL Server is the only process writing to the files.
OneDrive is not coordinating with MySQL at all. It will periodically check for files that have changed since it last did a sync. The interval OneDrive checks files is about every 10 minutes, and this is not configurable.
OneDrive may choose to sync the files at a moment after you've done some INSERT/UPDATE/DELETE operations, and the data is modified in RAM, but hasn't yet been updated in the tablespace files on disk.
Once you've committed a change, it must also be safely stored in the transaction log, even if it isn't updated in the tablespace. But if your friends receive the files in this state (transaction log contains changes which aren't present in the tablespace), they can reconstruct the data from your RAM that they didn't get. This is called InnoDB crash recovery. MySQL Server does if automatically if it starts up and finds the transaction log contains changes that aren't in the tablespace. It assumes you had a sudden reboot and lost what was in RAM.
If your friends try to just keep their MySQL Server running continuously, reading a datadir that is simultaneously being updated by their one OneDrive, it'll basically overwrite their files and MySQL will become confused. It only checks to see if it should do crash recovery as MySQL Server starts. So if the files change in unexpected ways while MySQL Server is already running, it'll just conclude that your hard drive has gotten corrupted. It'll probably report a fatal error and shut down MySQL.
Also if your friends try to make changes of their own to the database, their changes would conflict with the updates from OneDrive. Then their attempts to overwrite files would eventually be synchronized in the opposite direction via OneDrive, and would eventually corrupt your database too. This would happen without warning at 10 minute intervals, whenever OneDrive chose to do its file sync.
So I'm afraid OneDrive is not the solution to share your database.
Alternatives that do have a chance of working include:
Hosting a single instance of MySQL Server on a website that you all share, and giving each of you a client that can use the database. A popular free client is phpMyAdmin. That way there'd be only one instance of MySQL Server, one datadir, even though you each would be concurrent clients reading and writing data. This is the simplest solution, most likely to work.
Exporting the data from your MySQL Server instance using mysqldump periodically, and putting the export file on OneDrive, or sending it to your friends through email or any other means. Then they would have to import that data to their MySQL Server manually. This would overwrite any changes they had done to their database, but it wouldn't appear as corruption. If they want to send changes back to you, they could do a similar operation: export their database, put the dump file on OneDrive, then you'd get their dump file and import it to your MySQL Server instance, overwriting any changes you had done locally since the last time you sent your export to your friends.
Use a MySQL add-on for multi-server synchronous replication, such as InnoDB Group Replication or Percona XtraDB Cluster. But this is probably too complex for you to set up if you're a newbie with MySQL.
I accidentally deleted records in MSQL database and there is no backup no rollback transaction and i reallly want to recover deleted data back is there any tool to help ? how could i do ? please help me .
Is it MySQL or MSSQL they both are not same. MSSQL stands for SQL Server but by any means for you to recover deleted data either you should have taken a recent full/differential backup in place (or) if you have maintained transaction log. Then yes possible to get back your data by restoring from your backup (or) by reading from transaction log. Else, No there is no way to recover those data.
For InnoDB tables, if binary logs are enabled, one could use:
mysqlbinlog path_to_binary_log_file > query_log.sql
If binary logs are not enabled, there is a recovery tool provided by Percona which may help.
Created a script to automate the steps from the Percona tool which recovers the deleted rows (if exist) and provides SQL queries to load the data back into database.
Please note:
The time between the deletion of rows and the database stop is
crucial. If pages are reused you can’t recover the data.
We've got a database (mysql) driven application which contains business critical information, were looking at building a system that will allow us to backup the db frequently (say every 15 mins) essentially so that we minigate the danger of any data loss. Where torn between two setups :
Adding a backup jobs too a queue every 15 mins on a cron and storing these backups on another server. (To save space we would then delete most of these backups after 3 days, but keep the 06:00, 12:00, 18;00 hour versions.)
or
Is there a RAID like setup were all our data will be automatically copied to another hard drive or in this case server, in which case what would happen if we lost data, would the loss be carried to the other server (we would also run standard daily backups for our archives in edition to this) ?
or
Is there another established method for creating frequent backups ?
In my opinion, the optimal backup scheme would be following.
Delayed slave. It allows you quickly restore your database in case of master failure. It may help in case of DROP DATABASE or other wrong SQL. So, you need something additionally.
Incremental backups every day with Xtrabackup from the delayed slave. Optionally you could also check TwinDB for incremental backups.
As long as you need 15 minutes granularity you may pull binary logs from the master with mysqlbinlog from MySQL 5.6 (even if the master is 5.5 or 5.1). So, mysqlbinlog runs on a remote host and pulls logs from the master.
If you need to restore the database you have two ways.
If you can restore from the delayed slave you use that slave as a new master.
If on some reason you can't use the delayed slave (you missed the DROP command) then you restore last night copy from the incremental backup and apply binary logs since the last backup up to the moment of accident (again, if the accident is wrong DROP table you replay logs up to the last event before the DROP).
This schema will be optimal from performance standpoint (no impact on an application) and allows no data loss at all.
If you're doing backups more often than one hour, what you need is replication. Setting up a secondary database server that can serve as a hot-standby is a lot better than abusing your database with repeated reads.
If you're backing up your database frequently, look at innobackupex to snapshot your tables, or possibly LVM snapshots.
I have several slave dbs replicated from the same master db, however, for one of the slaves, i would like to keep it as a backup db, which will never have rows updated or deleted.
basically the purpose is to have a backup db with all rows stored by using the replication(mysqldump is waaay slow to do the backup), no update/delete query get replicated, insert query only. i know there will be some conflicts going on no doubt, but still wonder if any filtering options on statement/query on the slave end or any other solutions.
You should never run a production database without a working backup scheme in place - at least as long as you value your data. If you fear that a wrong sql instruction can ruin your database, then you may try point in time recovery.
If you already use replication your master server will log all write/update operations to its binlog - which it will send to the slave servers for replication. You can do for example nightly backups of you complete database. If you destroy your database in the morning, you can import the backup from the night and reapply the instructions from the binlog from after the backup till before the instruction that killed your database.
You could then skip this instruction and apply the instructions that came afterwards. This can also cause consistency issues, as the instruction after the skipped instruction see different data in the database as they did when they were originally executed.
I have similar problem. I know it's old thread but it can help others:
link: mysql replication works only if I choose database by USE database
We have a new filestream database that will be initially loaded with 65GB data, for which we'd like to configure log shipping to a remote (different continent) location.
For the initial setup of log shipping, is there any threshold for the time between the backup of the primary and it's restore onto the secondary? The new database will essentially be offline until we have the log shipping configured. Due to the size of the database, it may be some time (days) between the database initially being backed-up and then restored on the target. Will this be a problem?
I believe the only concern is that the Log Sequence Number (LSN) chain is unbroken. This will start with your full backup, but can also have as many subsequent transaction log backups as you need. You mentioned the database will be offline until the log shipping is configured, so you shouldn't have an issue with transactions building up on the primary until the backup finishes copy/restore. However, if you wanted to bring the primary online, you might have to take frequent transaction log backups to avoid running out of space as the transactions build up (depending on usage).
This is okay, because you could easily copy those transaction logs over to the secondary, restore them, then enable log shipping. As long as all the backups taken on the primary have been restored on the secondary, the LSN chain is maintained, so the first log backup that is shipped over should restore correctly. Time doesn't matter in this case.