I am having to restore a mysql database from .myd and .frm files. And Ii have no idea where to begin, I tried just copying them over while preserving file permisssions but to no avail what other steps do i need to take?
I have a feeling its got something to do with the ib_logfile0, ib_logfile1 and ib_data files. But dont know what to do.
You shouldn't be doing backups by dealing with the raw MySQL files. It's trivially easy to end up with a corrupted database that way. Consider what happens when you start copying the files to your backup medium - the copy will take a non-zero amount of time, during which the database could potentially write new data in various places, including the parts you've already copied. Now you're copying a modified file, a mix of old data and new data. This modified copy is almost guaranteed to be corrupted.
You should be using mysqldump or mysqlhotcopy instead, which guarantee a consistent backup.
However, on the chance that your database is relatively quiet and you managed to get a good clean backup copy, the files you need to restore depend on the type of tables you're restoring. InnoDB stores all of its data in the ib* files, regardless of database/table name. MyISAM uses files in directories named according to the database/table names.
After copying the backup copies into the correct locations, you'll have to restart MySQL, as it will still be accessing the original copies of the files.
Related
We want to run BI analytics on a (copy of a) mysql database.
Currently we get a full mysql dump file daily, but it takes up to 16 hours to import this file into our "BI" server. The dump file itself is about 9GB big, resulting in 360 tables with a total of 20GB (on disk); the biggest table file (.ibd) is around 6GB.
I am no mysql expert, but I think that the import takes so much time since the dump file only imports data, and the database needs to re-index everything from scratch...( as a side question: Any thoughts maybe on how to improve the import?) We have been looking at having a separate SSD for the datafiles, added some more RAM/CPU to the server, but that does not really improve the import speed...
Is there a way to have some sort of "snapshot" of the source database, so that it can be copied over as-is instead of importing?
I think about a .zip file with all the .ibd files (and other necessary files)
Yes, importing a large mysqldump is notoriously slow. The dump file contains the definitions of indexes, but not the index storage itself. So the indexes must be rebuilt every time you import the dump file.
At my company we use Percona XtraBackup.
It's a physical backup tool, meaning the result is not an SQL dump file that must be imported. It just makes a copy of the .ibd files and the iblog files to reconcile transactions. The .ibd files contain both rows of data and indexes.
We use this backup & restore solution to clone databases up to 100x as large as yours.
Percona XtraBackup is free and open-source.
There are a few caveats:
Doesn't work on Windows last I checked (I haven't checked in several years, because I don't use Windows).
You can backup without interrupting service on the source instance, but to "restore" you need to shut down your local MySQL Server, copy the backup into the datadir, and restart the MySQL Server.
It's nearly impossible to import just one database to an instance. In other words, backup and restore is for the full instance, with all tables and schemas. This will overwrite anything else you have on your local instance. Whereas mysqldump is more flexible because you can dump & import just one table or just one schema, and you can import to a running MySQL Server instance without stopping it.
It's worth mentioning that if you don't use a proper backup tool, you should not try to make a zip archive of the MySQL data directory of a running MySQL Server. You're almost certain to get a non-consistent copy of the data files, meaning they will be corrupt and not restorable.
I have a mysql database, and today I woke up and noticed that some of my tables are empty (like if they were truncated) but some aren't. This is not an SQL injection since I took care of that, but still haven't figured out what really happened.
Now I need to recover that data that was deleted, but the only thing I have are the lib files, so, is there a way to recover a database data using those files? We never deleted any file.
Basically I have ib_logfile, ibdata files but I never enabled Binary Logging.
Yesterday, the hard drive from my dedicated server crashed. The only choice I had was to save what I could, before replacing the hard drive with a new one.
The only thing I could save was the /var/lib/mysql folder. Now I have a new hard drive with a new MySQL database running, but empty. I want to restore the data from the database I had.
I tried several things, and read over the internet that InnoDB export wasn't as easy as copying all the /var/lib/mysql folder, so I'm not sure what solution I have.
I tried to replace the new /var/lib/mysql folder with the new, but now mysqld won't run. The error log looks a lot like this, but adding innodb_force_recovery = 1 is not helping at all.
Now I'm kinda lost and out of options. Anyone faced something similar recently?
Thanks
There is a tool that can retrieve MySQL database from corrupted disk https://launchpad.net/undrop-for-innodb .
I hope you still have old hard drive attached to the server or at least an image taken from it.
Download the last revision and compile it. You'd need gcc, flex and bison as the dependency.
Let's say you have attached the old (corrupted) disk as /dev/sdb
First, you need to scan the disk and find InnoDB pages
./stream_parser -f /dev/sdb
Then you need to recover InnoDB dictionary to know table name -> index_id correspondence. See "Recover InnoDB dictionary" for details.
You would also need tables structure. Take it from some old backup. (Other options would be to recover the structure from .frm files or from InnoDB dictionary)
When you know table structure and index_id of a table you can extract the table records from InnoDB pages:
./c_parser -6f pages-actor.ibd/FIL_PAGE_INDEX/0000000000002655.page -t actor.sql
actor.sql is CREATE TABLE statement
c_parser will generate LOAD DATA INFILE command to load the dump back in MySQL.
I am attempting to restore my old databases to a new Wamp installation. However, I did not take a backup of Mysql files before I did this. Thankfully the uninstallation has not deleted all the files, but in my data folder I have :
1) All the databases (As folders)
2) auto.cnf
There are no ib* files in the old folder. Is it still possible to restore my databases somehow?
I have tried to replace the entire mysql5.6.17 folder (there has been no change of version between the installations), but Wamp remains orange and never turns green.
If I replace only the data folder, the MySQL service starts, I can see the databases, but not the tables.
If I change the path in my.ini to point to the old wamp's data folder, I cannot see the old databases at all.
EDIT : Was experimenting a bit, copying the database folders directly into the new Wamp's data folder allows me to get a list of all the tables properly, but there is no actual data in the tables, and trying to query it using MySQLWorkbench gives me the error :
Error Code: 1146. Table 'testdb.users' doesn't exist
EDIT 2 : Found this in the error log file in my data folder :
2014-08-03 06:24:46 25164 [Warning] InnoDB: Cannot open table testdb/users from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
As I thought, its a problem with InnoDB's own mapping. Is there a way to resolve this manually?
If you created an INNODB database, and made no other config changes to store the actual data in seperate table spaces then all the actual data for that INNODB databases will have been stored in the \wamp\bin\mysql\mysql5.6.17\ibdata1 file.
If you do not have that file from the old install then your data is lost.
There will also be info in a \wamp\bin\mysql\mysql5.6.17\data\DATA_BASE_NAME\ folder, one .frm file for each table that was in the database, but this is basically just schema information and not actual data.
For future reference, InnoDB Backup and Recovery
GOLDEN RULE:
Always backup database (from phpmyadmin) before major changes.. better to spend 1 minute for that, than going into trouble..
How to Copy data folder
If you want to copy the database, then INSTALLATION/mysql versions should be same.. then rename newly installed data folder to i.e. data_my_old,so now there you can place the backuped data folder (it should contain ibdata1 file).
Depends on innodb_file_per_table setting.
If it was OFF then all your tables were in ibdata1. Now you have quite new ibdata1 (since you see errors "Cannot open table testdb/users from the internal data dictionary", that means InnoDB runs fresh new dictionary). That means that now the data is somewhere on the disk. The recovery is quite difficult and requires a lot of tedious work. You need to take an image of the disk, scan it with stream_parser to find InnoDB pages, then recover the InnoDB dictionary, then find index_id of every table and if the stream_parser has found this index run c_parser to extract records from the index.
If innodb_file_per_table was ON and you do see .ibd files in the databases directories then recovery is much easier. First, you may try to reconnect individual ibd files with ibdconnect or extract records from the ibd files with stream_parser/c_parser.
The mentioned tools are part of TwinDB data recovery toolkit
i have just done a google search and found this....
Go to C:\wamp\bin\mysql\mysql5.5.24\data
Find a file called "mysql-bin.index"
Rename that file to "mysql-bin.index-corrupt"
Restart the server and it will work fine, all your database will be where you left them.
By some reason these files ibdata1, ib_logfile0, ib_logfile1 were been deleted from hard drive, mysql server was restarted and of course one of the database stoped with error "db.table does not exist!". Is it possible to restore somehow these files, if there are only *.frm, db.opt files?
Thanks.
Take disk image as soon as possible or mount MySQL partition read-only(if it was not on /).
Then you need to compile data recovery toolkit(install dependencies if "make" fails):
make
Then scan the disk image with page_parser. It will find InnoDB pages and sort them per page type, per index_id.
./page_parser -f /path/to/disk.img
Then you need table structures (either take them from old backup or restore from .frm). You need the structure to compile constraints_parser - a tool that fetches records from InnoDB page(s).
./constraints_parser -5f pages-XXXXXX/FIL_PAGE_INDEX/0-x/
Save output in some file. It will also generate LOAD DATA to stderr to upload the dump back to MySQL.
Check the documentation on Percona's website for more details.
From my experience, 70-80%% of data should be recoverable after this kind of accident.