Good day to you.
I used my MySQL server with "innodb_file_per_table" option, and now server is crashed. I want to recover this server using this way:
Uninstall old MySQL
Install new MySQL
Add "innodb_file_per_table" in MySQL configuration
Copy databases folders (only my, not mysql) from old MySQL/data to new MySQL/data
In every folder I have two files, .frm and .ibd, and looks like this files have all data from my databases tables.
But, after copying, tables in this databases didn't work: when I try to open some table, I get error: Table xxx doesn't exist in engine.
I tried REPAIR command, but it isn't helpful.
If you know how to finish my way of repair — please help.
...I know that I need to copy ibdata1 also, but it looks so dead for recovery, so I try way that I try.
REPAIR command won't help with InnoDB.
If you're lucky enough the best you can do is:
1. Start MySQL with innodb_force_recovery=4 (try 5, 6 values if InnoDB fails to start). Make sure innodb_purge_threads=0.
2. Dump the database with mysqldump tool. Yes, it may be slow, but there is no other choice.
3. Create new empty InnoDB table space and reload the dump.
If MySQL fails to start with innodb_force_recovery=6, then recovery from backups is the only option. Well, you can fetch records from *.ibd files, but this is tedious job - google data recovery tool from percona
UPDATE: Data recovery toolkit moved to GitHub
You need to copy everything, not only the data folder
For example, without the ibdata file, mysql don't know where the tables are stored.
https://serverfault.com/questions/487159/what-is-the-ibdata1-file-in-my-var-lib-mysql-directory
Related
I am running Windows 10 with XAMPP and several dozen Drupal sites installed on localhost. Everything has been working fine for months.
This morning I performed a Windows restore from a restore point two days ago to get rid of an unwanted Windows update. After I did that, my MySQL stopped working. I tried deleting the file ibdata1 (I now know that that was a bad idea), but when things got worse, I restored the initial ibdata1 that I'd deleted. All the table data (.frm and .ibd files) is still in C:\xampp\mysql\data. Now MySQL will at least start, but all the tables are "gone"... I can load phpMyAdmin in the browser, and when I drop down the databases at left, all the tables show... but when I try to click on one, it tells me "table not found."
In mysql_error.log there are several errors like InnoDB: Cannot open table mysql/slave_master_info from the internal data dictionary of InnoDB though the .frm file for the table exists. There is a URL mentioned in the error message that supposedly tells you how to resolve this issue, but it is uninformative.
I don't think ibdata1 is corrupted... and all the articles regarding recovering data are complicated to a degree that is insanely out of proportion to the problem. No one has a simple spelled-out solution. Am I Googling wrong? This has happened to me at least twice before, and each time the problem spontaneously resolved itself before I was able to start following the 10-hour-long procedures I found online. Not this time, unfortunately. Surely I'm missing something simple. I had hundreds and hundreds of tables in a few dozen databases, and it seems hopeless to try to find the schema for each one and manually plug it in, which is what most recovery articles seem to suggest.
Shouldn't the database just know how to read the .frm and .ibd files, and, you know... work? It feels like there's just one value out of place somewhere, and if I could just find it, everything would just "pop" back into place.
What did the work for us was this link (Restoring Orphan File-Per-Table ibd Files).
In brief, you do:
ALTER TABLE ..... DISCARD TABLESPACE;
For each affected table.
Then, for each affected table:
ALTER TABLE .... IMPORT TABLESPACE; SHOW WARNINGS;
You could do that on the same schema or create a new schema in other database and copy all *.ibd files. In this case:
Create the new schema with table definitions.
Do the discard tablespace for all tables.
Copy the new *.ibd files to the right folder of new schema.
Do de import tablespace for all tables.
IMPORTANT! It's better that the new schema has the same name as the old one.
What worked for me:
https://dba.stackexchange.com/a/42932
My dbs was InnoDB, so there is a problem with id and when upgrading the macos, I got this. I will quote the main part:
You must have:
-ibdata1
-ib_logfile0
-ib_logfile1
-.FRM files from your mysql_database folder
-Fresh installation of MAMP / MAMP Pro that you are willing to destroy (if need be)
SSH into your web server (dev, production, no difference) and browse to your mysql folder (mine was at /var/lib/mysql for a Plesk
installation on Linux)
Compress the mysql folder
Download an archive of mysql folder which should contain all mySQL databases, whether MyISAM or innoDB (you can scp this file, or move
this to a downloadable directory, if need be)
Install MAMP (Mac, Apache, MySQL, PHP)
Browse to /Applications/MAMP/db/mysql/
Backup /Applications/MAMP/db/mysql to a zip archive (just in case)
Copy in all folders and files included in the archive of the mysql folder from the production server (mt Plesk environment in my case)
EXCEPT DO NOT OVERWRITE:
-/Applications/MAMP/db/mysql/mysql/
-/Applications/MAMP/db/mysql/mysql_upgrade_info
-/Applications/MAMP/db/mysql/performance_schema
In my case, I was restoring the backup (done by galera-arbitrator and xtrabackup. Using the first one we took state-snapshot and using the last one we backed up the physical data) for Percona_XtraDB_Cluster. So, after placing the backed up data to the dir /var/lib/mysql of the new mysql server, we did the following select query and got error.
$ mysql -uroot -p$MYSQL_ROOT_PASSWORD -h<NEW_SERVER_HOST_NAME> -e 'show columns from <MY_DB>.<MY_TABLE>'
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1146 (42S02) at line 1: Table 'playground.equipment' doesn't exist
And the server log was:
[Warning] InnoDB: Cannot open table <MY_DB>/<MY_TABLE> from the internal data dictionary of InnoDB though the .frm file for the table exists. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue.
We just restarted the container the new data. That's it
Just rename slave_master_info.frm to slave_master_info.BAK and that's it. Restart the mysql service.
Yesterday, the hard drive from my dedicated server crashed. The only choice I had was to save what I could, before replacing the hard drive with a new one.
The only thing I could save was the /var/lib/mysql folder. Now I have a new hard drive with a new MySQL database running, but empty. I want to restore the data from the database I had.
I tried several things, and read over the internet that InnoDB export wasn't as easy as copying all the /var/lib/mysql folder, so I'm not sure what solution I have.
I tried to replace the new /var/lib/mysql folder with the new, but now mysqld won't run. The error log looks a lot like this, but adding innodb_force_recovery = 1 is not helping at all.
Now I'm kinda lost and out of options. Anyone faced something similar recently?
Thanks
There is a tool that can retrieve MySQL database from corrupted disk https://launchpad.net/undrop-for-innodb .
I hope you still have old hard drive attached to the server or at least an image taken from it.
Download the last revision and compile it. You'd need gcc, flex and bison as the dependency.
Let's say you have attached the old (corrupted) disk as /dev/sdb
First, you need to scan the disk and find InnoDB pages
./stream_parser -f /dev/sdb
Then you need to recover InnoDB dictionary to know table name -> index_id correspondence. See "Recover InnoDB dictionary" for details.
You would also need tables structure. Take it from some old backup. (Other options would be to recover the structure from .frm files or from InnoDB dictionary)
When you know table structure and index_id of a table you can extract the table records from InnoDB pages:
./c_parser -6f pages-actor.ibd/FIL_PAGE_INDEX/0000000000002655.page -t actor.sql
actor.sql is CREATE TABLE statement
c_parser will generate LOAD DATA INFILE command to load the dump back in MySQL.
I am attempting to restore my old databases to a new Wamp installation. However, I did not take a backup of Mysql files before I did this. Thankfully the uninstallation has not deleted all the files, but in my data folder I have :
1) All the databases (As folders)
2) auto.cnf
There are no ib* files in the old folder. Is it still possible to restore my databases somehow?
I have tried to replace the entire mysql5.6.17 folder (there has been no change of version between the installations), but Wamp remains orange and never turns green.
If I replace only the data folder, the MySQL service starts, I can see the databases, but not the tables.
If I change the path in my.ini to point to the old wamp's data folder, I cannot see the old databases at all.
EDIT : Was experimenting a bit, copying the database folders directly into the new Wamp's data folder allows me to get a list of all the tables properly, but there is no actual data in the tables, and trying to query it using MySQLWorkbench gives me the error :
Error Code: 1146. Table 'testdb.users' doesn't exist
EDIT 2 : Found this in the error log file in my data folder :
2014-08-03 06:24:46 25164 [Warning] InnoDB: Cannot open table testdb/users from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
As I thought, its a problem with InnoDB's own mapping. Is there a way to resolve this manually?
If you created an INNODB database, and made no other config changes to store the actual data in seperate table spaces then all the actual data for that INNODB databases will have been stored in the \wamp\bin\mysql\mysql5.6.17\ibdata1 file.
If you do not have that file from the old install then your data is lost.
There will also be info in a \wamp\bin\mysql\mysql5.6.17\data\DATA_BASE_NAME\ folder, one .frm file for each table that was in the database, but this is basically just schema information and not actual data.
For future reference, InnoDB Backup and Recovery
GOLDEN RULE:
Always backup database (from phpmyadmin) before major changes.. better to spend 1 minute for that, than going into trouble..
How to Copy data folder
If you want to copy the database, then INSTALLATION/mysql versions should be same.. then rename newly installed data folder to i.e. data_my_old,so now there you can place the backuped data folder (it should contain ibdata1 file).
Depends on innodb_file_per_table setting.
If it was OFF then all your tables were in ibdata1. Now you have quite new ibdata1 (since you see errors "Cannot open table testdb/users from the internal data dictionary", that means InnoDB runs fresh new dictionary). That means that now the data is somewhere on the disk. The recovery is quite difficult and requires a lot of tedious work. You need to take an image of the disk, scan it with stream_parser to find InnoDB pages, then recover the InnoDB dictionary, then find index_id of every table and if the stream_parser has found this index run c_parser to extract records from the index.
If innodb_file_per_table was ON and you do see .ibd files in the databases directories then recovery is much easier. First, you may try to reconnect individual ibd files with ibdconnect or extract records from the ibd files with stream_parser/c_parser.
The mentioned tools are part of TwinDB data recovery toolkit
i have just done a google search and found this....
Go to C:\wamp\bin\mysql\mysql5.5.24\data
Find a file called "mysql-bin.index"
Rename that file to "mysql-bin.index-corrupt"
Restart the server and it will work fine, all your database will be where you left them.
I have a script that needs to add data to a folder created in the mysql data folder and then in some way force the mysql server to reload the data so the database reliably shows up. I am currently using a system call to the /etc/init.d/mysql script to restart the server but this is quite slow, I couldn't find anyhting in mysql that would reload the database through a query of some kind.. I was just wondering if there was a way to do this that was slightly quicker?
This depends on the storage engine you're using.
MyISAM tables will be reloaded automatically as soon as the table is next touched by MySQL (obviously, you need to make sure the .MYI, .MYD and .frm files are consistent).
InnoDB is a bit more complex, and depends on whether it's a traditional configuration with all tables in the same tablespace, or if you're using innodb_file_per_table. With the former, you pretty much have to restart the server. With file-per-table, you can copy individual .ibd files, but then you have to use an ALTER TABLE statement to import that tablespace.
See here for information on working with individual .ibd files without restarting the server.
This page is also useful for seeing how to move InnoDB data from place to place.
What do you mean by 'quite slow'?
You could do a dump/load of the new db in a one named orginal_db_name_temp, rename the original DB, and then rename orginal_db_name_temp to orginal_db_name - then you will have your new DB in place.
However, I don't know which method is faster by your means - because a dump&load roundtrip will be probably slower than a file-copy. But renaming the DBs will probably be faster than restarting the server
The short question:
Is it safe skip ib_logfile* files while creating custom-made backup of MySQL installation with InnoDB engine used?
Custom-made backup is just copy all (or subset of) /var/lib/mysql + /etc/my.cnf to safe place.
As I know ib_logfile* files are recreated by MySQL on start + the files are really huge, so that may not be wise to store them in backup. And yes, assume MySQL is (correctly) stopped before backup.
It's not safe to back up a database in the manner you're describing.
The reason is that you cannot know if some data pages are still in MySQL Server's memory, pending an I/O flush to disk. So copying the files has a good chance of copying garbage, whether or not you include the ib_log files.
Don't do what you're doing. Use mysqlbackup or InnoDB's Hot Backup product.
edit:
Read Farhan Mashraqi's blog article about InnoDB Performance Optimization, in which he says you can delete the ib_log files if you have shut down MySQL Server.
InnoDB log resizing: is not as simple
as changing option and restarting. You
need to shut down MySQL server and
make sure it shuts down normally
(check error log for errors to
confirm). Then move away InnoDB log
files ib_log*. Then start the MySQL
server and check error log files to
see if it successfully created new log
files.
So yes, you should be able to do a filesystem copy of the ibdata files only. You don't need to include the ib_log files, because they should be recreated as MySQL Server starts up. I reiterate, however, that I do not recommend backing up databases with filesystem copy.
Suppose you back up a database today in this manner, using MySQL 5.0. Someday later you need to restore the backup, but by that time you'll be using some future version like MySQL 6.5 for example. Do the ib_data files still use the same internal format in that future version? Will MySQL 6.5 read those files you backed up today? You can't be sure. The backup format is intended to be more portable.