If innodb database is same, and I need to restore table from previous backup, is it ok to copy paste ibd.frm file or do I need to restore separately and then import it?
NO. If all you have is files, not a mysqldump, then it is "all or none" for InnoDB.
You could create another MySQL instance (on the same machine or different), mysqldump the one table, then reload it.
Related
I am running Windows 10 with XAMPP and several dozen Drupal sites installed on localhost. Everything has been working fine for months.
This morning I performed a Windows restore from a restore point two days ago to get rid of an unwanted Windows update. After I did that, my MySQL stopped working. I tried deleting the file ibdata1 (I now know that that was a bad idea), but when things got worse, I restored the initial ibdata1 that I'd deleted. All the table data (.frm and .ibd files) is still in C:\xampp\mysql\data. Now MySQL will at least start, but all the tables are "gone"... I can load phpMyAdmin in the browser, and when I drop down the databases at left, all the tables show... but when I try to click on one, it tells me "table not found."
In mysql_error.log there are several errors like InnoDB: Cannot open table mysql/slave_master_info from the internal data dictionary of InnoDB though the .frm file for the table exists. There is a URL mentioned in the error message that supposedly tells you how to resolve this issue, but it is uninformative.
I don't think ibdata1 is corrupted... and all the articles regarding recovering data are complicated to a degree that is insanely out of proportion to the problem. No one has a simple spelled-out solution. Am I Googling wrong? This has happened to me at least twice before, and each time the problem spontaneously resolved itself before I was able to start following the 10-hour-long procedures I found online. Not this time, unfortunately. Surely I'm missing something simple. I had hundreds and hundreds of tables in a few dozen databases, and it seems hopeless to try to find the schema for each one and manually plug it in, which is what most recovery articles seem to suggest.
Shouldn't the database just know how to read the .frm and .ibd files, and, you know... work? It feels like there's just one value out of place somewhere, and if I could just find it, everything would just "pop" back into place.
What did the work for us was this link (Restoring Orphan File-Per-Table ibd Files).
In brief, you do:
ALTER TABLE ..... DISCARD TABLESPACE;
For each affected table.
Then, for each affected table:
ALTER TABLE .... IMPORT TABLESPACE; SHOW WARNINGS;
You could do that on the same schema or create a new schema in other database and copy all *.ibd files. In this case:
Create the new schema with table definitions.
Do the discard tablespace for all tables.
Copy the new *.ibd files to the right folder of new schema.
Do de import tablespace for all tables.
IMPORTANT! It's better that the new schema has the same name as the old one.
What worked for me:
https://dba.stackexchange.com/a/42932
My dbs was InnoDB, so there is a problem with id and when upgrading the macos, I got this. I will quote the main part:
You must have:
-ibdata1
-ib_logfile0
-ib_logfile1
-.FRM files from your mysql_database folder
-Fresh installation of MAMP / MAMP Pro that you are willing to destroy (if need be)
SSH into your web server (dev, production, no difference) and browse to your mysql folder (mine was at /var/lib/mysql for a Plesk
installation on Linux)
Compress the mysql folder
Download an archive of mysql folder which should contain all mySQL databases, whether MyISAM or innoDB (you can scp this file, or move
this to a downloadable directory, if need be)
Install MAMP (Mac, Apache, MySQL, PHP)
Browse to /Applications/MAMP/db/mysql/
Backup /Applications/MAMP/db/mysql to a zip archive (just in case)
Copy in all folders and files included in the archive of the mysql folder from the production server (mt Plesk environment in my case)
EXCEPT DO NOT OVERWRITE:
-/Applications/MAMP/db/mysql/mysql/
-/Applications/MAMP/db/mysql/mysql_upgrade_info
-/Applications/MAMP/db/mysql/performance_schema
In my case, I was restoring the backup (done by galera-arbitrator and xtrabackup. Using the first one we took state-snapshot and using the last one we backed up the physical data) for Percona_XtraDB_Cluster. So, after placing the backed up data to the dir /var/lib/mysql of the new mysql server, we did the following select query and got error.
$ mysql -uroot -p$MYSQL_ROOT_PASSWORD -h<NEW_SERVER_HOST_NAME> -e 'show columns from <MY_DB>.<MY_TABLE>'
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1146 (42S02) at line 1: Table 'playground.equipment' doesn't exist
And the server log was:
[Warning] InnoDB: Cannot open table <MY_DB>/<MY_TABLE> from the internal data dictionary of InnoDB though the .frm file for the table exists. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue.
We just restarted the container the new data. That's it
Just rename slave_master_info.frm to slave_master_info.BAK and that's it. Restart the mysql service.
I have DB InnoDb innodb_db_1. I have turned on innodb_file_per_table.
If I go to var/lib/mysql/innodb_db_1/ I will find files table_name.ibd, table_name.frm, db.opt.
Now, I'm trying to copy these files to another DB for example to innodb_db_2(var/lib/mysql/innodb_db_2/) but nothing happened.
But if my DB will be MyIsam, I can copy in such way and everything be ok.
What suggestions to move the DB by copying the file of InnoDb DB?
Even when you use file-per-table, the tables keep some of their data and metadata in /var/lib/mysql/ibdata1. So you can't just move .ibd files to a new MySQL instance.
You'll have to backup and restore your database. You can use:
mysqldump, included with MySQL, reliable but slow.
mydumper, a community contributed substitute for mysqldump, this supports compression and parallel execution and other neat features.
Percona XtraBackup, which is free and performs high-speed physical backups of InnoDB (and also supports other storage engines). This is recommended to minimize interruption to your live operations, and also if your database is large.
Re your comment:
No, you cannot just copy .ibd files. You cannot turn off the requirement for ibdata1. This file includes, among other things, a data dictionary which you can think of like a table of contents for a book. It tells InnoDB what tables you have, and which physical file they reside in.
If you just move a .ibd file into another MySQL instance, this does not add it to that instance's data dictionary. So InnoDB has no idea to look in the new file, or which logical table it goes with.
If you want a workaround, you could ALTER TABLE mytable ENGINE=MyISAM, move that file and its .frm to another instance, and then ALTER TABLE mytable ENGINE=InnoDB to change it back. Remember to FLUSH TABLES WITH READ LOCK before you move MyISAM files.
But these steps are not for beginners. It would be a lot safer for you to use the backup & restore method unless you know what you're doing. I'm trying to save you some grief.
There is an easy procedure to move the whole Mysql InnoDB from pc A to pc B.
The conditions to perform the procedure are:
You need to have innodb_file_per_table option set
You need to be able to make a shutdown of the database
In my case i had to move whole 150Gb MySql database (the biggest table had aprox. 60Gb). Making sqldumps and loading them back was not an option (too slow).
So what I did was I made a "cold backup" of the mysql database (mysql doc) and then simply move files to another computer.
The steps to make after moving the databse are described here dba stackexchange.
I am writing this, because (assuming you are able to follow mentioned conditions) this is by far the fastest (and probalby the easiest) method to move a (large) MySql InnoDb and nobody mentioned it yet.
You can copy MyISAM tables all day long (safely, as long as they are flushed and locked or the server is stopped) but you can't do this with InnoDB, because the two storage engines handle tables and tablespaces very differently.
MyISAM automatically discovers tables by iterating the files in the directory named for the database.
InnoDB has an internal data dictionary stored in the system tablespace (ibdata1). Not only do the tables have to be consistent, there are identifiers in the .ibd files that must match what the data dictionary has stored internally.
Prior to MySQL 5.6, with the introduction of transportable tablespaces, this wasn't a supported operation. If you are using MySQL 5.6, the link provides you with information on how this works.
The alternatives:
use mysqldump [options] database_name > dumpfile.sql without the --databases option, which will dump the tables in the specified database but will omit any DATABASE commands (DROP DATABASE, CREATE DATABASE and USE), some or all of which, based on the combination of options specified, are normally added to the dump file. You can then import this with mysql [options] < dumpfile.sql.
CREATE TABLE db2.t1 LIKE db1.t1; INSERT INTO db2.t1 SELECT * FROM db1.t1; (for each table; you'll have to add any foreign key constraints back in)
ALTER TABLE on each table, changing it to MyISAM, then flushing and locking the tables with FLUSH TABLES WITH READ LOCK;, copying them over, then altering everything back to InnoDB. Not the greatest idea, since you'll lose all foreign key declarations and have to add them back on the original tables, but it is an alternative.
As far as I know, "hot copying" table files is a very bad idea (I've done it two times, and only made it work with MyISAM tables, and i did it only because I had no other choice).
My personal recomendation is: Use mysqldump. On your shell:
mysqldump -h yourHost -u yourUser -pYourPassword yourDatabase yourTable > dumpFile.sql
To copy the data from a dump file to another database, on your shell:
mysql -h yourHost -u yourUser -pYourPassword yourNewDatabase < dumpFile.sql
Check: mysqldump — A Database Backup Program.
If you insist on copying InnoDB files by hand, please read this: Backing Up and Recovering an InnoDB Database
Good day to you.
I used my MySQL server with "innodb_file_per_table" option, and now server is crashed. I want to recover this server using this way:
Uninstall old MySQL
Install new MySQL
Add "innodb_file_per_table" in MySQL configuration
Copy databases folders (only my, not mysql) from old MySQL/data to new MySQL/data
In every folder I have two files, .frm and .ibd, and looks like this files have all data from my databases tables.
But, after copying, tables in this databases didn't work: when I try to open some table, I get error: Table xxx doesn't exist in engine.
I tried REPAIR command, but it isn't helpful.
If you know how to finish my way of repair — please help.
...I know that I need to copy ibdata1 also, but it looks so dead for recovery, so I try way that I try.
REPAIR command won't help with InnoDB.
If you're lucky enough the best you can do is:
1. Start MySQL with innodb_force_recovery=4 (try 5, 6 values if InnoDB fails to start). Make sure innodb_purge_threads=0.
2. Dump the database with mysqldump tool. Yes, it may be slow, but there is no other choice.
3. Create new empty InnoDB table space and reload the dump.
If MySQL fails to start with innodb_force_recovery=6, then recovery from backups is the only option. Well, you can fetch records from *.ibd files, but this is tedious job - google data recovery tool from percona
UPDATE: Data recovery toolkit moved to GitHub
You need to copy everything, not only the data folder
For example, without the ibdata file, mysql don't know where the tables are stored.
https://serverfault.com/questions/487159/what-is-the-ibdata1-file-in-my-var-lib-mysql-directory
I have a script that needs to add data to a folder created in the mysql data folder and then in some way force the mysql server to reload the data so the database reliably shows up. I am currently using a system call to the /etc/init.d/mysql script to restart the server but this is quite slow, I couldn't find anyhting in mysql that would reload the database through a query of some kind.. I was just wondering if there was a way to do this that was slightly quicker?
This depends on the storage engine you're using.
MyISAM tables will be reloaded automatically as soon as the table is next touched by MySQL (obviously, you need to make sure the .MYI, .MYD and .frm files are consistent).
InnoDB is a bit more complex, and depends on whether it's a traditional configuration with all tables in the same tablespace, or if you're using innodb_file_per_table. With the former, you pretty much have to restart the server. With file-per-table, you can copy individual .ibd files, but then you have to use an ALTER TABLE statement to import that tablespace.
See here for information on working with individual .ibd files without restarting the server.
This page is also useful for seeing how to move InnoDB data from place to place.
What do you mean by 'quite slow'?
You could do a dump/load of the new db in a one named orginal_db_name_temp, rename the original DB, and then rename orginal_db_name_temp to orginal_db_name - then you will have your new DB in place.
However, I don't know which method is faster by your means - because a dump&load roundtrip will be probably slower than a file-copy. But renaming the DBs will probably be faster than restarting the server
I need to dump a database from a shared hosting that somehow doesn't have mysqldump installed. In fact, I only have mysql and mysqladmin available from the whole set of MySQL utilities.
Is it doable or I'll need to resort to installing something like phpMyAdmin?
You could use the following methods (from Database Backups in the documentation)
Making Backups by Copying Files
MyISAM tables are stored as files, so it is easy to do a backup by copying files. To get a consistent backup, do a LOCK TABLES on the relevant tables, followed by FLUSH TABLES for the tables. You need only a read lock; this allows other clients to continue to query the tables while you are making a copy of the files in the database directory. The FLUSH TABLES statement is needed to ensure that the all active index pages are written to disk before you start the backup.
FLUSH TABLES WITH READ LOCK;
Closes all open tables and locks all tables for all databases with a read lock until you explicitly release the lock by executing UNLOCK TABLES. This is very convenient way to get backups if you have a file system such as Veritas that can take snapshots in time.
UNLOCK TABLES;
Making Delimited-Text File Backups
To create a text file containing a table's data, you can use:
SELECT * INTO OUTFILE 'file_name' FROM tbl_name
This method works for any kind of data file, but saves only table data, not the table structure.
To reload the output file, use"
LOAD DATA INFILE
how about shutting down the server and copying the datadir itself?
You can get SQLYog. It has a back up database as SQL Dump option for each database.
Anyways,
I had to resort to using the Sypex dumper, a web-based tool for fast (really fast, much faster than phpMyAdmin for instance) MySQL database dumping. It's in Russian, but the interface is fairly obvious.
You can connect to a server remotely with mysqldump. For example:
mysqldump -u poweruser -h remote.mysql.host database
Maatkit seems quite appropriate for this with mk-parallel-dump and mk-parallel-restore.