I'm trying to clone raw data from all databases on a MySQL instance in Live to a test environment. The network guys have told me the data has been synched and copied across but I can't start the MySQL instance in the test environment. I'm using the innodb engine and I can see the ibdata1 file, mysql-bin files and ib_logfiles copied over along with the relevant db folders.
The error I'm getting in the error log looks like the following:
130911 13:53:08 InnoDB: Error: table <table-name>
InnoDB: in InnoDB data dictionary has tablespace id <id>,
InnoDB: but tablespace with that id or name does not exist. Have
InnoDB: you deleted or moved .ibd files?
The cloning process doesn't stop the Live MySQL instance and I'm wondering is this the problem. I don't want to use mysqldump or another backup tool. I just want to copy the raw data across. Thanks for any advice.
You can't hot-copy these files and expect them to magically work.
You can use the innobackupex tool to create a stable snapshot. This will take care of adjusting the files as necessary to be consistent and complete.
Related
I am attempting to restore my old databases to a new Wamp installation. However, I did not take a backup of Mysql files before I did this. Thankfully the uninstallation has not deleted all the files, but in my data folder I have :
1) All the databases (As folders)
2) auto.cnf
There are no ib* files in the old folder. Is it still possible to restore my databases somehow?
I have tried to replace the entire mysql5.6.17 folder (there has been no change of version between the installations), but Wamp remains orange and never turns green.
If I replace only the data folder, the MySQL service starts, I can see the databases, but not the tables.
If I change the path in my.ini to point to the old wamp's data folder, I cannot see the old databases at all.
EDIT : Was experimenting a bit, copying the database folders directly into the new Wamp's data folder allows me to get a list of all the tables properly, but there is no actual data in the tables, and trying to query it using MySQLWorkbench gives me the error :
Error Code: 1146. Table 'testdb.users' doesn't exist
EDIT 2 : Found this in the error log file in my data folder :
2014-08-03 06:24:46 25164 [Warning] InnoDB: Cannot open table testdb/users from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
As I thought, its a problem with InnoDB's own mapping. Is there a way to resolve this manually?
If you created an INNODB database, and made no other config changes to store the actual data in seperate table spaces then all the actual data for that INNODB databases will have been stored in the \wamp\bin\mysql\mysql5.6.17\ibdata1 file.
If you do not have that file from the old install then your data is lost.
There will also be info in a \wamp\bin\mysql\mysql5.6.17\data\DATA_BASE_NAME\ folder, one .frm file for each table that was in the database, but this is basically just schema information and not actual data.
For future reference, InnoDB Backup and Recovery
GOLDEN RULE:
Always backup database (from phpmyadmin) before major changes.. better to spend 1 minute for that, than going into trouble..
How to Copy data folder
If you want to copy the database, then INSTALLATION/mysql versions should be same.. then rename newly installed data folder to i.e. data_my_old,so now there you can place the backuped data folder (it should contain ibdata1 file).
Depends on innodb_file_per_table setting.
If it was OFF then all your tables were in ibdata1. Now you have quite new ibdata1 (since you see errors "Cannot open table testdb/users from the internal data dictionary", that means InnoDB runs fresh new dictionary). That means that now the data is somewhere on the disk. The recovery is quite difficult and requires a lot of tedious work. You need to take an image of the disk, scan it with stream_parser to find InnoDB pages, then recover the InnoDB dictionary, then find index_id of every table and if the stream_parser has found this index run c_parser to extract records from the index.
If innodb_file_per_table was ON and you do see .ibd files in the databases directories then recovery is much easier. First, you may try to reconnect individual ibd files with ibdconnect or extract records from the ibd files with stream_parser/c_parser.
The mentioned tools are part of TwinDB data recovery toolkit
i have just done a google search and found this....
Go to C:\wamp\bin\mysql\mysql5.5.24\data
Find a file called "mysql-bin.index"
Rename that file to "mysql-bin.index-corrupt"
Restart the server and it will work fine, all your database will be where you left them.
I am running MySQL 5.1 on Ubuntu 10.04, and until recently, I used the default data dir location. Some other parts of the config file were tuned for performance, but the paths stayed default.
Recently, I started running out of space and decided to add another hard disk, mount it as /mysql and use it solely for MySQL data. So, I changed the paths, copied the old data dir into the new data dir and thought that would be the end of it.
Unfortunately, it wasn't - and it later turned out that apparmor was the issue, even though I updated the MySQL profile in apparmor to reflect the new path(s). After some messing about, disabling apparmor, the server would work and I was able to import the big database that is the original reason I needed more space.
Now, that was yesterday - the whole 200GB database was imported, keys were sorted and everything seemed fine until I tried to start the server today. Here's the error that I see in the log:
120913 13:53:38 InnoDB: Operating system error number 2 in a file operation.
InnoDB: The error means the system cannot find the path specified.
InnoDB: If you are installing InnoDB, remember that you must create
InnoDB: directories yourself, InnoDB does not create them.
InnoDB: File name /home/{my_username}/mysql/data/ibdata1
InnoDB: File operation call: 'create'.
InnoDB: Cannot continue operation.
Here's a few strange things with that:
a) I'm sudo-ed in as root, and I'm using the 'service mysql start' command to start it
b) There's no mention of /home/{my_username}... path ANYWHERE in any of the configs.
I couldn't find any info or bug reports regarding this type of a problem. I don't even know what I would search for, since the problem can't really be explained in less than 2 paragraphs.
Further information: Manually setting innodb_data_home_dir eliminates the earlier problem, however, now I get this instead:
120913 14:08:06 InnoDB: Operating system error number 2 in a file operation.
InnoDB: The error means the system cannot find the path specified.
InnoDB: If you are installing InnoDB, remember that you must create
InnoDB: directories yourself, InnoDB does not create them.
InnoDB: File name /home/poplar/mysql/innodb-logs/ib_logfile0
InnoDB: File operation call: 'create'.
InnoDB: Cannot continue operation.
Now, there's no "poplar" user on this box, and I haven't got the faintest idea why would it want to be trying to put the log file there.
Well, it turned out that setting innodb_log_group_home_dir to the new MySQL data dir (where my log was before, and where it should default to anyway) did the trick.
The server now starts properly and all the data seems to be there.
I still don't know where it got the 'poplar' username as a good idea to try placing log files, but it could be some leftover (mis)configuration from AppArmor that wasn't cleanly re-set when I uninstalled it.
I want to purge the ibdata1, ib_logfile0 and ib_logfile1 files in MySQL. Because these files are taking too much space of my C:\ disk. And I don't want to move these files anywhere else in the Hard disk. I heard about purging which can shrink the data. I am using Windows Vista. What are the steps to be done to purge the data ?
The log files can be removed anytime. Only the most recent log is kept open/locked. However, deleting id_data will trash all of your InnoDB databases - that's the actual data store.
As pointed out by MarcB, you can delete the (old) logfiles without problems; MySQL does not need them to run.
The data file ibdata* must not be deleted, it contains the actual database data.
I will grow as the database grows; unfortunately there is not way to shrink it, even if you later delete data from the database. The only way is to dump and reimport the whole database.
Also see this question: MySQL data file won't shrink
It explains various options to better cope with growing data files.
Short answer: There's no way to reduce the size of the ibdata files once they grew to a certain size. MySQL recommends dumping all innodb type data, shutting down mysql, deleting the ibdata files and restarting, then loading the data again. A one-click "compact" solution like other systems have is still missing.
Here's the case:
After computer being hacked, we are in a rush to backup all data out to other computer.
As a result, the mysql databases are not backup out as sql statement.
What we have done is backup out all the physical files/folders in the C drive to new computer.
Eg:
C:\Program Files\MySQL\MySQL Server 4.1\data
In this case, all data for mysql are inside unreadable file.
Inside data folder consist of files like ib_logfile0, ib_logfile1, but not ib_data1
All database's table structure format are inside each respective folder. (Some folder have .frm, .opt) (some other folder have .frm, .myd, .myi)
How can I retrieve back the data from the database in a new computer?
I tried to install the same mysql version(4.1) at new computer, then replace all backup files inside data folder into this mysql in new computer. Then restart mysql service.
When I restart, it fail: Could not start mysql service on local computer. error 1067: process terminated unexpectedly.
Error log showing:
InnoDB: The first specified data file .\ibdata1 did not exist:
InnoDB: a new database to be created!
090930 10:24:49 InnoDB: Setting file .\ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Error: log file .\ib_logfile0 is of different size 0 87031808 bytes
InnoDB: than specified in the .cnf file 0 25165824 bytes!
090930 10:24:49 [ERROR] Can't init databases
090930 10:24:49 [ERROR] Aborting
090930 10:24:49 [Note] C:\Program Files\MySQL\MySQL Server 4.1\bin\mysqld-nt: Shutdown complete
your error log indicates that your innodb configuration does not match your innodb files. you need to look at the configuration file on your old server to figure out where mysql was configured to store your innodb files. then go find those files and copy them to the new server. you should also copy the innodb configuration settings exactly from the old server and apply them to the new server.
you need to look at the mysql.err log in the data folder to see what's wrong.
did the mysql service start properly when you first installed it?
looks like you're on windows, so i'm going to guess that you moved the files instead of copying them. when you move files in windows, they retain their permissions. when you copy, they inherit the permissions from the folder. so check to make sure they have the right permissions.
Sounds like your innodb_log_file_size in my.ini was not the same as before.
Have a look a the ib_logfile0 filesize in the data folder and if it's 102,400KB it should be:
innodb_log_file_size=100M
Yours looks like it should be:
innodb_log_file_size=83M
(set to 24M now)
I've had this issue when someone used the installer to "repair" a mysql installation.
Have Fun.
I have to restore a database that has been inadvertently DROPped in MySQL 5.0. From checking the backup files, I only seem to have .FRM files to hold the database data.
Can anyone advise whether this is all I need to perform a database restore/import from the backup, or are there other files I should have to hand to complete this?
.frm files are not the data files, they just store the "data dictionary information" (see MySQL manual). InnoDB stores its data in ib_logfile* files. That's what you need in order to do a backup/restore. For more details see here.
Restoring innodb:
(assuming your data folder is C:\ProgramData\MySQL\MySQL Server 5.5\data)
Copy the folders of the databases (named after the database name) you want to restore to C:\ProgramData\MySQL\MySQL Server 5.5\data
Copy the 3 ibdata files to the data folder ex. (C:\ProgramData\MySQL\MySQL Server 5.5\data)
_ib_logfile0
_ib_logfile1
_ibdata1
Get the size of the _ib_logfile0 in MB (it should be the same as _ib_logfile1) by File Right click -> Properties
Edit the mysql config file (mysql\bin\my.ini) for the innodb_log_file_size=343M to be exactly the ibdata files size
Run
mysqld --defaults-file=mysql\bin\my.ini --standalone --console
--innodb_force_recovery=6
Now your data should be back in your database. Export them using phpmysql or any other tool
The detailed solution you can found here:
http://www.unilogica.com/mysql-innodb-recovery/ (Article in Portuguese)
Besides the flag of innodb_force_recovery, I found another solution: innodb_file_per_table, that splits InnoDB tables in each file like MyISAM tables.
In a crash recovery you can lost less data than in single file ibdata1.