The problem is, that one MYI and one MYD file from MySQL database has been accidentally deleted. The only file left intact is FRM one. Only one table from the whole database is damaged that way, all other tables are OK and the database works generally fine, except the table with deleted files, which is obviously inaccessible.
There's a full database dump in pure SQL format available.
The question is, how do I re-create these files and table in safe and proper manner?
My first idea was to extract the full create table command from the dump and run it on live database. It's not so easy, as the whole dump file has over 10GB, so any operations within its content are really pain in . Yes, I know about sed and know how to use it - but I consider it the last option to choose.
Second and current idea is to create copy of this database on independent server, make a dump of the table in question and then use resulting SQL file to create the table again on the production server. I'm not quite experienced with MySQL administration tasks (well, just basic ones), but for me this option seems to be safe and reasonable.
Will the second option work as I expect?
Is it the best option, or are there any more recommendable solutions?
Thank you in advance for your help.
The simplest solution is to copy the table you deleted. There's a chance mysqld still has an open file handle to the data files you deleted. On UNIX/Linux/OS X, a file isn't truly deleted while some process still has an open file handle to it.
So you might be able to do this:
mysql> CREATE TABLE mytable_copy LIKE mytable;
mysql> INSERT INTO mytable_copy SELECT * FROM mytable;
If you've restarted MySQL Server since you deleted the files, this won't work. If the server has closed its file handle to the data file, this won't work. If you're on Windows, I have no idea.
The next simplest solution is to restore your existing 10GB dump file to a temporary instance of MySQL Server, as you said. I'd use MySQL Sandbox but some people would use a virtual machine, or if you're using an AWS environment, launch a spot EC2 instance or a small RDS instance.
Then dump just the table you need:
mysqldump -h tempserver mydatabase mytable > mytable.sql
Then restore it to your real server.
mysql -h realserver mydatabase < mytable.sql
(I'm omitting the user & password options, I prefer to put those in .my.cnf anyway)
Related
I'm on Windows using mySQL Workbench when I absolutely have to, otherwise ADO.
I'm experienced with Jet databases where you just create your mdb file, put it wherever you want whenever you want, and open the database with an ADO connection string containing the path to the database.
Now I've got to do this wretched mySQL thing ... for some reason I don't understand there's this Data directory where all the databases go and you can't change that when creating the database(?).
So I create my database as a shell (1 junk table to be deleted later, no records) and it's stuck in this worthless Data directory. Now I want to copy/paste this shell to where I really want it, and start working on it through ADO to create my "real" database ... but it doesn't work; my ADO connection string contains the new path and the connection opens with no errors, but all operations only change the copy in the Data directory.
Yes I have searched this topic, and the only "solutions" I see are to change the default Data directory in the ini file ... this seems silly and unworkable; what if I want to create another database and put it somewhere else?
Can someone please shed some light here?
Once you connect to the MySQL database as a user that can create databases, usually the root user after install, you can just write:
CREATE DATABASE my_database;
and that database will be created on the machine that's running the MySQL server process in the datadir that MySQL is using. There are default locations for these things, but you probably won't need to worry about them. You don't need to worry about where the files end up going as the first commenter says.
Here is the information for finding your my.cnf file on Windows. Once you find that file, update your datadir to the location that you want like this:
datadir="C:\New Data\Path"
and then stop your MySQL server and move your files. Restart the MySQL server after you copy all of the files in your current datadir to it. You can find your current datadir with the following command (connected to the server as the root user):
SHOW VARIABLES LIKE '%datadir%';
Make sure to stop the server before moving your files.
The CREATE TABLE documentation shows that you can create a table in a TABLESPACE that's outside of your datadir. First create a tablespace like the following:
CREATE TABLESPACE my_tablespace ADD DATAFILE 'C:\my_space';
which needs to be a single file if you want to use the InnoDB engine. You can then create tables in that table space by specifying it in your table's creation like:
CREATE TABLE new_table (a INT) TABLESPACE my_tablespace;
I haven't done this before, but it should work.
I have a local database thats about 1GB and my remote host is a free host that I am using for testing. want to make sure everything works before i spend money on a paid host. The problem is the phpmyadmin on the remote server only allows 50mb files which which just doesn't cut it, especially since the restore usually fails due to execution time limits. Below is the list of everything I've tried.
LOCAL
phpmyadmin -----> backingup of table no longer work because of timeout even with modified php.ini settings because of shear size of db
mysqldumper -----> program creates dumps with inserts, there is no option for me to make it create insert ignores. ill explain the problem later below.
mysqlworkbench -----> creates database using database name of my local server (problem is my remote server has a different database name and i cant open a 1gb .sql file to edit the database name at the very top. computer just craps out and I have to force quit workbench)
sqlsplitter (mac program) cuts up large .sql or .sql.gz files
REMOTE
phpmyadmin with .gz/.sql files cut up into 20mb chunks
-----> timeout. phpmyadmin resume function doesnt work either. it just overwrites old data
mysqldumper -----> process ends up in an error randomly midway through my restore on remote server using a backup created with mysqldumper on my local computer (single file or multipart, both dont work). could be at 10% completion, could be at 50%.
bigdump -----> used single and multipart dumps from mysqldumper, same problem. randomly quits halfway through. some multiparts were successful in completing, but when one failed and I tried the failed part again, it would give me an error saying unique key already exists in table. i dont want to unset all my unique key stuff and have to go through and delete all duplicates later.
mysqldumper -----> does not work with dump from mysqlworkbench
bigdump -----> gives me an error sql error denied for creating database using dump from mysqlworkbench (i cannot open up a 1 gb file to delete that 1 line that says create database)
Does anybody know of a better method to upload to my host? I have no command line access on there and only a 500mb space limit (no limit on sql space though).
Thanks
Use mysqldump. Figure out what the error you're seeing is, and fix it. The mysqldump utility works. I've restored dumpfiles with hundreds of gigabytes of data to servers, and never use anything else. If it doesn't work for you, you're doing something incorrectly.
You can prevent it from writing a USE database-name; statement at the top of the file by invoking it with the database name as the last argument, without using the --databases option before it.
You can add the --insert-ignore command line option to write all the INSERT statements as INSERT IGNORE to work around your partial insert issues
You can use --no-data to extract a dump file that contains table definitions, not data, and get all of the tables declared, first.
You can use the --no-create-info option to extract a dump file with just the inserts, not the table definitions.
http://dev.mysql.com/doc/refman/5.6/en/mysqldump.html
You can also use a simple bash loop to extract each table into its own file, so you have smaller files to work with:
for TABLE in `mysql [args] -e 'show tables in database-name'`; do mysqldump [args] database-name $TABLE > $TABLE.sql; done
When restoring the files, add the --compress option to the mysql command line arguments for a faster transfer, and specify your (new) database name as the last argument, so the client will use the correct database before applying the file, which no longer contains the database name.
Is it possible to duplicate a MySQL database from their files? [I know mysqldump would be the best method to duplicate a db, but that's not possible as all we have is the backed up files from the mysql folder].
We have the ibdata1 file, the ib_log* files, and the full directories for the three db's we want to restore from the backed up files (folders seem to contain all needed frm and par files). Obviously already tried just to copy all the files into /var/lib/mysql and though it appeared the structure was intact attempts to access the data were unsuccessful.
i.e. show databases will show the db's, use dbname works, and show tables properly displays the tables, but when trying to access the data from [any] table (via query) it says ERROR 1146 (42S02): Table 'dbname.dbtable' doesn't exist - despite mysql having happily showed us the table does exist when we did show tables.
Should also point out the service was stopped prior to copying files and all files chown'ed to have mysql as owner and then the service restarted prior to attempting to access the data.
To answer your question indirectly, there is some information here regarding setting up replication using a raw file copy. So I guess the answer is possibly yes, but it may depend.
http://dev.mysql.com/doc/refman/5.0/en/replication-howto-rawdata.html
Not wishing to add to your current pain, but were you relying on a backup that you have never tried / don't know how to restore?
I've got a database (all MyISAM tables) and the machine where MySQL was running is no longer bootable. However, we have all the MySQL data files from the data directory. How can I restore the data from the MYD and FRM files, or whatever other files I should be looking at in the data directory?
I've been doing some searching on this and it sounds like for MyISAM I should just be able to copy the database subdirectory from the old MySQL data directory to the new MySQL data directory. However, that's not working for me. A database with the name of the database I'm trying to recover shows up in the list of databases in phpMyAdmin, but all the tables show "in use" and have no information (e.g., number of rows, number of bytes, column information, etc.). Any operation on those tables (e.g., SELECT * FROM {table}, REPAIR {table}, CHECK {table}) returns a "no such table" error.
One of the tools I ran across in my search is DBACentral by MicroOLAP. It's got component that's supposed to restore data from FRM/MYD files, but when I tried to run it, it didn't list any tables that it could recover from my FRM/MYD files.
This is on a developer workstation that's running Vista Business 32bit. MySQL version is 5.0.27. After fixing the machine, I went and got the exact same version of MySQL (v5.0.27), thinking that if I'm just going to drop in the binary data files I should do it with the same version of MySQL. It still didn't work.
Any insights would be greatly appreciated... thanks!
-Josh
Install the same version of mysql.
Remove mysql directory from data directory of the server and copy it from the crashed server. This is the key element
copy directory of database you want to recover into data directory of new server
start mysql.
switch to mysql database: USE mysql; and run REPAIR TABLE <table name> on every table.
Do the same with database you want to recover
tip: make sure the 2 directories have the same permissions like data directory
If you did not save mysql database (mysql directory in your old server's data dir, then you can try to:
create database with the same name as database you want to recover.
Then you can create each table (it would be good to use the same structure - you'd have bigger chance of recovery).
then stop mysql server and delete files from database directory and overwrite them with files from old server
start mysql and repair each table.
I wound up giving up. I think the answer is that, with my particular version of MySQL, this doesn't work. Hopefully things have improved since then.
Sorry for a noob question regarding MySQL. I downloaded FlightStats to learn about mysql but I can't figure out how to register it with my localhost mysql db. I know in MS SQL you can simply register any sql db using sql studio. I tried to google but come up with no result. Perhaps, my search phrase is wrong. I'm searching with "how to register a mysql database, register a mysql database...etc.". How do you register or setup an database from existing database like FlightStats? I'm using DBVisualizer. Is there a way in dbVis that I'm not aware of to regsiter a database?
Thanks
edit: sorry for the bad wording. I found this. I have the .myd, .myi and .frm and I want to get it to restore(?) with my local mysql instance. I look at all the answers but I'm still confuse as how you restore the database from those 3 files.
A little background first. The FlightStats download page linked to in the original question appears to provide zipped tarballs of the binary table storage files from the MySQL data directory. Given that this is considered a viable means of distribution, and combined with the use of MERGE tables, I would surmise that this tarball contains a bunch of MyISAM data files (.myi, .myd). Jack's edit confirms that this is the situation.
This is an atypical means of distributing a MySQL data set, although not at all uncommon when backing up MyISAM storage, and probably not all that unheard of for moving large data sets around; it likely works out considerably more space-efficient than a corresponding dump file. Of course, in SQL Server land, it's pretty common to attach database files into an instance.
Broadly speaking, you'd recover the database as follows:
Locate the MySQL data directory; typically /var/mysql or similar
Create a new directory with the desired database name e.g. flightdata
Extract the .myi, .myd and other files from the tarball into this directory
Make sure the entire directory is owned by the user MySQL runs as (usually mysql) - use chmod -R to make sure you get everything
Open a MySQL console
USE <database-name>
SHOW TABLES
You should see some tables listed. In addition, the downloads page linked includes a couple of SQL scripts, which contain SQL commands that you need to run against your database once it's in place. These will cause the merge definitions and table indexes to be rebuilt. You can pipe these into the command-line client, e.g. mysql -u<username> -p<password> <database-name> < <sql-file>.
It may be a good idea to shut down the MySQL server while you're doing this; use e.g. /etc/init.d/mysql stop or similar, and restart once the files are extracted in place.
There's generally a way to import sql files using a GUI database tool. I'm not familiar with DBVisualizer, but as long as you have a MySQL command line client installed you can do it there as well. It's pretty easy:
Create a blank schema. You can do this in your GUI tool or on the command line client. Just use CREATE DATABASE flightstats;, or whatever name you want.
Use the following command line syntax to import/run an sql file on the new schema: mysql -u <username> -p flightstats < /path/to/file.sql
The -p option prompts for a password. I generally set up the database using step 1 as the root user, then GRANT some permissions on it to a new user id, then use that user id to run the SQL file.
This process is pretty much what a GUI tool will do in the background.
Registering a database? dont know what that means however mysql gui tools can help you creating a database. Have a look at it or better you download phpmyadmin.
Google WAMP for Windows.
Google MAMP for Mac.
Google LAMP for Linux.
Any questions?