I have database on the server but as a developer when we found some bug in the product then to resolved that bug quickly we need to take dump of database which is currently present on the server.As the db size is much larger so it is not possible everyday to create dump and download it which is wasting some times.So I wanted know is there any tool or way which will only give me data which is not present on my local machine and I can integrate that new data into db which is present on the local host machine. So it will save development time.I know some db difference tools like mysql-diff, Toad for MySql are there but I dont think they will solved problem as they are useful to see the differences between two db only.If they can solved my problem then please let me know how?
Any help to achieve this will be appreciable.
As you're talking about a production database, I'd err on the side of caution, and just use mysqldump to dump out the relevant tables, rather than the whole database.
mysqldump -u dbuser -p -h 127.0.0.1 database_name table1 table2 table_etc
Alternatively, you could try rsync to synchronise the actual database files. You'll need to flush the tables, too - to ensure the data is written to disk, rather than hanging around in buffers.
If you do try the rsync method, just be sure to test it extensively.
Related
SO guys, we had legacy website that has a 12GB MySQL database. We decided to port the legacy website to WordPress and need to import the database into PHPmyAdmin without breaking the database and to implement this in the shortest possible time (although time is not such a critical matter if this can be done painlessly but correctly). So would appreciate if someone can guide me in the right direction and provide detailed instructions.
Thanks guys.
I dont know much but i can recommend to back up you're files befored doing any transfers. If you have a fast internet speed then you guys can try to download the SQL database as a compressed file and dump it into the wordpress new database. The database for mysql should be in your cpanel folder (since all cpanel folders/designs are different it would be best to look up "where to find the msql database on hosting company name's cpanel" and once you find it then you can transfer it whereever is possible)
This is the easiest and most correct way.
mysql -u username -p "dbname" < "path/to/bigdata.sql"
If you have FTP/SSH access, then this is the most probable way.
Upload your .sql via FTP
If you have access to CLI , just run the command above
Normally , this will be fast, but since its 12GB, it might take 30min-1hour depending on server resources.
You can also use other options like this one .
I've already experienced importing 61GB before with just this one-liner.
Well I'm not much of a good developer or a database expert. But I have a little understanding of these things. I'm trying to dump a database on a VPS using "mysqldump" command which works perfectly. But when I tried to restore locally after downloading the dump, gives me a time out error.
Can anyone advise me how to dump a database by splitting it into tables separately. The database I'm referring to is pretty large (6 - 7 GB). I actually tried searching and it confuses me.. even this link here confuses me as where to start.
Any help is highly appreciated.
Are you restoring with phpmyadmin? If you try to upload the import it is probably too large.
You can set a directory where the backup files are stored, then you can select the file in phpmyadmin without uploading it.
For the timeout with importing you can increase the timeout settings, or use something like "BigDump"
If you're using mysqldump I'll assume you're familiar with the command line.
On your local machine, use
mysql -u [root] -p [database_name] < [database_dump.sql] -v
enter password: ********
The empty database needs to be created your local machine first before you can import the structure and data to it (as simple as doing CREATE DATABASE [database_name];)
The -v flag will do it in 'verbose' mode so you can see the queries as they run. Omitting '-v' will stop it filling your window with the queries but will also give you that 'is it working or not?' nervous feeling after a few minutes.
This will work on Windows as well as it works on Linux / Mac / anything else
Replace my [placeholders] with your own values.
Thank you so much for all your answers! Well, what I was looking for is a dumping method or a similar script to dump the database table by table. Finally I tried the dumping the output file with a .txt extension which returned me with success.
Below is the command I used (I know its pretty long proceess, but I finally got all tables dumped);
mysqldump -u users -p database_name table_name > table_name.txt
I used the current directory to output the file assuming I'm already in the directory where I need to dump. If you need to dump the output file to a specific dir, then use /path/to/the/dump/table_name.txt instead of just mentioning the table name. Ans make sure you don't enter password after -p. I don't know why, but I left it blank and it prompts for the password. Then when I type password it dumps to a text file.
I hope this helps.!
Once again thank you so much for the users who came in the first place to help me. :)
I'm considering switching to a new hosting provider, and I would like to transfer my database for my production site to the new hosting provider. I'm using mysql. What are the steps I would need to take to transfer my db?
Appreciate any help.
Thank you,
Brian
Assuming a relatively simple app (PHP, something like that), one app server, one db server, then briefly:
On the new host, create the necessary accounts on the database that you're using on the old host's database.
Copy the app code over.
"Lock" your app on the old host so no data changes can occur (if this is feasible.)
http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html is your friend. Dump schema and data, and capture it to a file. Here is the command I used to dump the database exampledb that has the login of example:
mysqldump --add-drop-table -u example -p exampledb > output.sql
(The --add-drop-table makes it easier to re-run the script if you need to later. But it does create a script that will destroy your database, so careful how you run it.)
Now copy (maybe using scp) the output.sql file to your new host.
On the new host, run mysql to build the database with the schema and data from the old host. I use a command like this one, assuming user "example" and a database name of "exampledb":
mysql -u example -p exampledb < output.sql
(Be careful to run this ONLY ON THE NEW HOST. It will obliterate your database.)
The nice thing is, you've got a blank slate of a new machine. You can keep trying different things on that machine without breaking anything.
Turn on the app on new host. Test. If it's been a while, you may need to make changes to get your code up to a newer version of the language. (I did in my case. But maybe you were better about keeping your code up to date.)
Shut down app on old host.
Point DNS/router/whatever to new host.
What'd I miss? (Just went through this moving my silly website to a new machine.)
It's pretty simple, especially for just a single database?
mysqldump followed by a mysqlimport.
MySQL Dump
Generating the .sql file is all you need, because that will contain all of the table information such as CREATE INDEXES, which when you then run through all of your inserts, will add the indexes.
If you struggle with command lines, may I suggest using Navicat Lite. It is free, and is the best GUI that I've seen on the market.
Navicat Lite
Sorry for a noob question regarding MySQL. I downloaded FlightStats to learn about mysql but I can't figure out how to register it with my localhost mysql db. I know in MS SQL you can simply register any sql db using sql studio. I tried to google but come up with no result. Perhaps, my search phrase is wrong. I'm searching with "how to register a mysql database, register a mysql database...etc.". How do you register or setup an database from existing database like FlightStats? I'm using DBVisualizer. Is there a way in dbVis that I'm not aware of to regsiter a database?
Thanks
edit: sorry for the bad wording. I found this. I have the .myd, .myi and .frm and I want to get it to restore(?) with my local mysql instance. I look at all the answers but I'm still confuse as how you restore the database from those 3 files.
A little background first. The FlightStats download page linked to in the original question appears to provide zipped tarballs of the binary table storage files from the MySQL data directory. Given that this is considered a viable means of distribution, and combined with the use of MERGE tables, I would surmise that this tarball contains a bunch of MyISAM data files (.myi, .myd). Jack's edit confirms that this is the situation.
This is an atypical means of distributing a MySQL data set, although not at all uncommon when backing up MyISAM storage, and probably not all that unheard of for moving large data sets around; it likely works out considerably more space-efficient than a corresponding dump file. Of course, in SQL Server land, it's pretty common to attach database files into an instance.
Broadly speaking, you'd recover the database as follows:
Locate the MySQL data directory; typically /var/mysql or similar
Create a new directory with the desired database name e.g. flightdata
Extract the .myi, .myd and other files from the tarball into this directory
Make sure the entire directory is owned by the user MySQL runs as (usually mysql) - use chmod -R to make sure you get everything
Open a MySQL console
USE <database-name>
SHOW TABLES
You should see some tables listed. In addition, the downloads page linked includes a couple of SQL scripts, which contain SQL commands that you need to run against your database once it's in place. These will cause the merge definitions and table indexes to be rebuilt. You can pipe these into the command-line client, e.g. mysql -u<username> -p<password> <database-name> < <sql-file>.
It may be a good idea to shut down the MySQL server while you're doing this; use e.g. /etc/init.d/mysql stop or similar, and restart once the files are extracted in place.
There's generally a way to import sql files using a GUI database tool. I'm not familiar with DBVisualizer, but as long as you have a MySQL command line client installed you can do it there as well. It's pretty easy:
Create a blank schema. You can do this in your GUI tool or on the command line client. Just use CREATE DATABASE flightstats;, or whatever name you want.
Use the following command line syntax to import/run an sql file on the new schema: mysql -u <username> -p flightstats < /path/to/file.sql
The -p option prompts for a password. I generally set up the database using step 1 as the root user, then GRANT some permissions on it to a new user id, then use that user id to run the SQL file.
This process is pretty much what a GUI tool will do in the background.
Registering a database? dont know what that means however mysql gui tools can help you creating a database. Have a look at it or better you download phpmyadmin.
Google WAMP for Windows.
Google MAMP for Mac.
Google LAMP for Linux.
Any questions?
I have a test database on a separate remote server than my production DB. Every once in awhile, I want to try and test things by uploading a copy of my production DB to my testing DB. Unfortunately, the backup file is now half a gig and I'm having trouble transferring it via FTP or SSH. Is there an easy way that I can use the mysql restore command between servers? Also, is there another way to move over large files that I'm not considering? Half a gig doesn't seem that big, I would imagine that people run into this issue frequently.
Thanks!
Are the servers accessible to each other?
If so, you can just pipe the data from one db to another without using a file.
ex: mysqldump [options] | mysql -h test -u username -ppasswd
0.Please consider whether you really need production data (especially if it contains some sensitive information)
1.The simplest solution is to compress the backup on the source server (usually gzip), transfer it across the wire, then decompress on the target server.
http://www.techiecorner.com/44/how-to-backup-mysql-database-in-command-line-with-compression/
2.If you don't need the exact replica of production data (e.g. you don't need some application logs, errors, some other technical stuff) you can consider creating a backup and restore on a source server to a different DB name, delete all unnecessary data and THEN take a backup that you will use.
3.Restore full backup once on your reference server in your Dev environment and then copy transaction logs only (to replay them on the reference server). Depending on the usage pattern transaction logs may take a lot less space as the whole database.
Mysql allows you to connect to a remote database server to run sql commands. Using this feature, we can pipe the output from mysqldump and ask mysql to connect to the remote database server to populate the new database.
mysqldump -u root -p rootpass SalesDb | mysql --host=185.32.31.96 -C SalesDb
Use an efficient transfer method, rather than ftp.
If you have a dump file created by mysqldump, on the test db server, and you update it every so often. I think you could save time (if not disk space) by using rsync to transfer it. Rsync will use ssh and compress data for the transfer, but I think both the local and remote files should/could be uncompressed.
Rsync will only transfer the changed portion of a file.
It may take some time to decide what, precisely, has changed in a dump file, but the transfer should be quick.
I must admit though, I've never done it with a half-gigabyte dump file.