ssh Mysql dump table parts (11GB DB to smaller pieces) - mysql

I'm facing the following:
We have a DB table of 11GB with over 257 million records and need a backup. Exporting via PHPmyAdmin isn't possible (chrome keeps crashing) and backing up with SSH mysqldump tablename will give a insufficient space disk error (error 28).
Now I'd like to know if there is a way to export a mysqldump with a row 0 till ~100.000.000 command so we can make 3 parts (or smaller parts if required).
What I'm using:
mysqldump -p -u username database_name database_table > dbname.sql
[EDIT]
Found out how to get a row of <50.0000.0000 to SQL with the following:
mysqldump -p -u db_name db_table --where='id<50000000'
But the big question remains now, how to go further? Now I want to get all records between 50.000.000 and 100.000.000 ..
Anybody knows the answer if it's possible and what command I should use?

Problem solved:
Part 1 (<50.000.000):
mysqldump -p -u db_name db_table --where='id<50000000' >part_1.sql
Part 2 (>50.000.000 till <100.0000.000):
mysqldump -p -u db_name db_table --where='id>=50000000 &&
id<100000000' >part_2.sql
Part last (>250.000.000)
mysqldump -p -u db_name db_table --where='id>250000000' >part_final.sql
And so on..

mysqldump creates a text file that contains sql statements, if want to take mysql backup in parts then you will have to run mysqldump like this
mysqldump --where "id%2=0" database_name table > table_even.sql
mysqldump --where "id%2=1" database_name table > table_odd.sql
OR
you need to write some program, script to achieve that

I found a nice solution for heavy transfers! This might also help you to avoid to transfer your database in parts (as in this example) - since it does this super fast:
Exporting a full database or in parts as mentioned using mysqldump:
mysqldump -p -u db_name db_table --where='id<50000000' >part_1.sql
To import to the new database - login via terminal to the new database:
mysql -h localhost -upotato -p123456
Enter the database:
USE databasename;
Use the source command:
source /path/to/file.sql;
This works X1000 faster than the standard:
mysql -h localhost_new -upotato -p1234567 table_name < /path/to/file.sql
Since you enter the database.

Related

"successful" MySQL dump with no result

I am having a weird problem trying to upload a heavy MySQL dump file into a database.
The PuTTY command I'm using is the following:
homedirectory/userfolder/ mysql -u username –-password=password database_name < file.sql
The procedure runs as it would normally do (I've done this many times already) but when I look for the table that was supposed to be created in 'database_name' I dont see it!
Maybe it is because the file is too big? (4,5GB).
The command used to obtain the dump file was:
mysqldump --user=user --password=password --no-create-db
-h localhost --single-transaction --lock-tables=false
--skip-tz-utc database_name table_name > file.sql
After getting the .sql file, from the same location I ran the command and it seems to work for a few moments, dont get any error messages or any of the like.
After it finishes "loading" I check my database but the table is not there.
Not sure what else to specify, this is a simple procedure I do regularly and I have never had any complications, or at least never without at least an error message.
I recently tried creating sql dump using these commands :
backup: # mysqldump -u root -p[root_password] [database_name] > dumpfilename.sql
restore:# mysql -u root -p[root_password] [database_name] < dumpfilename.sql
Please refer to this link for more information you may need:
http://www.thegeekstuff.com/2008/09/backup-and-restore-mysql-database-using-mysqldump/
Hope this helps!

How do I import an SQL file using the command line in MySQL?

I have a .sql file with an export from phpMyAdmin. I want to import it into a different server using the command line.
I have a Windows Server 2008 R2 installation. I placed the .sql file on the C drive, and I tried this command
database_name < file.sql
It is not working. I get syntax errors.
How can I import this file without a problem?
Do I need to create a database first?
Try:
mysql -u username -p database_name < file.sql
Check MySQL Options.
Note 1: It is better to use the full path of the SQL file file.sql.
Note 2: Use -R and --triggers with mysqldump to keep the routines and triggers of the original database. They are not copied by default.
Note 3 You may have to create the (empty) database from MySQL if it doesn't exist already and the exported SQL doesn't contain CREATE DATABASE (exported with --no-create-db or -n option) before you can import it.
A common use of mysqldump is for making a backup of an entire database:
mysqldump db_name > backup-file.sql
You can load the dump file back into the server like this:
Unix
mysql db_name < backup-file.sql
The same in the Windows command prompt:
mysql -p -u [user] [database] < backup-file.sql
PowerShell
cmd.exe /c "mysql -u root -p db_name < backup-file.sql"
MySQL command line
mysql> use db_name;
mysql> source backup-file.sql;
Regarding the time taken for importing huge files: most importantly, it takes more time because the default setting of MySQL is autocommit = true. You must set that off before importing your file and then check how import works like a gem.
You just need to do the following thing:
mysql> use db_name;
mysql> SET autocommit=0 ; source the_sql_file.sql ; COMMIT ;
Among all the answers, for the problem above, this is the best one:
mysql> use db_name;
mysql> source file_name.sql;
Easiest way to import into your schema:
Login to mysql and issue below mention commands.
mysql> use your_db_name;
mysql> source /opt/file.sql;
We can use this command to import SQL from the command line:
mysql -u username -p password db_name < file.sql
For example, if the username is root and password is password. And you have a database name as bank and the SQL file is bank.sql. Then, simply do like this:
mysql -u root -p password bank < bank.sql
Remember where your SQL file is. If your SQL file is in the Desktop folder/directory then go the desktop directory and enter the command like this:
cd ~/Desktop
mysql -u root -p password bank < bank.sql
And if you are in the Project directory and your SQL file is in the Desktop directory. If you want to access it from the Project directory then you can do like this:
cd ~/Project
mysql -u root -p password bank < ~/Desktop/bank.sql
If you already have the database, use the following to import the dump or the sql file:
mysql -u username -p database_name < file.sql
if you don't you need to create the relevant database(empty) in MySQL, for that first log on to the MySQL console by running the following command in terminal or in cmd
mysql -u userName -p;
And when prompted provide the password.
Next, create a database and use it:
mysql>create database yourDatabaseName;
mysql>use yourDatabaseName;
Then import the sql or the dump file to the database from
mysql> source pathToYourSQLFile;
Note: if your terminal is not in the location where the dump or sql file exists, use the relative path in above.
Open the MySQL command line
Type the path of your mysql bin directory and press Enter
Paste your SQL file inside the bin folder of mysql server.
Create a database in MySQL.
Use that particular database where you want to import the SQL file.
Type source databasefilename.sql and Enter
Your SQL file upload successfully.
A solution that worked for me is below:
Use your_database_name;
SOURCE path_to_db_sql_file_on_your_local;
While most answers here just mention the simple command
mysql -u database_user -p [db_name] < database_file.sql
today it's quite common that databases and tables have utf8-collation where this command is not sufficient.
Having utf8-collation in the exported tables it's required to use this command:
mysql -u database_user -p --default-character-set=utf8 [db_name] < database_file.sql
An according export can be done with
mysqldump -u database_user -p --default-character-set=utf8 [db_name] > database_file.sql
Surely this works for other charsets too, how to show the right notation can be seen here:
https://dev.mysql.com/doc/refman/5.7/en/show-collation.html
One comment mentioned also that if a database never exists an empty database had to be created first. This might be right in some cases but depends on the export file. If the exported file includes already the command to create the database then the database never has to be created in a separate step, which even could cause an error on import. So on import, it's advisable to have a look first in the file to know which commands are included there, on export, it's advisable to note the settings, especially if the file is very large and hard to read in an editor.
There are still more parameters for the command which are listed and explained here:
https://dev.mysql.com/doc/refman/5.7/en/mysql-command-options.html
If you use another database version consider searching for the corresponding version of the manual too. The mentioned links refer to MySQL version 5.7.
EDIT:
The same parameters are working for mysqldump too. So while the commands for export and import are different, the mentioned parameters are not.
Nevertheless there exists a special site in the manual that describes the options for mysqldump: https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html
To dump a database into an SQL file use the following command.
mysqldump -u username -p database_name > database_name.sql
To import an SQL file into a database (make sure you are in the same directory as the SQL file or supply the full path to the file), do:
mysql -u username -p database_name < database_name.sql
I think it's worth mentioning that you can also load a gzipped (compressed) file with zcat like shown below:
zcat database_file.sql.gz | mysql -u username -p -h localhost database_name
Go to the directory where you have the MySQL executable. -u for username and -p to prompt for the password:
C:\xampp\mysql\bin>mysql -u username -ppassword databasename < C:\file.sql
To import a single database, use the following command.
mysql -u username -p password dbname < dump.sql
To import multiple database dumps, use the following command.
mysql -u username -p password < dump.sql
To import a database, use the following command.
mysql> create new_database;
mysql> use new_database;
mysql> source (Here you need to import the path of the SQL file);
E.g.:
mysql> source E:/test/dump.sql;
You need to use forward slashes (/) even on Windows, e.g., E:/test/dump.sql instead of E:\test\dump.sql
Or double backslashes (\\) because of escaping, i.e., E:\\test\\dump.sql
mysql --user=[user] --password=[password] [database] < news_ml_all.sql
I kept running into the problem where the database wasn't created.
I fixed it like this:
mysql -u root -e "CREATE DATABASE db_name"
mysql db_name --force < import_script.sql
For exporting a database:
mysqldump -u username -p database_name > file.sql
For importing a database:
mysql -u username -p database_name < file.sql
For importing multiple SQL files at one time, use this:
# Unix-based solution
for i in *.sql ; do mysql -u root -pPassword DataBase < $i ; done
For simple importing:
# Unix-based solution
mysql -u root -pPassword DataBase < data.sql
For WAMP:
REM mysqlVersion - replace with your own version
C:\wamp\bin\mysql\mysqlVersion\bin\mysql.exe -u root -pPassword DataBase < data.sql
For XAMPP:
C:\xampp\mysql\bin\mysql -u root -pPassword DataBase < data.sql
You do not need to specify the name of the database on the command line if the .sql file contains CREATE DATABASE IF NOT EXISTS db_name and USE db_name statements.
Just make sure you are connecting with a user that has the permissions to create the database, if the database mentioned in the .sql file does not exist.
Import a database
Go to drive:
d:
MySQL login
c:\xampp\mysql\bin\mysql -u root -p
It will ask for pwd. Enter it:
pwd
Select the database
use DbName;
Provide the file name
\.DbName.sql
Use:
mysql -u root -p password -D database_name << import.sql
Use the MySQL help for details - mysql --help.
I think these will be useful options in our context:
[~]$ mysql --help
mysql Ver 14.14 Distrib 5.7.20, for osx10.12 (x86_64) using EditLine wrapper
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Usage: mysql [OPTIONS] [database]
-?, --help Display this help and exit.
-I, --help Synonym for -?
--bind-address=name IP address to bind to.
-D, --database=name Database to use.
--delimiter=name Delimiter to be used.
--default-character-set=name Set the default character set.
-f, --force Continue even if we get an SQL error.
-p, --password[=name] Password to use when connecting to server.
-h, --host=name Connect to host.
-P, --port=# Port number to use for connection or 0 for default to, in order of preference, my.cnf, $MYSQL_TCP_PORT, /etc/services, built-in default (3306).
--protocol=name The protocol to use for connection (tcp, socket, pipe,
-s, --silent Be more silent. Print results with a tab as separator, each row on new line.
-v, --verbose Write more. (-v -v -v gives the table output format).
-V, --version Output version information and exit.
-w, --wait Wait and retry if connection is down.
What is fun, if we are importing a large database and not having a progress bar. Use Pipe Viewer and see the data transfer through the pipe
For Mac, brew install pv
For Debian/Ubuntu, apt-get install pv.
For others, refer to pv - Pipe Viewer
pv import.sql | mysql -u root -p password -D database_name
1.45GiB 1:50:07 [339.0KiB/s] [=============> ] 14% ETA 11:09:36
1.46GiB 1:50:14 [ 246KiB/s] [=============> ] 14% ETA 11:09:15
1.47GiB 1:53:00 [ 385KiB/s] [=============> ] 14% ETA 11:05:36
Go to the directory where you have MySQL.
c:\mysql\bin\> mysql -u username -p password database_name <
filename.sql
Also to dump all databases, use the -all-databases option, and no databases’ name needs to be specified anymore.
mysqldump -u username -ppassword –all-databases > dump.sql
Or you can use some GUI clients like SQLyog to do this.
You can try this query.
Export:
mysqldump -u username –-password=your_password database_name > file.sql
Import:
mysql -u username –-password=your_password database_name < file.sql
and detail following this link:
https://chartio.com/resources/tutorials/importing-from-and-exporting-to-files-using-the-mysql-command-line/
Add the --force option:
mysql -u username -p database_name --force < file.sql
The following command works for me from the command line (cmd) on
Windows 7 on WAMP.
d:/wamp/bin/mysql/mysql5.6.17/bin/mysql.exe -u root -p db_name < database.sql
Providing credentials on the command line is not a good idea. The above answers are great, but neglect to mention
mysql --defaults-extra-file=etc/myhost.cnf database_name < file.sql
Where etc/myhost.cnf is a file that contains host, user, password, and you avoid exposing the password on the command line. Here is a sample,
[client]
host=hostname.domainname
user=dbusername
password=dbpassword
Import into the database:
mysql -u username -p database_name < /file path/file_name.sql
Export from the database:
mysqldump -u username -p database_name > /file path/file_name.sql
After these commands, a prompt will ask for your MySQL password.
Similarly to vladkras's answer to How do import an SQL file using the command line in MySQL?.
Key differences for me:
The database has to exist first
No space between -p and the password
shell> mysql -u root -ppassword #note: no space between -p and password
mysql> CREATE DATABASE databasename;
mysql> using databasename;
mysql> source /path/to/backup.sql
I am running Fedora 26 with MariaDB.
I thought it could be useful for those who are using Mac OS X:
/Applications/xampp/xamppfiles/bin/mysql -u root -p database < database.sql
Replace xampp with mamp or other web servers.

How to import multiple database at once through SSH?

I am attempting to restore my mysql database to my website, and all of the tables in my database get dropped into individual files, so I am trying to figure out how can I restore all of the database .sql files through SSH with a single (or easy command) instead of restoring all 100 tables individually.
cat *.sql > data.sql
mysql -u <username> -p < data.sql
It depends on how you created the individual files - if they have all the instructions for recreating the tables (i.e., "Drop if exists...", "Create ...", and "Insert into ..."), then you can either concatenate them into mysql:
cat *.sql | mysql -u xxx -pxxx dbname
or write a script to do it
#!/bin/sh
mysql -u xxx -pxxx dbname < file001.sql
mysql -u xxx -pxxx dbname < file002.sql
The second choice lets you more easily control the order of files processed.
Finally, you might want to create your backups in a more convenient way - check out mysqldump for how to dump a database (or several!) into one file (basically, "mysqldump -u xxx -pxxx dbname > dbname.sql", but there are some helpful flags you might want to add).

Restoring a MySQL table back to the database

I have a trouble in restoring MySQL table back to the database from command line. Taking backup of a table is working with mysqldump.Taking backup and restoring of a database is also working properly. I have used:
mysql -uroot -p DatabaseName TableName < path\TableName.sql
Thanks in advance
Ah, I think I see the problem here.
Your backup script looks fine. tbl_name works correctly as the optional 2nd argument.
To restore, you should simply run
mysql -uroot -p DatabaseName < path\TableName.sql
Running man mysql would have shown you the correct arguments and options
mysql [options] db_name
As your backup script only contains one table, only that table will be restored into your database.
Taking backup
mysqldump -u -p mydatabase table1 > database_dump.sql
restoring from backup flie need not include table name
mysql -u -p mydatabase < database_dump.sql
Best way to restore your database:
open cmd at bin folder
login to mysql:
mysql -uroot -pyour_password
show databases;
use db_name;
now hit source and put the complete path from address bar where your sql file is stored and hit ;
for example :
source db_name.sql;
Copy your db.sql file to your Mysql Server if you are in a remote machine:
$rsync -Cravzp --progress db.sql user#192.168.10.1:/home/user
Now you can go to your remote server as:
$ssh -l user 192.168.10.1
In the Mysql Server you must to do this:
user#machine:~$mysql -h localhost -u root -p
Obs: The file db.sql must be in the same place (/home/user).
Now type this command in you Mysql Server:
mysql>'\'. db.sql + Enter. Obs: Remove all ' from this command to work

generating database with different name from mysqldump backup

The database "db" is backuped in backup.sql. Is there a way to restore database from script with different from "db" name?
thank you in advance!
Sure, when you import it you do this right:
mysql -uuser -ppassword databasename < mydump.sql
You can put anything you want where I wrote databasename - as long as that database actually exists :)
This depends on how you created your MySQL dB dump file
for example, if you do
mysqldump -h localhost -u user mydb -pXXX > mydb.sql
There won't be any CREATE DATABASE statements in your sql dump file. But I think you can only backup one database.
If you create your mysql dump file with --database or --all-databases option
for example
mysqldump -h localhost -u user --database mydb -pXXX > mydb.sql
mysqldump -h localhost -u user --all-databases -pXXX > alldb.sql
then you will see CREATE DATABASE statement in your mysql dump file. If you want a different dB name, you will need to change it before DB restore.
If the name of the database is include the SQL file, I didn't find any other way than modify the SQL file.
My favorite command to do it :
sed -i "s/\`old_db_name\`/\`new_db_name\`/g" my_sql_file.sql
Open up the .sql file and change the database name inside.
You can use a text editor, like Notepad or gedit.