I want to import over 1GB size sql file to MySQL database in localhost WAMP/phpmyadmin. But phpmyadmin UI doesn't allow to import such big file.
What are the possible ways to do that such as any SQL query to import .sql file ?
Thanks
I suspect you will be able to import 1 GB file through phpmyadmin But you can try by increasing the following value in php.ini and restart the wamp.
post_max_size=1280M
upload_max_filesize=1280M
max_execution_time = 300 //increase time as per your server requirement.
You can also try below command from command prompt, your path may be different as per your MySQL installation.
C:\wamp\bin\mysql\mysql5.5.24\bin\mysql.exe -u root -p db_name < C:\some_path\your_sql_file.sql
You should increase the max_allowed_packet of mysql in my.ini to avoid MySQL server gone away error, something like this
max_allowed_packet = 100M
Step 1:
Find the config.inc.php file located in the phpmyadmin directory. In my case it is located here:
C:\wamp\apps\phpmyadmin3.4.5\config.inc.php
Note: phymyadmin3.4.5 folder name is different in different version of wamp
Step 2:
Find the line with $cfg['UploadDir'] on it and update it to:
$cfg['UploadDir'] = 'upload';
Step 3:
Create a directory called ‘upload’ within the phpmyadmin directory.
C:\wamp\apps\phpmyadmin3.2.0.1\upload\
Step 4:
Copy and paste the large sql file into upload directory which you want importing to phymyadmin
Step 5:
Select sql file from drop down list from phymyadmin to import.
The values indicated by Ram Sharma might need to be changed in Wamp alias configuration files instead.
In <wamp_dir>/alias/phpmyadmin.conf, in the <Directory> section:
php_admin_value upload_max_filesize 1280M
php_admin_value post_max_size 1280M
php_admin_value max_execution_time 1800
Make sure to check the phpMyAdmin config file as well! On newer WAMP applications it is set to 128Mb by default. Even if you update php.ini to desired values you still need to update the phpmyadmin.conf!
Sample path: C:\wamp64\alias\phpmyadmin.conf
Or edit through your WAMP icon by: ->Apache -> Alias directories -> phpMyAdmin
I also faced the same problem and, strangely enough, changing the values in php.ini did not work for me.
But I found out one more solution that worked for me.
Click your Wamp server icon -> MySQL -> MySQL console
Once MySQL console is open. Enter your MySQL password and enter these commands:
use user_database_name
source path/to/your/sql/path/filename.sql
If you still have problems, watch this video.
What are the possible ways to do that such as any SQL query to import .sql file ?
Try this
mysql -u<user> -p<password> <database name> < /path/to/dump.sql
assuming dump.sql is your 1 GB dump file
A phpMyAdmin feature called UploadDir permits to upload your file via another mechanism, then importing it from the server's file system. See http://docs.phpmyadmin.net/en/latest/faq.html#i-cannot-upload-big-dump-files-memory-http-or-timeout-problems.
If you will try to load such a large file through phpmyadmin then you would need to change upload_file_size in php.ini to your requirements and then after uploading you will have to revert it back. What will happen? If you would like to load a 3GB file. You will have to change those parameters in php.ini again.
The best solution to solve this issue to open command prompt in windows.
Find path of wamp mysql directory.
Usually, it is C:/wamp64/bin/mysql/mysqlversion/bin/mysql.exe
Execute mysql -u root
You will be in mysql command prompt
Switch database with use command.
mysql> use database_name
mysql> source [file_path]
In case of Windows, here is the example.
mysql> source C:/sqls/sql1GB.sql
That's it. If you will have a database over 10GB or 1000GB. This method will still work for you.
Before importing just make sure you have max_allowed_pack value set some thing large else you will get an error: Error 2006 MySQL server gone away.
Then try the command: mysql -u root -p database_name < file.sql
You can do it in following ways;
You can go to control panel/cpanel and add host %
It means now the database server can be accessed from your local machine.
Now you can install and use MySQL Administrator or Navicat to import and export database with out using PHP-Myadmin, I used it several times to upload 200 MB to 500 MB of data with no issues
Use gzip, bzip2 compressions for exporting and importing. I am using PEA ZIP software (free) in Windows. Try to avoid Winrar and Winzip
Use MySQL Splitter that splits up the sql file into several parts. In my personal suggestion, Not recommended
Using PHP INI setting (dynamically change the max upload and max execution time) as already mentioned by other friends is fruitful but not always.
I suggest you'd definitely use mysql command prompt that would be faster option. because there are limitation in phpmyadmin UI and browsers itself for process request
following are steps to use mysql command line.
doesnt matter if you user xampp/wamp/mamp
find mysql directory in xampp/wamp/mamp in your system directory
search for bin folder path is system dir/(xampp/wamp)/mysql/bin
now open command prompt i'll refer to windows
please change directory in command line and point to path_to_mysql_bin
then user following command
"mysql-u root -p -h localhost" press enter system will ask for password press enter again
finally you're in
please use command "use Database_name" to point to specific database and you're good to go
if you want to upload database in to for ex. temp table
then follow these steps
use temp;
source path_to_sql_file_where_db_is_store_in_your_system.sql;
this will upload sql file database in temp db
if you didnt get any part from this then please pm me i'll definitely help you out.
Mitesh
In Your Case with Xammp it not work.
To Slove this problem In Xammp?
Just make database to Zip file and upload that's
it will work in xammp.
if the size is too large it will show you time out error
but submit the same zip folder again and after resubmitting will continue from position when it Force stop.
I've tried with the SQLyog and It has done the work.
Go to c:/wamp/apps/phpadmin3.5.2
Make a new subfolder called ‘upload’
Edit config.inc.php to find and update this line: $cfg[‘UploadDir’] = ‘upload’
Now when you import a database, you will give a drop-down list in web server upload directory with all the files in this directory. Chose the file you want and you are done.
Related
I am using phpmyadmin on my windows os. I have a database with one table which has 100M records with the size of 20GB. I want to export this table and have the table.sql file. Whenever I try to do this, the size of the exported file is 0 bytes. When I check the apache error log, the following would show up:
Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1066139648 bytes)
Any idea how to solve this problem?!
Thanks :)
I would suggest to try using the command line and the mysqldump.exe utility, as suggested here
If you have shared hosting and you are using Cpanel then they are providing you the option to backup your database in the following section.
Files => Backup => Download a MySQL Database Backup.
If you are on a shared hosting or you don't have access to shell, then use mysqldumper script; copy it to your server and start it in your browser under "yourDomain.com/path_to_mysqlumper/"
MySQLDumper is a PHP and Perl based tool for backing up MySQL
databases. You can easily dump your data into a backup file and - if
needed - restore it. It is especially suited for shared hosting
webspaces, where you don't have shell access.
If you have shell access to your host servers (if provided by your host since not all shared server hosters give this access) then you may use SSH access like in this tutorial using Putty that you install and configure then import or export your databases like in this third tutorial.
I try mysqldump for many hours but didnt work, until I started a superuser console.
First, start a superuser console
sudo su
Then, try the complete command
/opt/lampp/bin/mysqldump -u root -p [DATABASE NAME] > [PATH_FOR_BACKUPFILE]/[FILE_NAME].sql
I my case, it was something like /opt/lampp/bin/mysqldump -u root -p database > /home/user/backup.sql
MySQLDumper worked like a charm for me at my hosted website. I had to copy one database and "paste" it into a new database. In MySQLDumper, it isn't apparent right away how to do this, but the key is to create a new configuration file in MySQLDumper and that will allow you to copy/restore to different databases.
On the home screen in MySQLDumper, click Configuration, then Configuration Files. There is a text box at the top allowing you to create a new Configuration file. In there, put in the information for the second database you need (you created a connection to the first database when you install MySQLDumper). Save it. Then you can click Restore where you can select the dump of the first database and restore it in the second one.
This was a lifesaver. Thanks!
increase
max_post_size
variable in php.ini file. Then you will be able to download it.
I had a different issue when I was downloading from phpmyadmin in the middle like 180MB download stops with message - network error
So I used ssh connection which you can find in your cpanel sometimes they provide browser based terminal or sometime you have to access it using putty .
In terminal I go inside my public_html folder where all my files are stored . Followed by this command:
mysqldump -u [username] -p [database-you-want-to-dump] > [path-to-place-data-dump.sql]
This did the job in few minutes and saved a sql file in my public_html folder. Then I opened the folder in File manager and downloaded it from there.
You can also use FTP or you can download it directory by accessing by url.
Make sure you delete it after your download finishes.
We have a Magento site.
We have a large database of 1000MB for that site.
When we try to import database through phpmyadmin, it's not uploading the full database. Lot of tables are missing after uploading.
Qe are getting this error :
Script timeout passed, if you want to finish import, please resubmit
same file and import will resume
We have a GoDaddy cpanel and phpmyadmin.
Is there any way to upload a large database completely?
I am planning to change php.ini settings as mentioned in this question:
How to solve time out in phpmyadmin?
But I could not able to find php.ini file.
Have I need to edit the default Magento phi.ini.sample?
use sqlsplitter, It will created small chunk of you large sql file and then upload all these small files
Try like this in your Godaddy SSH,
Log on to your server.
At the command line, type su -. This gives you root access.
At the command line, type mysql -uroot -p
Enter your MySQL root password.
Select yourdatabase name for insert data by writing this command, USE YOURDATABASENAME
Then load source sql file as SOURCE D:\profile20\database.sql
Press enter for insert data.
This video link may help you.
I am using phpmyadmin on my windows os. I have a database with one table which has 100M records with the size of 20GB. I want to export this table and have the table.sql file. Whenever I try to do this, the size of the exported file is 0 bytes. When I check the apache error log, the following would show up:
Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1066139648 bytes)
Any idea how to solve this problem?!
Thanks :)
I would suggest to try using the command line and the mysqldump.exe utility, as suggested here
If you have shared hosting and you are using Cpanel then they are providing you the option to backup your database in the following section.
Files => Backup => Download a MySQL Database Backup.
If you are on a shared hosting or you don't have access to shell, then use mysqldumper script; copy it to your server and start it in your browser under "yourDomain.com/path_to_mysqlumper/"
MySQLDumper is a PHP and Perl based tool for backing up MySQL
databases. You can easily dump your data into a backup file and - if
needed - restore it. It is especially suited for shared hosting
webspaces, where you don't have shell access.
If you have shell access to your host servers (if provided by your host since not all shared server hosters give this access) then you may use SSH access like in this tutorial using Putty that you install and configure then import or export your databases like in this third tutorial.
I try mysqldump for many hours but didnt work, until I started a superuser console.
First, start a superuser console
sudo su
Then, try the complete command
/opt/lampp/bin/mysqldump -u root -p [DATABASE NAME] > [PATH_FOR_BACKUPFILE]/[FILE_NAME].sql
I my case, it was something like /opt/lampp/bin/mysqldump -u root -p database > /home/user/backup.sql
MySQLDumper worked like a charm for me at my hosted website. I had to copy one database and "paste" it into a new database. In MySQLDumper, it isn't apparent right away how to do this, but the key is to create a new configuration file in MySQLDumper and that will allow you to copy/restore to different databases.
On the home screen in MySQLDumper, click Configuration, then Configuration Files. There is a text box at the top allowing you to create a new Configuration file. In there, put in the information for the second database you need (you created a connection to the first database when you install MySQLDumper). Save it. Then you can click Restore where you can select the dump of the first database and restore it in the second one.
This was a lifesaver. Thanks!
increase
max_post_size
variable in php.ini file. Then you will be able to download it.
I had a different issue when I was downloading from phpmyadmin in the middle like 180MB download stops with message - network error
So I used ssh connection which you can find in your cpanel sometimes they provide browser based terminal or sometime you have to access it using putty .
In terminal I go inside my public_html folder where all my files are stored . Followed by this command:
mysqldump -u [username] -p [database-you-want-to-dump] > [path-to-place-data-dump.sql]
This did the job in few minutes and saved a sql file in my public_html folder. Then I opened the folder in File manager and downloaded it from there.
You can also use FTP or you can download it directory by accessing by url.
Make sure you delete it after your download finishes.
I have a 400 MB large SQL backup file. I'm trying to import that file into MySQL database using WAMP → import, but the import was unsuccessful due to many reasons such as upload file size is too large and so on. So, I've gone ahead and made a few changes to the configuration of PHP and MySQL settings under WAMP. Changes made are
Under WAMP->MySQL->my.ini
max_allowed_packet = 800M
read_buffer_size = 2014K
Under WAMP->PHP->PHP.ini
max_input_time = 20000
memory_limit = 128M
Is this the right way to import such a large SQL file using WAMP → import?
If so, did I make the right changes to the configuration files? What are the maximum values allowed for the above variable?
You can import large files this command line way:
mysql -h yourhostname -u username -p databasename < yoursqlfile.sql
Since you state (in a clarification comment to another person's answer) that you are using MySQL Workbench, you could try using the "sql script" option there. This will avoid the need to use the commandline (although I agree with the other answer that it's good to climb up on that horse and learn to ride).
In MySQL Workbench, go to File menu, then select "open script". This is probably going to take a long time and might even crash the app since it's not made to open things that are as big as what you describe.
The better way is to use the commandline. Make sure you have MySQL client installed on your machine. This means the actual MySQL (not Workbench GUI or PhpMyAdmin or anything like that). Here is a link describing the command-line tool. Once you have that downloaded and installed, open a terminal window on your machine, and you have two choices for slurping data from your file system (such as in a backup file) up into your target database. One is to use 'source' command and the other is to use the < redirection operator.
Option 1:
from the directory where your backup file is:
$mysql -u username -p -h hostname databasename < backupfile.sql
Option 2:
from the directory where your backup file is:
$mysql -u username -p -h hostname
[enter your password]
> use databasename;
> source backupfile.sql
Obviously, both of these options require that you have a backup file that is SQL.
Use this from mysql command window:
mysql> use db_name;
mysql> source backup-file.sql;
You can make the import command from any SQL query browser using the following command:
source C:\path\to\file\dump.sql
Run the above code in the query window ... and wait a while :)
This is more or less equal to the previously stated command line version. The main difference is that it is executed from within MySQL instead. For those using non-standard encodings, this is also safer than using the command line version because of less risk of encoding failures.
We have experienced the same issue when moving the sql server in-house.
A good solution that we ended up using is splitting the sql file into chunks. There are several ways to do that. Use
http://www.ozerov.de/bigdump/ seems good (but never used it)
http://www.rusiczki.net/2007/01/24/sql-dump-file-splitter/ used it and it was very useful to get structure out of the mess and you can take it from there.
Hope this helps :)
If you are using the source command on Windows remember to use f:/myfolder/mysubfolder/file.sql and not f:\myfolder\mysubfolder\file.sql
I believe the easiest way is to upload the file using MYSQL command line.
using the command from the terminal to access MySQL command line and run source
mysql --host=hostname -uuser -ppassword
source filename.sql
or directly from the terminal
mysql --host=hostname -uuser -ppassword < filename.sql
at the prompt.
Three things you have to do, if you are doing it locally:
In php.ini or php.cfg of your PHP installation
post_max_size=500M
upload_max_filesize=500M
memory_limit=900M
Or set other values. Restart Apache.
Or
Use the PHP big dump tool. It’s best ever I have seen. It’s free and open source.
Had a similar problem, but in Windows. I was trying to figure out how to open a large MySQL SQL file in Windows, and these are the steps I had to take:
Go to the download website (http://dev.mysql.com/downloads/).
Download the MySQL Community Server and install it (select the developer or full install, so it will install client and server tools).
Open the MySQL command-line client from the Start menu.
Enter your password used in install.
In the prompt, mysql>, enter:
CREATE DATABASE database_name;
USE database_name;
SOURCE myfile.sql
That should import your large file.
The question is a few months old but for other people looking --
A simpler way to import a large file is to make a sub directory 'upload' in your folder c:/wamp/apps/phpmyadmin3.5.2
and edit this line in the config.inc.php file in the same directory to include the folder name
$cfg['UploadDir'] = 'upload';
Then place the incoming .sql file in the folder /upload.
Working from inside the phpmyadmin console, go to the new database and import. You will now see an additional option to upload files from that folder. Chose the correct file and be a little patient. It works.
If you still get a time out error try adding $cfg['ExecTimeLimit'] = 0; to the same config.inc.php file.
I have had difficulty importing an .sql file where the user name was root and the password differed from my the root password on my new server. I simply took off the password before I exported the .sql file and the import worked smoothly.
On item 1.16 of phpMyAdmin they say:
1.16 I cannot upload big dump files (memory, HTTP or timeout problems).
Starting with version 2.7.0, the import engine has been re–written and these problems should not occur. If possible, upgrade your phpMyAdmin to the latest version to take advantage of the new import features.
The first things to check (or ask your host provider to check) are the values of upload_max_filesize, memory_limit and post_max_size in the php.ini configuration file. All of these three settings limit the maximum size of data that can be submitted and handled by PHP. One user also said that post_max_size and memory_limit need to be larger than upload_max_filesize. There exist several workarounds if your upload is too big or your hosting provider is unwilling to change the settings:
Look at the $cfg['UploadDir'] feature. This allows one to upload a file to the server via scp, ftp, or your favorite file transfer method. PhpMyAdmin is then able to import the files from the temporary directory. More information is available in the Configuration of this document.
Using a utility (such as BigDump) to split the files before uploading. We cannot support this or any third party applications, but are aware of users having success with it.
If you have shell (command line) access, use MySQL to import the files directly. You can do this by issuing the “source” command from within MySQL:
source filename.sql;
The simplest solution is MySQL Workbench. Just copy the .sql file text to the query window of MySQL Workbench and just execute it. All renaming things will done by it.
if you want to see the import progress. Install pv with brew and run this:
brew install pv
pv -p dump.sql | mysql -h localhost -u root -p db_name
my.ini
max_allowed_packet = 800M
read_buffer_size = 2014K
PHP.ini
max_input_time = 20000
memory_limit = 128M
post_max_size=128M
I have a database which has the size over 100 MB. It has the .sql.gz which means it is compressed. When I try to import it using PHPMyAdmin I get time out errors. I even tried partial imports ( Allow the interruption of an import in case the script detects it is close to the PHP timeout limit. (This might be good way to import large files, however it can break transactions.) ) which does not work for me. Given that I'm using CPanel & PHPMyAdmin to get the job done.
How can import this database?
You can use BigDump for this
Adjust the database configuration and charset in this file
Remove the old tables on the target database if your dump doesn't contain "DROP TABLE"
Create the working directory (e.g. dump) on your web server
Upload bigdump.php and your dump files (.sql, .gz) via FTP to the working directory
Run the bigdump.php from your browser via URL like
http://www.yourdomain.com/dump/bigdump.php
BigDump can start the next import session automatically if you enable the JavaScript
Wait for the script to finish, do not close the browser window
IMPORTANT: Remove bigdump.php and your dump files from the web server
If Timeout errors still occure you may need to adjust the $linepersession setting in this file. Read more
Very Simple and Effective Technique to import large mysql databases.
Using Command Line console
you can access command line using putty for server and execute following command
mysql -u 'database_user' -p'database_password' database_name < sql_file_name_with_full_path
Using Cron Job [if console is not available]
add a cron job in cpanel with following command
mysql -u 'database_user' -p'database_password' database_name < sql_file_name_with_full_path
Do not forget to delete cron job after importing database. Also delete sql file from server.
for both techniques commands are same
Example
mysql -u 'test_user' -p'123456' test_db < /home1/test/public_html/db.sql
or if you do not have password
mysql -u 'test_user' test_db < /home1/test/public_html/db.sql
In my opinion and experience with hepsiawebhosting.com server, it's better and faster to set those parameters under WHM settings called "Tweak setting". Under Tweak settings you can modify maximum upload and maximum post, and you can could, for example, change upload to 120 MiB and post to 125 MiB.
Also, in PHP settings under WHM, you can change the default socket and SQL time out. You need to increase those settings.
You can also unzip the database and try to upload it as a simple .sql file.
I hope this will help you.
why this is not published, what is missing here?