I have a database in mysql workbench, and i want to load data from .csv files. Now i'm doing with the option of 'Table Data Import Wizard' but takes to many time. My .csv files have million of rows, but each one takes about 12 hours to load. And i have about 100 files to load. My version of MySQL is the 8.0.
There is any way to load the data files fastest? Thanks in advice
You can try something like that:
LOAD DATA INFILE 'c:/myfile.csv'
INTO TABLE table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
Be careful with the values you give to FIELDS TERMINATED BY, ENCLOSED BY and LINES TERMINATED BY. Only use IGNORE 1 ROWS if you have a header line, containing the field names, for example.
And if you need to store the data on a remote server, you can use
LOAD DATA LOCAL INFILE 'c:/myfile.csv'
INTO TABLE table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
You can find more infos here.
I have a total of 6077 data. However after running this command
LOAD DATA INFILE '..admin staff.csv' INTO TABLE `adstaff` FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n'
The total data read is 1 mil++. But when I check my database it only contains 4033 real data and any other row after were null.
I am utterly confuse with another 2000++ data whereabouts because it does not came out when I do manual searching. I hope anyone can help me out thank you
Ive tried to follow examples of how to import a csv infile with no luck. I have the following CSV-
And from what I understand, I need to set up a table in Workbench, with column names that match the CSV columns. I've done this:
Using this code-
LOAD DATA LOCAL INFILE 'C:/Users/Sam/Desktop/Eviction_Notices_2012to2017_AddressthruZip.csv' INTO TABLE evictions.test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(Address, City, State, Zip);
I've had varying degrees of success. Right now, it only imports first 99 rows even though there is no limit specified.
Also, is there a way to copy your column names from excel easily into the alter table window, rather than creating each column name manually?
Am I doing something wrong here?
What I did is to type this query:
SELECT * FROM report;
then click the Import Records from an External File in MySQL Workbench. It worked and it was imported in my database. The problem is that the header is also imported.
What should I do to import everything except the header?
Ive never really used MySQL workbench, but the best way to import a .csv is with LOAD DATA INFILE.
Example:
LOAD DATA INFILE '{$file_path}' INTO TABLE {$table} FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES
Note that your LINES TERMINATED BY may be different based on the way your .csv is formatted. It could also be \n' or less commonly '\r'.
For smaller csv files, the following has always worked for me...
LOAD DATA LOCAL INFILE 'C:/store_emails2.csv' INTO TABLE
mytable FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
Now, I am trying to do this with a file that has 100K rows. Mysql only takes about 50K rows, though. The odd part is that it takes random rows, not just the first 50K. I have no idea what the problem could be, as the file is formatted correctly and I've imported data this way many times. Any ideas as to what may be the problem?
EDIT: including what part of the csv looks like...
CUST_NUM,LAST_NAME,FIRST_NAME,MIDDLE_INIT,PREFIX,SUFFIX,ADDRESS_1,ADDRESS_2,CITY,STATE,ZIP,HOME_TEL,BUS,MAILING,MOD_DATE,SIGN_UP_DATE,BIRTHDAY_DATE,CASHIER,STR,EMAIL
1234556767,name,name,,,,,,,,,9999999999,,Y,6007,607,0101,341,71,email