This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Tool for importing CSV files into MySQL database?
A guy at work gave me a .csv file with thousands of records in it. There's about 5 columns (out of 20) that I would love to get inserted into a mysql database.
Any idea how I might go about that?
Using LOAD DATA INFILE. The example in the docs for CSV is:
LOAD DATA INFILE 'data.txt' INTO TABLE tbl_name
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES;
You should remove the IGNORE 1 LINES clause if there is no header row in the CSV data.
Also, note that the order of the data in the file should match the order of the columns in the table. If they do not, you will need to specify the order like this:
LOAD DATA INFILE 'data.txt' INTO TABLE tbl_name
(column1, column2, ...)
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES;
use the LOAD DATA or BULK INSERT command
Related
I have a single csv file with data that I am trying to load into MySQL using LOAD DATA. My problem is the csv file has data that belongs in 3 separate tables. Line 1 goes into table1, line 2-3 goes into table 2, and all remaining lines go into table 3. I can use IGNORE 3 LINES for the 3rd table, but not sure how to ignore lines after my required data for the first 2 tables.
Here is my LOAD DATA so far:
LOAD DATA INFILE 'data.csv' INTO TABLE tbl_name1
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n';
LOAD DATA INFILE 'data.csv' INTO TABLE tbl_name2
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES;
LOAD DATA INFILE 'data.csv' INTO TABLE tbl_name3
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 3 LINES;
I am trying to import a csv file to mysql table, But I need to remove First two characters on particular column before importing to mysql.
This is my statment :
string strLoadData = "LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets FIELDS terminated by ',' ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (SiteId,DateTime,Serial,DeviceId,AgentAID,VehicleRegistration,CarPark,SpaceNumber,GpsAddress,VehicleType,VehicleMake,VehicleModel,VehicleColour,IssueReasonCode,IssueReason,NoticeLocation,Points,Notes)";
Column IssueReasoncode' has data like 'LU12' , But i need to remove the first 2 characters it should have only integers on it and not alpha numeric .
I need to remove 'LU' from that column.
Is it possible to write like this on left(IssueReasonCode +' '2). This column is varchar(45) and cant be changed now because of large data on it.
Thanks
LOAD DATA INFILE has the ability to perform a function on the data for each column as you read it in (q.v. here). In your case, if you wanted to remove the first two characters from the IssueReasonCode column, you could use:
RIGHT(IssueReasonCode, CHAR_LENGTH(IssueReasonCode) - 2)
to remove the first two characters. You specify such column mappings at the end of the LOAD DATA statement using SET. Your statement should look something like the following:
LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets
FIELDS terminated by ','
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(SiteId, DateTime, Serial, DeviceId, AgentAID, VehicleRegistration, CarPark, SpaceNumber,
GpsAddress, VehicleType, VehicleMake, VehicleModel, VehicleColour, IssueReasonCode,
IssueReason, NoticeLocation, Points, Notes)
SET IssueReasonCode = RIGHT(IssueReasonCode, CHAR_LENGTH(IssueReasonCode) - 2)
Referencing this and quoting this example , you can try the below to see if it works
User variables in the SET clause can be used in several ways. The
following example uses the first input column directly for the value
of t1.column1, and assigns the second input column to a user variable
that is subjected to a division operation before being used for the
value of t1.column2:
LOAD DATA INFILE 'file.txt' INTO TABLE t1 (column1, #var1) SET
column2 = #var1/100;
string strLoadData = "LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets FIELDS terminated by ',' ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (SiteId,DateTime,Serial,DeviceId,AgentAID,VehicleRegistration,CarPark,SpaceNumber,GpsAddress,VehicleType,VehicleMake,VehicleModel,VehicleColour,#IRC,IssueReason,NoticeLocation,Points,Notes) SET IssueReasonCode = substr(#IRC,2) ;";
I am importing my data from a csv file like this:
LOAD DATA LOCAL INFILE "c:\myfile.csv" INTO TABLE historic FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
This ALMOST works perfect. My table has one different row, the first: An auto incremented id column. Otherwise the csv file and the MySQL table match perfectly. When I run the command above, it puts the first column from the csv into the id field. It want it to actually go into the second.
Is there a way I can modify the above statement to either specify the columns or just offset it by 1? (skipping the first column on import).
You can optionally name the columns to be populated by LOAD DATA, and simply omit your id column:
LOAD DATA LOCAL INFILE "c:\myfile.csv" INTO TABLE historic
FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n'
(column2, colum3, column4, column5, ...)
You can load your data specifing the order columns that you're going to use into your table:
LOAD DATA LOCAL FILE '/tmp/test.txt' INTO TABLE historic
(name_field1, name_field2, name_field3...);
Or if you prefer you can load it first to a temporary table and use a select into statement to laod it into your final table (it's slower).
I would take data from csv file into my table. I'm using command:
LOAD DATA INFILE 'C:\\...\\file.csv' INTO TABLE table FIELDS TERMINATED BY ',' IGNORE 1 LINES;
The problem is that I have data as below:
Col1,Col2,Col3,Col4
1,name11,name21,name31
2,"name21, aaa.",name22,name23
First row is ok, but second not, because "name21, aaa." is reading as two columns so I don't have name23 in table.
Any idea how can I resolve this problem?
You need to understand LOAD DATA INFILE syntax.
Check this link MySQL - LOAD DATA INFILE.
LOAD DATA LOCAL INFILE 'C:\\...\\file.csv' INTO TABLE table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(col1, col2, col3, col4, col5...)
I have a CSV file which contains 10 columns. I want to select only some columns from that file and load them into a MySQL database using the LOAD DATA INFILE command.
Load data into a table in MySQL and specify columns:
LOAD DATA LOCAL INFILE 'file.csv' INTO TABLE t1
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
(#col1,#col2,#col3,#col4) set name=#col4,id=#col2 ;
#col1,2,3,4 are variables to hold the csv file columns (assume 4 ) name,id are table columns.
Specify the name of columns in the CSV in the load data infile statement.
The code is like this:
LOAD DATA INFILE '/path/filename.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
(column_name3, column_name5);
Here you go with adding data to only two columns(you can choose them with the name of the column) to the table.
The only thing you have to take care is that you have a CSV file(filename.csv) with two values per line(row). Otherwise please mention. I have a different solution.
Thank you.
LOAD DATA INFILE 'file.csv'
INTO TABLE t1
(column1, #dummy, column2, #dummy, column3, ...)
FIELDS TERMINATED BY ',' ENCLOSED BY '"' ESCAPED BY '"'
LINES TERMINATED BY '\r\n';
Just replace the column1, column2, etc.. with your column names, and put #dummy anwhere there's a column in the CSV you want to ignore.
Full details here.
Example:
contents of the ae.csv file:
"Date, xpto 14"
"code","number","year","C"
"blab","15885","2016","Y"
"aeea","15883","1982","E"
"xpto","15884","1986","B"
"jrgg","15885","1400","A"
CREATE TABLE Tabletmp (
rec VARCHAR(9)
);
For put only column 3:
LOAD DATA INFILE '/local/ae.csv'
INTO TABLE Tabletmp
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 2 LINES
(#col1, #col2, #col3, #col4, #col5)
set rec = #col3;
select * from Tabletmp;
2016
1982
1986
1400
if you have number of columns in your database table more than number of columns in your csv you can proceed like this:
LOAD DATA LOCAL INFILE 'pathOfFile.csv'
INTO TABLE youTable
CHARACTER SET latin1 FIELDS TERMINATED BY ';' #you can use ',' if you have comma separated
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\r\n'
(yourcolumn,yourcolumn2,yourcolumn3,yourcolumn4,...);
For those who have the following error:
Error Code: 1290. The MySQL server is running with the
--secure-file-priv option so it cannot execute this statement
You can simply run this command to see which folder can load files from:
SHOW VARIABLES LIKE "secure_file_priv";
After that, you have to copy the files in that folder and run the query with LOAD DATA LOCAL INFILE instead of LOAD DATA INFILE.