Consider a table to store SSN and DOB.
CREATE TABLE fbi
(
ssn BIGINT,
dob DATE
)
Data was loaded into the table:
LOAD DATA LOCAL INFILE C:\test.csv
INTO TABLE fbi
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n';
SELECT * FROM fbi;
It is showing null values for DOB. I don't understand the error.
"ssn","dob"
5,"1952-11-15"
6,"1973-12-23"
6,"1951-12-23"
1,"1962-03-21"
It most likely has to do with the date formatting in your csv file. MySQL likes dates in the format yyyy-mm-dd. e.g. 2010-10-09. You might be able to get more information by issuing the following command in the MySQL command console immediately after your import:
show warnings;
UPDATE:
I see that your date field is quoted. If you have a quoted date, you'll need to tell MySQL that by adding OPTIONALLY ENCLOSED BY '"' to your import command (see MySQL manual for LOAD DATA INFILE). Try this:
LOAD DATA LOCAL INFILE C:\test.csv
INTO TABLE fbi
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n';
BTW: I hope those SSN's you've posted are fake or mangled in some unrecoverable way.
Related
Below is the code that I use to import CSV file to MySQL database. It works well to divide all the field and its record.
LOAD DATA INFILE 'file.csv'
INTO TABLE customer FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n'
(
ID, name, salary, address, status
);
However, when there is a VARCHAR or TEXT field which consist of comma (','), it works improperly. It is because I use FIELDS TERMINATED BY ',' that used to separate each field record.
So, for example, if a customer with salary 50,000 (double), it split the field normally. But, if the customer address is Java Road 15, Hong Kong (varchar/text), Java Road 15 will be saved in address field, while the Hong Kong will be saved to status field. This basically remove any record inside the status field. Any clue for this problem? Thanks in advance.
Are the fields enclosed by double quotes or something else? If so, you can add the "ENCLOSED BY" in your query.
FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\r\n';
"Enclosed by" specifies the character to identify the start and end of a field. In your case, field is enclosed by a double quote such as "Java Road 15, Hong Kong". It helps MYSQL to extract the field correctly even if there is a field delimiter in the field.
MYSQL manual: https://dev.mysql.com/doc/refman/5.7/en/load-data.html
LOAD DATA INFILE 'file.csv'
INTO TABLE customer FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
(
ID, name, salary, address, status
);
Try this one.
If anyone happens to stumble upon this answer, I wanted to share b/c this took me way too long to figure out - using mac, datagrip, mysql 5.7+, survey response questions:
DROP TABLE IF EXISTS surveyQuestion_ID;
#create table
CREATE TABLE surveyQuestion_ID
(
surveyQuestion_ID INT(11) NOT NULL,
surveyDescription TEXT,
surveyResponse VARCHAR(25) DEFAULT NULL,
PRIMARY KEY (surveyQuestion_ID)
);
#load query
LOAD DATA LOCAL INFILE 'file_location/file.csv' INTO TABLE surveyQuestion_ID
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(surveyQuestion_ID, surveyDescription, surveyResponse);
Hope this helps.
I am trying to import a csv file to mysql table, But I need to remove First two characters on particular column before importing to mysql.
This is my statment :
string strLoadData = "LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets FIELDS terminated by ',' ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (SiteId,DateTime,Serial,DeviceId,AgentAID,VehicleRegistration,CarPark,SpaceNumber,GpsAddress,VehicleType,VehicleMake,VehicleModel,VehicleColour,IssueReasonCode,IssueReason,NoticeLocation,Points,Notes)";
Column IssueReasoncode' has data like 'LU12' , But i need to remove the first 2 characters it should have only integers on it and not alpha numeric .
I need to remove 'LU' from that column.
Is it possible to write like this on left(IssueReasonCode +' '2). This column is varchar(45) and cant be changed now because of large data on it.
Thanks
LOAD DATA INFILE has the ability to perform a function on the data for each column as you read it in (q.v. here). In your case, if you wanted to remove the first two characters from the IssueReasonCode column, you could use:
RIGHT(IssueReasonCode, CHAR_LENGTH(IssueReasonCode) - 2)
to remove the first two characters. You specify such column mappings at the end of the LOAD DATA statement using SET. Your statement should look something like the following:
LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets
FIELDS terminated by ','
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(SiteId, DateTime, Serial, DeviceId, AgentAID, VehicleRegistration, CarPark, SpaceNumber,
GpsAddress, VehicleType, VehicleMake, VehicleModel, VehicleColour, IssueReasonCode,
IssueReason, NoticeLocation, Points, Notes)
SET IssueReasonCode = RIGHT(IssueReasonCode, CHAR_LENGTH(IssueReasonCode) - 2)
Referencing this and quoting this example , you can try the below to see if it works
User variables in the SET clause can be used in several ways. The
following example uses the first input column directly for the value
of t1.column1, and assigns the second input column to a user variable
that is subjected to a division operation before being used for the
value of t1.column2:
LOAD DATA INFILE 'file.txt' INTO TABLE t1 (column1, #var1) SET
column2 = #var1/100;
string strLoadData = "LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets FIELDS terminated by ',' ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (SiteId,DateTime,Serial,DeviceId,AgentAID,VehicleRegistration,CarPark,SpaceNumber,GpsAddress,VehicleType,VehicleMake,VehicleModel,VehicleColour,#IRC,IssueReason,NoticeLocation,Points,Notes) SET IssueReasonCode = substr(#IRC,2) ;";
I've got a csv file I made with a bunch of info, but I cant get it to import properly...
Ive got these values in info.csv: id firstname lastname address state gpa credits
following a video, I used this:
LOAD DATA LOCAL INFILE '/IT101/info.csv' INTO TABLE 'student' FIELDS TERMINATED BY ','
ENCLOSED BY '"' ESCAPED BY '\' LINES TERMINATED BY '\n';
I get back:
PAGER set to stdout
and the values aren't there. What am I doing wrong?
EDIT: By the way, I alaready have two rows in the table from using insert into, just doing it 30 more times seemed like a waste of time
You can try...
LOAD DATA INFILE '/IT101/info.csv'
INTO TABLE students
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
I want to insert data into mysql table from csv file. Import data from region_codes.csv file. In region_codes.csv file having 3 columns in 3rd columns it had , separated data, include those commas how to insert in mysql.
DROP TABLE IF EXISTS `region_codes`;
CREATE TABLE `region_codes` (
`country_code` CHAR(2) NULL,
`region_no` varchar(5) NOT NULL,
`region` VARCHAR(45) NULL,
INDEX `idx_country_code` (`country_code`)
) COLLATE='utf8_bin' ENGINE = MyISAM;
Using LOAD DATA LOCAL INFILE I import the data but only 1000 rows are imported outof 4066 rows.
LOAD DATA LOCAL INFILE 'C:/region_codes.csv' INTO TABLE `region_codes` FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
How to insert huge amount of data into mysql region_codes table from csv file.
Screenshot:
You can try below syntax if it works for you otherwise provide csv data:
LOAD DATA LOCAL INFILE 'C:/region_codes.csv' INTO TABLE `region_codes` FIELDS ESCAPED BY '\\' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n';
If above syntax does not work then export data by below command again and import by below given command.
select * into outfile 'C:/region_codes.csv' fields terminated by ',' optionally enclosed by '"' lines terminated by '\n' from `region_codes`;
Now use below command (to ignore column heading line)
LOAD DATA LOCAL INFILE 'C:/region_codes.csv' INTO TABLE `region_codes` FIELDS ESCAPED BY '\\' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES;
Note: If data is prepared manually then need to correct it manually.
If still not work then attach your csv data to check exact problem.
You can use mysqlimport tool this way:
mysqlimport --ignore-lines=1 --fields-terminated-by=,
--columns='ID,Name,Phone,Address' --local -u root -p
Database /path/to/csvfile/TableName.csv
Here is a full explanation: http://chriseiffel.com/everything-linux/how-to-import-a-large-csv-file-to-mysql/
I am using Load data in file query to insert csv into table. I have to format a date column inside the csv,
LOAD DATA INFILE '/invoices/invoice1381301986.csv' INTO TABLE invoice_table FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' (`code`,#var1) set datefield=STR_TO_DATE(#var1, '%m/%d/%Y');
I can format the date using the above query.
But the problem is, I have different formats for the csv date column. Possible formats are, "m/d/Y","m-d-Y", "m/d/y", "m-d-y", "Y-m-d", "Y/m/d".
so my query should be according to date format from the csv, so that I can modify my queries like,
datefield=STR_TO_DATE(#var1, '%m/%d/%Y')
OR
datefield=STR_TO_DATE(#var1, '%m-%d-%Y')..
How can I read the in which format the csv date field is?
you need to use a CASE :
LOAD DATA INFILE '/invoices/invoice1381301986.csv'
INTO TABLE invoice_table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(`code`,#var1)
SET datefield= (
CASE
WHEN #var1 REGEXP '[0-9]{2}/[0-9]{2}/[0-9]{4}' THEN STR_TO_DATE(#var1,'%m/%d/%Y')
...
END
)