Hi I have a csv dataset like
ukwn,34,2018-10-01,"757,271"
ukwn,3,2018-10-01,"7,342"
"hi",23,2018-10-01,"3,483,887"
i want to insert it in the database so i made the code:
LOAD DATA LOCAL INFILE 'data.csv' INTO TABLE app_spend_metric
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(col1,col2,col3,col4)
But i fails to insert the col4 (4th row) as there is ',' inside a "" like "7,345"
I tried then,
LOAD DATA LOCAL INFILE 'data.csv' INTO TABLE app_spend_metric
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(col1,col2,col3,col4)
But this time it enters the partial data in col4 like out of "7,344" it only enters "7
If col4 is numeric (eg, INT), then the problem is as follows:
Parse the line to get the string "7,344"
Strip the enclosing ": 7,344
Store the string into the INT column. This requires converting that string to a number.
Conversion stops at the first non-numeric character, namely the comma.
Result: The col4 is set to 7, and ,344 is tossed.
MySQL cannot deal with "thousands-separators" in numbers. But you could strip them:
LOAD ...
(col1, col2, col3, #num)
SET col4 = REPLACE(#num, ',', '')
Related
I am trying to load up data to table from a csv file that has comma as decimal seperator.
To fix the problem I have used the following code, but the replace formula does not seem to work ...
clickthroughrate and sales are the column names in question
LOAD DATA LOCAL INFILE 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/testowy import.csv' INTO TABLE sponsored_products
FIELDS TERMINATED BY '\t'
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
SET
clickthroughrate = REPLACE(clickthroughrate, ',', '.'),
sales = REPLACE(sales, ',', '.')
;
I am trying to import a csv file to mysql table, But I need to remove First two characters on particular column before importing to mysql.
This is my statment :
string strLoadData = "LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets FIELDS terminated by ',' ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (SiteId,DateTime,Serial,DeviceId,AgentAID,VehicleRegistration,CarPark,SpaceNumber,GpsAddress,VehicleType,VehicleMake,VehicleModel,VehicleColour,IssueReasonCode,IssueReason,NoticeLocation,Points,Notes)";
Column IssueReasoncode' has data like 'LU12' , But i need to remove the first 2 characters it should have only integers on it and not alpha numeric .
I need to remove 'LU' from that column.
Is it possible to write like this on left(IssueReasonCode +' '2). This column is varchar(45) and cant be changed now because of large data on it.
Thanks
LOAD DATA INFILE has the ability to perform a function on the data for each column as you read it in (q.v. here). In your case, if you wanted to remove the first two characters from the IssueReasonCode column, you could use:
RIGHT(IssueReasonCode, CHAR_LENGTH(IssueReasonCode) - 2)
to remove the first two characters. You specify such column mappings at the end of the LOAD DATA statement using SET. Your statement should look something like the following:
LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets
FIELDS terminated by ','
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(SiteId, DateTime, Serial, DeviceId, AgentAID, VehicleRegistration, CarPark, SpaceNumber,
GpsAddress, VehicleType, VehicleMake, VehicleModel, VehicleColour, IssueReasonCode,
IssueReason, NoticeLocation, Points, Notes)
SET IssueReasonCode = RIGHT(IssueReasonCode, CHAR_LENGTH(IssueReasonCode) - 2)
Referencing this and quoting this example , you can try the below to see if it works
User variables in the SET clause can be used in several ways. The
following example uses the first input column directly for the value
of t1.column1, and assigns the second input column to a user variable
that is subjected to a division operation before being used for the
value of t1.column2:
LOAD DATA INFILE 'file.txt' INTO TABLE t1 (column1, #var1) SET
column2 = #var1/100;
string strLoadData = "LOAD DATA LOCAL INFILE 'E:/park/Export.csv' INTO TABLE tickets FIELDS terminated by ',' ENCLOSED BY '\"' lines terminated by '\n' IGNORE 1 LINES (SiteId,DateTime,Serial,DeviceId,AgentAID,VehicleRegistration,CarPark,SpaceNumber,GpsAddress,VehicleType,VehicleMake,VehicleModel,VehicleColour,#IRC,IssueReason,NoticeLocation,Points,Notes) SET IssueReasonCode = substr(#IRC,2) ;";
I am loading csv files in MySQL tables using this instruction :
LOAD DATA INFILE 'myfile.csv'
INTO TABLE MYTABLE
FIELDS TERMINATED BY ";"
LINES TERMINATED BY "\r\n";
Wanting to put null instead of empty fields I wrote this :
LOAD DATA INFILE 'myfile.csv'
INTO TABLE MYTABLE
FIELDS TERMINATED BY ";"
LINES TERMINATED BY "\r\n"
(field1, #field2, field3)
SET
field2 = NULLIF(#field2, '');
Works good but as I use NULLIF, even blank fields are put to NULL.
How can I say that just empty fields go to NULL but blanks fields remain the same?
I have the following CSV File:
AFRICA,Zimbabwe,7,Telecel Zimbabwe,1,0,1,0,0,1
AFRICA,Zambia,7,Celtel Zambia Plc,1,0,1,0,0,1
I am using the following query but for some reason it's giving an error:
LOAD DATA LOCAL INFILE 'pathtofile' INTO TABLE databasename.tablename FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
It is giving a data truncated warning and not importing anything
First of all you need to ensure that the receiving data fields are in the correct order and are of the correct type and size.
Then you specify ENCLOSED BY '"', but your example data is not actually enclosed...
Check The Query in database :
LOAD DATA LOCAL INFILE 'abc.csv' INTO TABLE abc
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(col1, col2, col3, col4, col5...)
I have a CSV file which contains 10 columns. I want to select only some columns from that file and load them into a MySQL database using the LOAD DATA INFILE command.
Load data into a table in MySQL and specify columns:
LOAD DATA LOCAL INFILE 'file.csv' INTO TABLE t1
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
(#col1,#col2,#col3,#col4) set name=#col4,id=#col2 ;
#col1,2,3,4 are variables to hold the csv file columns (assume 4 ) name,id are table columns.
Specify the name of columns in the CSV in the load data infile statement.
The code is like this:
LOAD DATA INFILE '/path/filename.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
(column_name3, column_name5);
Here you go with adding data to only two columns(you can choose them with the name of the column) to the table.
The only thing you have to take care is that you have a CSV file(filename.csv) with two values per line(row). Otherwise please mention. I have a different solution.
Thank you.
LOAD DATA INFILE 'file.csv'
INTO TABLE t1
(column1, #dummy, column2, #dummy, column3, ...)
FIELDS TERMINATED BY ',' ENCLOSED BY '"' ESCAPED BY '"'
LINES TERMINATED BY '\r\n';
Just replace the column1, column2, etc.. with your column names, and put #dummy anwhere there's a column in the CSV you want to ignore.
Full details here.
Example:
contents of the ae.csv file:
"Date, xpto 14"
"code","number","year","C"
"blab","15885","2016","Y"
"aeea","15883","1982","E"
"xpto","15884","1986","B"
"jrgg","15885","1400","A"
CREATE TABLE Tabletmp (
rec VARCHAR(9)
);
For put only column 3:
LOAD DATA INFILE '/local/ae.csv'
INTO TABLE Tabletmp
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 2 LINES
(#col1, #col2, #col3, #col4, #col5)
set rec = #col3;
select * from Tabletmp;
2016
1982
1986
1400
if you have number of columns in your database table more than number of columns in your csv you can proceed like this:
LOAD DATA LOCAL INFILE 'pathOfFile.csv'
INTO TABLE youTable
CHARACTER SET latin1 FIELDS TERMINATED BY ';' #you can use ',' if you have comma separated
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\r\n'
(yourcolumn,yourcolumn2,yourcolumn3,yourcolumn4,...);
For those who have the following error:
Error Code: 1290. The MySQL server is running with the
--secure-file-priv option so it cannot execute this statement
You can simply run this command to see which folder can load files from:
SHOW VARIABLES LIKE "secure_file_priv";
After that, you have to copy the files in that folder and run the query with LOAD DATA LOCAL INFILE instead of LOAD DATA INFILE.