I have a csv where each row starts with quotation marks and ends with them too. How do I ignore the quotations at the start and end of the row when loading the csv into a table?
LOAD DATA LOCAL INFILE '/path/data.csv'
INTO TABLE test1
FIELDS TERMINATED BY ';'
LINES TERMINATED BY '\"\n'
IGNORE 1 ROWS;
I have tried
OPTIONALLY ENCLOSED BY '"'
but this refers to each specific field and not the entire row.
As commented by Shadow and Barmar, the answer lies in the documentation :
If all the input lines have a common prefix that you want to ignore, you can use LINES STARTING BY 'prefix_string' to skip the prefix and anything before it. If a line does not include the prefix, the entire line is skipped. [...] The FIELDS TERMINATED BY, LINES STARTING BY, and LINES TERMINATED BY values can be more than one character.
Hence, use :
LOAD DATA LOCAL INFILE '/path/data.csv'
INTO TABLE test1
FIELDS TERMINATED BY ';'
LINES STARTING BY '"'
LINES TERMINATED BY '"\n'
IGNORE 1 ROWS;
Related
I'm a new to mysql, I try load csv file to mysql.
the csv like:
1,"a,b"
2,bc
3,d
the table like this:
create table test(ind varchar(10),var varchar(20));
when I load this csv file:
load data infile 'test.csv' into table test
fields terminated by ',' ;
I change this
the warning:
row 1 was truncated: it contained more data than there were input columns
I try this:
load data infile 'test.csv' into table test
fields terminated by ','
optionally enclosed by '"'
it doesn't work.
the common of "a,b" cause this error. but I don't know how to solve this question.
It sounds like maybe LOAD DATA isn't properly picking up on your line breaks. Try adding LINES TERMINATED BY ... to your call:
LOAD DATA INFILE 'test.csv' INTO TABLE test
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n' -- use '\n' if on Linux
(ind, var)
With the above call, MySQL should not view the comma inside the first quoted term "a,b" as being a field separator, but rather just part of the text of that column.
I'm having a seemingly basic problem with loading data to MySQL. I'm using:
LOAD DATA LOCAL INFILE 'C:/.../Contrato.txt'
INTO TABLE schema.contrato
FIELDS TERMINATED BY ';' ENCLOSED BY '|' LINES TERMINATED BY '/r/n' IGNORE 0 LINES;
Each line on the file looks like this, generated by another program:
|abc|;|cde|;|123|;|456|;|name|\r\n
When executing the load, everything seems to load properly, except the very last field. When I look at the table, the last field actually shows the '|' characters around the name. It's not the end of the world, but it's strange that it would do that. As of now, I'm fixing it by adding a ';' right before the \r\n characters.
Is this the way it's supposed to be done?? Why would I need to add the field terminator before the line terminator in order to delete the field enclosers?
I can duplicate the effect on a file with a single line in it, with multiple lines I only get a single entry which has the final column entry of "name| |abc" .
I changed '/r/n' to '\r\n' & the load worked correctly for files with a single entry & multiple entries & the surrounding | were correctly removed
LOAD DATA LOCAL INFILE 'C:/.../Contrato.txt'
INTO TABLE schema.contrato
FIELDS TERMINATED BY ';' ENCLOSED BY '|' LINES TERMINATED BY '\r\n' IGNORE 0 LINES;
The correct MySQL escape character is backslash not forwardslash which is not treated as a special character - so your original code was looking for the 4 character sequence "forwardslash r forwardslash n" to terminate the lines
i want to import a csv table into mysql with this command:
LOAD DATA LOCAL INFILE 'C:/Users/user/Desktop/SA01505.csv' INTO TABLE hks.orc CHARACTER SET 'utf8' COLUMNS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES (#var1,#var2,#var3,`TI 41-40`,`TI 41-41`,`TI 41-42`,`TI42-40`,`TI 42-41`,`TI 42-42`,`TI 42-43`,`TI 42-43.2`,`TI 42-44`,`TI 42-45`,`TI 42-46`,`TI 42-47`,`PI 42-71`,`PI 42-72`,`PI 42-73`,`PI 42-75`,`FI 42-90`,`TI 43-40`,`TI 43-41`,`TI 45-40`,`TI 45-41`,`TI 47-41`,`TI 47-42`,`TI 47-43`,`TI 47-44`,`LI 42-61`,`SI 44-81`,`QI 46-22`,`UV 41-10_Soll`,`UV 41-10_Ist`,`UV43-10_Soll`,`UV43-10_Ist`,`Tc2`,`Tc3`,`Tc4`)
SET Datum = STR_TO_DATE(CONCAT(#var2,#var3),'%Y/%m/%d%k:%i:%s');
This code works, but imports only every second line, although every line is terminated with "\n" (i checked this with an hex-editor)
I'd guess that the problem is here: OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"'. This is biting each other.
In the first line imported the last " of the enclosed column escapes the \n of the line terminator. Therefore always two lines are treated as one and you probably don't see the "second line" in the imported data because the data gets truncated.
Remove the ESCAPED BY '"' clause and see if it works. If it works but you need it to properly import your data, you'll have to work on your data.
this may just be the dumbest question I have asked yet. But I can not figure this out. I am trying to import a CSV into my mysql. But I can not get these areas correct:
Columns separated with:
Columns enclosed with:
Columns escaped with:
Lines terminated with:
Here is an example row in mycsv:
AA|Armed Forces Americas|US|United States
what would I place in each one?
If you have table that match the column configuration of the file what you do is:
LOAD DATA INFILE '<full file path>`
INTO TABLE <tbl_name>
FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n';
If some of the fields may contain | then these fields should be enclosed by " and then the last clause becomes:
FIELDS TERMINATED BY '|' ENCLOSED BY '"' TERMINATED BY '\n'
I have file in the following format:
`e00` `e01` `e02` `e03`
`e10` `e11` `e12` `e13
Trying to import the data with
LOAD DATA INFILE 'file' INTO TABLE 'foo' FIELDS TERMINATED BY ' ' ENCLOSED BY '`'
only seems to get the first 3 fields of each line. Is there a way to load the data without altering the file format?
Let's all jump in the way-back machine to answer a 5-year old question!
The fact that the last item is not being loaded is a big hint. According to the manual:
If LINES TERMINATED BY is an empty string and FIELDS TERMINATED BY is
nonempty, lines are also terminated with FIELDS TERMINATED BY.
So, it's looking for a trailing space at the end of the line.
You could add a space to the end of each line in your input file, or try LINES TERMINATED BY '\n'