I encountered an error when trying to upload a CSV file to my database via the command-line interface.
The CSV file is structured as follows:
Id;price;shorttext;text
2020;24;foo;just a longtext for the product
2019;10;bar;"sometimes there are ; in the column"
2018;45;foobar;next longtext for the product
However, in the last column, I have the problem that the character ';' can occur in the text itself, so this column is sometimes specified with "..." as shown in the code above.
During the import I tried the following statement:
LOAD DATA INFILE 'path/to/file.csv' INTO TABLE product Fields OPTIONALLY ENCLOSED BY '"' Terminated by ';' Lines terminated by '\n' IGNORE 1 ROWS;
But then I get an ERROR 1406: Data too long for column "text" at row 2. I think there is a problem because of the ';' in the text in the last column.
Do anyone know a solution, how to handle this optionally '"' in the last column?
Thank you very much :-)
Related
You can see my raw data above. I'm trying to import this data on the table I've created on MySQL. Here's the code I should be using:
LOAD DATA LOCAL INFILE 'mytbl.txt' INTO TABLE mytbl
-> FIELDS TERMINATED BY ',' ENCLOSED BY '"'
-> LINES TERMINATED BY '\r\n';
I can't understand which character is "fields terminated by", which one is "enclosed by" and which one is for "lines terminated by".
Can you read your .csv file into Excel or LibreOffice Calc (or any spreadsheet program) correctly? I guess you probably can. That means it is formatted correctly.
.csv files contain one line of text for each row of data in a table. These LOAD INFILE directives tell MySQL how to find the rows and columns in the .csv file.
FIELDS TERMINATED BY ',' means each column of data ends with a comma. Notice your first line of data :
De Ruijterkade,,123400000001234,,1,105...
The first column is the street name. The second is empty, the third is 1, the fourth 105 et cetera.
ENCLOSED BY '"' means columns of data which themselves contain a comma (a field terminator) must be enclosed in " characters. For example, if your street name had the value De Ruijterkade, Kade your file would contain
"De Ruijterkade, Kade",,123400000001234,,1,105...
Finally LINES TERMINATED BY '\r\n' means each line in your file (row in your table) ends with a Windows-style <return><linefeed> character pair.
Akina correctly pointed out the documentation. https://dev.mysql.com/doc/refman/8.0/en/load-data.html#load-data-field-line-handling
I have a csv that looks like this:
20050104,000020.SZ,11432.90832
20050104,000013.SZ,
20050104,000008.SZ,
20050104,000014.SZ,1817.418592
20050104,000015.SZ,
20050104,000016.SZ,-544.482028
` 20050104,000017.SZ,10034.32188
I've read theload-data documents of MySQL and right now I'm tring to load this file into MySQL using following codes:
LOAD DATA INFILE 'test.csv' INTO TABLE test1
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n';
but It gives me the error:
ERROR 1265 (01000): Data truncated for column 'value' at row 2
I tried to modified my code by changing \n into \r\n, that is:
LINES TERMINATED BY '\r\n'(date, code,value);
but it still gives me the same error.
I knew it was because of the empty fields that gives me an error, but I don't know how to avoid that. Can you help out?
I'm having a seemingly basic problem with loading data to MySQL. I'm using:
LOAD DATA LOCAL INFILE 'C:/.../Contrato.txt'
INTO TABLE schema.contrato
FIELDS TERMINATED BY ';' ENCLOSED BY '|' LINES TERMINATED BY '/r/n' IGNORE 0 LINES;
Each line on the file looks like this, generated by another program:
|abc|;|cde|;|123|;|456|;|name|\r\n
When executing the load, everything seems to load properly, except the very last field. When I look at the table, the last field actually shows the '|' characters around the name. It's not the end of the world, but it's strange that it would do that. As of now, I'm fixing it by adding a ';' right before the \r\n characters.
Is this the way it's supposed to be done?? Why would I need to add the field terminator before the line terminator in order to delete the field enclosers?
I can duplicate the effect on a file with a single line in it, with multiple lines I only get a single entry which has the final column entry of "name| |abc" .
I changed '/r/n' to '\r\n' & the load worked correctly for files with a single entry & multiple entries & the surrounding | were correctly removed
LOAD DATA LOCAL INFILE 'C:/.../Contrato.txt'
INTO TABLE schema.contrato
FIELDS TERMINATED BY ';' ENCLOSED BY '|' LINES TERMINATED BY '\r\n' IGNORE 0 LINES;
The correct MySQL escape character is backslash not forwardslash which is not treated as a special character - so your original code was looking for the 4 character sequence "forwardslash r forwardslash n" to terminate the lines
I'm trying to load data into a mysql table using LOAD DATA LOCAL INFILE using the code below.
Mysql:
LOAD DATA INFILE '/var/www/vhosts/domain.com/httpdocs/test1.csv' INTO TABLE temp_table FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (recloc,client_acc)
Edit: changed LOAD DATA LOCAL INFILE to LOADA DATA INFILE, removed SET id=null, added IGNORE 1 LINES
I'm getting no errors and no imported records. I believe the issue is related to the column names but i'm having a hard time fully understanding what those names should be. Should they be the actual column names within the CSV? or the field names in the DB Table? I would also like the have an auto_incremented primary key (id).
CSV:
recloc,client_acc
"NLGSX3","CORPORATE"
"7SC3BA","QUALITY ASSURANCE"
"3B9OHF","90717-6710"
Any suggestions to what I may be doing wrong? thanks!
Column names in CSV are not necessary, so you should add IGNORE 1 LINES clause.
Columns in your query (recloc,client_acc) need to match columns in table.
First column from CSV will be inserted into recloc, second into client_acc.
If you don't specifu AUTO_INCREMENT column in the statement, but there is one in the table, it should fill automatically.
Short and sweet solution for excel to mysql data import:
Working good for txt file formats.
IN DETAIL:
tbl name=t1
feilds are= name varchar,email varchar;
text.txt file <<== this text file first lines table column names:
name, email
"n1", "e1" next line
"n2", "e2" next line
"n3", "e3" next line
"n4", "e4" next line
"n5", "e5" next line
"n6", "e6" next line
"n7", "e7" next line
pls ignore next line statements
SQL query in wamp
LOAD DATA INFILE 'c:/wamp/www/touch/text.txt' INTO TABLE t1 FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES(name,email)
For this commnad run successfully we have create folders for separately.
Real one is
C:\wamp\mysql\data\wamp\www\touch\text.txt <<==pysical file path is.
But we mention c:/wamp/touch/text.txt
I have file in the following format:
`e00` `e01` `e02` `e03`
`e10` `e11` `e12` `e13
Trying to import the data with
LOAD DATA INFILE 'file' INTO TABLE 'foo' FIELDS TERMINATED BY ' ' ENCLOSED BY '`'
only seems to get the first 3 fields of each line. Is there a way to load the data without altering the file format?
Let's all jump in the way-back machine to answer a 5-year old question!
The fact that the last item is not being loaded is a big hint. According to the manual:
If LINES TERMINATED BY is an empty string and FIELDS TERMINATED BY is
nonempty, lines are also terminated with FIELDS TERMINATED BY.
So, it's looking for a trailing space at the end of the line.
You could add a space to the end of each line in your input file, or try LINES TERMINATED BY '\n'