I'm trying to load csv files into mysql table.
Delimiter : ,(comma)
As part of the source data few of the field values are enclosed in double quotes and inside the double quotes we have ,
There are few records for which / is part of the field data and we need to escape it.
By default / is getting escaped and when I specified the " as escape character " is getting escaped. As we have multiple special characters inside the same file, we need to escape multiple special characters.
Any suggestion
Eg:
id name location
1 A "Location , name here"
2 B "Different Location"
3 C Another Location
4 D Location / with escape character
LOAD DATA LOCAL INFILE 'data.csv' INTO TABLE table_name FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES;
I think it's not possible. Referring to LOAD DATA reference
Any of the field- or line-handling options can specify an empty string (''). If not empty, the FIELDS [OPTIONALLY] ENCLOSED BY and FIELDS ESCAPED BY values must be a single character.
Only a single char is supported for ESCAPED BY field.
My proposal is to use any programming language (e.g. PHP, C# etc.) for opening and processing file line-by-line using regexp
I have a .txt file in UTF-8 format with information I am trying to import in an already created table that has rows already in it.
The information in the .txt file is structured like this: (the quotes are included in the .txt file)
"Bob,Smith,25,California,,,,Single,"
"John,Doe,72,Nevada,,2,1,Married,"
"Will,Smith,22,Texas,1000005,2,1,Married,"
The query I'm using is:
LOAD DATA LOCAL INFILE 'myfile.txt' INTO TABLE mytable FIELDS ENCLOSED BY '"' TERMINATED BY ',' LINES TERMINATED BY '\n'
What happens is that all of these records get inserted but get inserted like this
Bob,null,null,null,null,null,null,null
John,null,null,null,null,null,null,null
Will,null,null,null,null,null,null,null
It's like the " is not being caught at the end or something weird . Am I doing something wrong here?
According to the example data provided your fields are not enclosed by quotes but rather the whole record is.
You can use the STARTING BY option to ignore the initial quote and the trailing one should be ignore automatically.
I think what you need is this:
LOAD DATA LOCAL INFILE 'your_file.txt' INTO TABLE your_table FIELDS TERMINATED BY ',' LINES STARTING BY '"' TERMINATED BY '\n';
The format of your text file does not make any sense. (Strangely enough, the way the importer handles it does not make any sense either. But we have no control over the importer, so let us ignore that.)
The double quotes must surround each field, not each line. The ENCLOSED BY '"' clause refers to the fields, not to the lines. So, you are telling the importer that your fields are enclosed in quotes, but you are enclosing your lines in quotes. So, the importer considers each one of your lines as a field.
(Then, the importer proceeds to further chop up your lines at the comma, which makes no sense because the commas are within the quotes, so it should be ignoring them, so the importer is kind of brain-damaged too.)
By using the FIELDS ENCLOSED BY '"' statement the input file should enclose EACH field data by a " therefor the input file should be as follows
"Bob","Smith","25","California","","","","Single",
"John","Doe","72","Nevada","","2","1","Married",
"Will","Smith","22","Texas","1000005","2","1","Married",
That should add the data into the fields
I'm having a seemingly basic problem with loading data to MySQL. I'm using:
LOAD DATA LOCAL INFILE 'C:/.../Contrato.txt'
INTO TABLE schema.contrato
FIELDS TERMINATED BY ';' ENCLOSED BY '|' LINES TERMINATED BY '/r/n' IGNORE 0 LINES;
Each line on the file looks like this, generated by another program:
|abc|;|cde|;|123|;|456|;|name|\r\n
When executing the load, everything seems to load properly, except the very last field. When I look at the table, the last field actually shows the '|' characters around the name. It's not the end of the world, but it's strange that it would do that. As of now, I'm fixing it by adding a ';' right before the \r\n characters.
Is this the way it's supposed to be done?? Why would I need to add the field terminator before the line terminator in order to delete the field enclosers?
I can duplicate the effect on a file with a single line in it, with multiple lines I only get a single entry which has the final column entry of "name| |abc" .
I changed '/r/n' to '\r\n' & the load worked correctly for files with a single entry & multiple entries & the surrounding | were correctly removed
LOAD DATA LOCAL INFILE 'C:/.../Contrato.txt'
INTO TABLE schema.contrato
FIELDS TERMINATED BY ';' ENCLOSED BY '|' LINES TERMINATED BY '\r\n' IGNORE 0 LINES;
The correct MySQL escape character is backslash not forwardslash which is not treated as a special character - so your original code was looking for the 4 character sequence "forwardslash r forwardslash n" to terminate the lines
I need to export data from a mysql table to a csv file. Two fields in particular are giving me trouble. Their data types are varchar and TEXT.
Data in these columns contain all sorts of gross characters, particularly new line (\n) and tabs (\t). I need to export these into a .csv file for a migration.
Unfortunately, I cannot enclose the fields with '"' because the destination database doesn't support that formatting.
So, my query looks like the following:
SELECT `varchar_col`,`text_col` FROM `db`.`tbl` INTO OUTFILE '/path/to/my/file.csv' FIELDS TERMINATED BY '\t' ESCAPED BY "\\" LINES TERMINATED BY '\n';
When I look at my output file (used gedit and nano), I simply see that each new line or tab in the file is preceded by a backslash (see example below). I would like a new line or tab instead to read the literal chacters '\n' or '\t' instead of actual new lines and tabs.
Example of problem with new line-
field value of:
value one
value two
exported to .csv gives me:
value one\
value two\
instead of:
value 1\nvalue 2
Can anyone tell me what im doing wrong?
Thanks
I'm not quite sure. But is this what you want? (Output literal chacters '\n' or '\t' instead of actual new lines and tabs.)
SELECT * FROM block INTO OUTFILE '/var/tmp/d.csv' FIELDS TERMINATED BY '\\t' ESCAPED BY "\\" LINES TERMINATED BY '\\n';
I am trying to upload a tab delimitted file with MySQL. I want a query something likes this: LOAD DATA LOCAL INFILE 'file' INTO TABLE tbl FIELDS TERMINATED BY 'TAB' Is there something I can subsitute for TAB to make this work?
have you tried '\t' the escape sequence + "T" is considered tab... haven't tried, but might be what you need
Just tried to find the answer to this question myself to save re-saving my file with commas separating instead of tabs...
From an old MySQL reference manual, a long way down the page, you can find that TAB is the default separater for files loaded using LOAD DATA on MySQL.
See: http://dev.mysql.com/doc/refman/4.1/en/load-data.html
I just loaded a CSV file in this way into MySQL5.1.
BW
fields terminated by '\t'
Try this one
Note :
Field and Line Handling
For both the LOAD DATA and SELECT ... INTO OUTFILE statements, the syntax of the FIELDS and LINES clauses is the same. Both clauses are optional, but FIELDS must precede LINES if both are specified.
If you specify a FIELDS clause, each of its subclauses (TERMINATED BY, [OPTIONALLY] ENCLOSED BY, and ESCAPED BY) is also optional, except that you must specify at least one of them. Arguments to these clauses are permitted to contain only ASCII characters.
If you specify no FIELDS or LINES clause, the defaults are the same as if you had written this:
FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\'
LINES TERMINATED BY '\n' STARTING BY ''
Backslash is the MySQL escape character within strings in SQL statements. Thus, to specify a literal backslash, you must specify two backslashes for the value to be interpreted as a single backslash. The escape sequences '\t' and '\n' specify tab and newline characters, respectively.
In other words, the defaults cause LOAD DATA to act as follows when reading input:
Look for line boundaries at newlines.
Do not skip any line prefix.
Break lines into fields at tabs.
Do not expect fields to be enclosed within any quoting characters.
Interpret characters preceded by the escape character \ as escape sequences. For example, \t, \n, and \ signify tab, newline, and backslash, respectively. See the discussion of FIELDS ESCAPED BY later for the full list of escape sequences.
Conversely, the defaults cause SELECT ... INTO OUTFILE to act as follows when writing output:
Write tabs between fields.
Do not enclose fields within any quoting characters.
Use \ to escape instances of tab, newline, or \ that occur within field values.
Write newlines at the ends of lines.
see: https://dev.mysql.com/doc/refman/8.0/en/load-data.html
for more details.