How to export a CSV file from the MySQL command line - mysql

I have a MySQL database containing a single table which is used as a temporary storage point for manipulating and querying data sets and is periodically deleted and replaced with new data. What I would like to be able to do is export the data from the MySQL command line to use for other purposes. I am using the XAMPP Apache server package with phpMyAdmin on Windows 10.
The issue I am having is the INTO OUTFILE syntax I am using returns an error relating to '\n'. Below is an example of the syntax:
SELECT *
FROM tablename
WHERE work_complete = 'Yes'
INTO OUTFILE 'C:\file path for the file to be exported to\file_name.csv'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY "''"
ESCAPED BY '\'
LINES TERMINATED BY '\n';
I have spent some time researching this without any luck, I have even tried using
LINES TERMINATED BY '\r\n'
but the error remained the same
ERROR: Unknown command '\''.
-> LINES TERMINATED BY '\r\n';
If anyone could provide any tips that would be greatly appreciated.


Use this
SELECT *
FROM tablename
#WHERE work_complete = 'Yes'
INTO OUTFILE 'C:\file path for the file to be exported to\file_name.csv'
FIELDS ENCLOSED BY '"'
TERMINATED BY ';'
ESCAPED BY '"'
LINES TERMINATED BY '\r\n';
I used some other syntax as you and deleted the OPTIONALLY which mysql doesn't like at that place


The problem is not in LINES TERMINATED BY but in ESCAPED BY.
This:
ESCAPED BY '\'
Is invalid syntax, because the backslash is interpreted as an escape character for the following quote. A decent text editor should let you see that.
You need to escape the backslash, like so:
ESCAPED BY '\\'
Alternatively, you can also use '\b':
ESCAPED BY '\b'
Another probem is that OPTIONALLY ENCLOSED accepts only a single character, while you are giving it two single quotes.
In your query:
SELECT * FROM tablename WHERE work_complete = 'Yes'
INTO OUTFILE 'C:\file path for the file to be exported to\file_name.csv'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY "'" ESCAPED BY '\\'
LINES TERMINATED BY '\n'

Related

How to export data to csv in mysql without the header?

MySQL has the functionality to export the data to a CSV file with following statement
SELECT
*
FROM
person
INTO OUTFILE 'person.csv'
FIELDS ENCLOSED BY '"'
TERMINATED BY ','
ESCAPED BY '"'
LINES TERMINATED BY '\r\n';
But by default it will has the column name as the CSV header, so how to remove the header in this statement?
I reviewed the mysql reference but seems these is no such info.

2 options are provided in this post : How can I suppress column header output for a single SQL statement?)
1 : invoke mysql with -N flag will skip all column headers
2 : fake it
select column1 as '', column2 as '' from some_table;

After complete the csv file, seems for the huge records, the big csv file has no header, so if you want to add the header, following:
Several users asked about including headers, i.e., column names or variable names, in the "INTO OUTFILE" syntax.
One approach is to use the "--column-names" option in the mysql invocation:
mysql --column-names -e 'SELECT * FROM mysql.user' > test.dat
(This creates a tab-delimited file test.dat with column names in the first row followed by the query results.)

SELECT
*
FROM
person
INTO OUTFILE 'person.csv'
FIELDS ENCLOSED BY '"'
TERMINATED BY ','
ESCAPED BY '"'
LINES TERMINATED BY '\r\n';
change FIELDS ENCLOSED BY '"' line
with OPTIONALLY ENCLOSED BY '"'

Using this code to export mysql table to csv - but I need syntax for MAC not PC

Code for windows environment is below:
SELECT *
INTO OUTFILE '/documents/products.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM products
Just need the 2nd line for MAC environment - thanks.

First of all you need to put your FROM clause before INTO OUTFILE.
Second, be sure that the directory specified in OUTFILE has the ability for MySQL to write to it (in your case /documents). You'll have a much easier time writing to /tmp (MySQL already has permissions to write by default) than trying to change permissions on another directory to accept writes form your MySQL database.
SELECT *
FROM products
INTO OUTFILE '/tmp/products.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n';

Using MySQL OUTFILE with non-simple data

I've been using the following command to expert mysql data to a csv file.
SELECT * INTO OUTFILE 'output.csv' FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' FROM table1;
It works for simple tables with simple data. However, if the table contains html tags, double quotes, single quotes, ascii characters etc, it does not work propertly, i.e. it will put tabs and new lines in incorrect places, breaking up data where it shouldn't. How can the sql script above be improved to export data with html?

I have tried SELECT...INTO OUTFILE statement, and then LOAD DATA INFILE statement, everything is OK, the HTML text was exported/imported without any mistakes (on MySQL 5.5).
Try to add ENCLOSED BY option, it should help you, e.g. -
SELECT *
INTO OUTFILE 'output.csv'
FIELDS TERMINATED BY '\t' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM
table1;
LOAD DATA INFILE 'output.csv'
INTO TABLE table1
FIELDS TERMINATED BY '\t' ENCLOSED BY '"'
LINES TERMINATED BY '\n';

PostgreSQL csv import from a MySQL csv export?

Will it work?
MySQL export:
SELECT * INTO OUTFILE 'C:/data.csv'
FIELDS TERMINATED BY '\t' OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM table;
PostgreSQL:
COPY table FROM 'C:/data.csv' WITH DELIMITER AS '\t' NULL AS '\\N' CSV
There is missing columns for some reason. So I believe there is problem in delimiter. Am I correct, what can I do? I can inspect row with cause error below. But which characters I must look for?
ERROR: missing data for column "Column21"
CONTEXT: COPY table, line 88219: ...

As mentioned by #knitti postgres need to know escape character: ESCAPE '\'
OPTIONALLY ENCLOSED BY '"' is bad format for csv. It's better to force quoting.
Full code:
mysql 5.5.15:
SELECT *
INTO OUTFILE 'C:/data.csv'
FIELDS TERMINATED BY '\t' ENCLOSED BY '"' ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM table;
postgres 9.0.3:
COPY table FROM 'C:/data.csv' WITH DELIMITER AS E'\t' NULL AS '\\N' ESCAPE E'\\' CSV

Default quote character in PostgreSQL is double-quote, but the documentation says, you can use any single one-byte-character. Try adding
ESCAPE '\\'

Extra backslash \ when SELECT ... INTO OUTFILE ... in MySQL

So I'm trying to export a MySQL table into CSV. I'm using this query:
SELECT * FROM business WHERE id > 0 AND id <= 20000 INTO OUTFILE "business.csv"
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY "\n";
That output something like this:
http://postimage.org/image/2ghyenh5w/full/
The problem with this is that there's always an extra backslash \ where there's newline such as in the address field.
However, CSV exported from phpMyAdmin doesn't have it:
http://postimage.org/image/2gi026tno/full/
Any way to make the SELECT ... OUTFILE ... do the same?
The table I'm dealing with has 20 million records, phpMyAdmin can only handle about 500,000 records for every export action - or it will go blank or mysql server gone away, etc.

It looks like it's impossible for a MySQL export to correctly export both Newlines and Quotes.
When exporting, MySQL will automatically escape both
Field delimiters, and
Line delimiters
By default, the escape character is a backslash. You can override this by adding ESCAPED BY '' to your query.
Unfortunately, in a "normal" (Excel-compatible) CSV file, you probably want different encodings for newlines and quotes. Specifically, you want newlines to be unescaped, and quotes to be doubled.
E.g. If a value contains a newline like so:
This is line 1
And this is "Line 2", which contains quotes
it should become
"This is line 1
And this is ""Line 2"", which contains quotes"
The solution I found was to pre-escape the quotes, and add ESCAPED BY '' (an empty string) to my query.
SELECT REPLACE(field1, '"', '""'),
REPLACE(field2, '"', '""'),
...
FROM ...
WHERE ...
INTO OUTFILE '/someFile.csv'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY ''
LINES TERMINATED BY '\n'

Try this:
SELECT * FROM business WHERE id > 0 AND id <= 20000 INTO OUTFILE "business.csv"
fields terminated by ',' OPTIONALLY ENCLOSED BY '"' escaped by '"'
LINES TERMINATED BY '\n';
I think the issue is that MySQL is trying to escape newline ('\n') in your text fields because it's your line terminator.
FIELDS ESCAPED BY controls how to write special characters. If the FIELDS ESCAPED BY character is not empty, it is used as a prefix that precedes following characters on output:
The FIELDS ESCAPED BY character
The FIELDS [OPTIONALLY] ENCLOSED BY character
The first character of the FIELDS TERMINATED BY and LINES TERMINATED BY values
ASCII NUL (the zero-valued byte; what is actually written following the escape character is ASCII “0”, not a zero-valued byte)
(MySQL)
I don't really understand why it's doing what it's doing in your case, but I was able to get something like that on my Mac and the query above seemed to fix the output in my case.
Hope that helps!

I had the same problem, and I found out (after importing the csv file into a spreadsheet) that there were line breaks in some varchar fields in the MySQL table. After deleting the line breaks, the export worked correctly.

Try this:
SELECT * FROM business WHERE id > 0 AND id <= 20000 INTO OUTFILE "business.csv"
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '\'
LINES TERMINATED BY '\n';
I realised using escaped by '\' makes removes the backslash on exported results.

Got an answer here https://bugs.mysql.com/bug.php?id=46434
Main points are:
1. INTO OUTFILE is intended to produce results ready to load by LOAD DATA
2. By default ESCAPED BY is '\'
3. To disable escaping, use ESCAPED BY ''

I solved this by specifying \r\n as the line terminator, rather than \n:
SELECT * FROM business WHERE id > 0 AND id <= 20000
INTO OUTFILE "business.csv"
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '\'
LINES TERMINATED BY '\r\n';
Each row is now separated by \r\n, but any newlines inside your data will be left unescaped – assuming that the line separators present therein are all \n, not \r\n.
Surprisingly, this worked fine on Linux for my purposes – importing using League\Csv (PHP). I'm guessing that whatever software will be importing your generated CSVs has to be smart enough to differentiate between \n and \r\n for line breaks.

SELECT * FROM business WHERE id > 0 AND id <= 20000 INTO OUTFILE "business.csv"fields terminated by ','
OPTIONALLY ENCLOSED BY '"' ESCAPED BY '' LINES TERMINATED BY '\n';
First, do not put '"" as escape, this will change your content.
Second, if you are using this query on a cli as bellow you need to also remote an extra '\n' add row with multiple lines.
mysql -e "SELECT * FROM business WHERE id > 0 AND id <= 20000 INTO OUTFILE "business.csv"
fields terminated by ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY ''
LINES TERMINATED BY '\n';"

I experienced similar problem.
To prevent extra backlash \ where there are newlines in field values, I added ESCAPED BY '\'.
To prevent MySQL terminating fields where newlines exist within a field value, I replaced newline \n with space ' '.
SELECT
REPLACE(field1, '\n', ' '),
REPLACE(field2, '\n', ' '),
REPLACE(fieldN, '\n', ' ')
FROM business
WHERE id > 0 AND id <= 20000 INTO OUTFILE "business.csv"
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '\' LINES TERMINATED BY '\n';