unable to add outfile prepared statement mysql - mysql

This is my query:
SET #query2 = CONCAT('
SELECT * FROM table_name
INTO OUTFILE "',arg_file_path,'/',#var_table_name,'_CURRENT_TIMESTAMP.csv" fields terminated by "," optionally enclosed by ''"'' lines terminated by "\n" ');
But it produces following output:
SELECT * FROM lcs_tbl_test
INTO OUTFILE "/data/test_outfile/lcs_tbl_test_CURRENT_TIMESTAMP.csv" fields terminated by "," optionally enclosed by '"' lines terminated by "
"
Basically I want "\n" to print as it is in my prepared statement. It is executing \n as line separator in current code.


You need to use back slash as double means for '\' you need to use '\'. Please use below statement.
SET #query2 = CONCAT('
SELECT * FROM table_name
INTO OUTFILE "',arg_file_path,'/',#var_table_name,'_CURRENT_TIMESTAMP.csv" fields terminated by "," optionally enclosed by ''"'' lines terminated by "\\n" ');


Use \ as extra escape character to not interpret \n as new line character.
And, you may also require to change current_timestamp string from literal to dynamic value. Change your concat parameters as below:
SET #query2 = CONCAT(
'SELECT * FROM table_name INTO OUTFILE "',
arg_file_path, '/', #var_table_name, '_', ( CURRENT_TIMESTAMP + 0 ), '.csv"
fields terminated by ","
optionally enclosed by ''"''
lines terminated by "\\n" ');
It produces output as:
SELECT * FROM lcs_tbl_test
INTO OUTFILE "/data/test_outfile/lcs_tbl_test_20140410132736.csv"
fields terminated by ","
optionally enclosed by '"'
lines terminated by "\n"

Related

How to use OUTFILE in codeigniter

I am trying to download the million of records from the database using OUTFILE. I am getting this error
Error Number: 1290
The MySQL server is running with the --secure-file-priv option so it cannot execute this statement
SELECT name, cp_id FROM campaign INTO OUTFILE '/var/www/html/test_csv/1184445179_180912015611.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY ' ';
Filename: models/Admin_model.php
Line Number: 976
below is the model code:
$pth = rand().'_'.date('ymdhis').'.'.'csv';
$path = $_SERVER['DOCUMENT_ROOT'].'/test_csv/'.$pth;
$ad = $this->db->query("SELECT name, cp_id FROM campaign INTO OUTFILE '$path' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n';")

Adding current time to INTO OUTFILE 'C:/Output/data_.csv'

I am looking for a way to add the current time to my filename after every new export.
Current code:
SELECT * INTO OUTFILE 'C:/Output/data_.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' FROM eloge_collector;
What i want to achieve:
SELECT * INTO OUTFILE 'C:/Output/data_2018-05-10-15-14.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' FROM collector;
or something like this.

you can use prepared statements to create your query string then executing it
https://dev.mysql.com/doc/refman/8.0/en/sql-syntax-prepared-statements.html
this is my solution
first create create variable to hold the query string.
use concat function to inject current date time to the query string.
use DATE_FORMAT and now functions to get current date time
set #sql = concat("SELECT * INTO OUTFILE 'C:/filePrefix_",DATE_FORMAT(NOW(), '%Y%m%d%H%i%s'),".fileExtension' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' FROM `table`");
prepare s1 from #sql; --create statment from variable
execute s1; -- execute prepared statements

How to set multiple escape letters in mysql 5.5

How to set multiple escape letters in MySQL 5.5 . When I set multiple letters ('\",\r,\015\n,\n,\b') in MySQL 5.0.22 it was ok , But in MySQL 5.5.28 is giving error code 'ERROR 1083 (42000): Field separator argument is not what is expected; check the manual'
My Query is as follows ,
select ID,MSISDN,IFNULL(TITLE,''),IFNULL(SECONDARY_TITLE,''),IFNULL (NAME,''),IFNULL(ACTIVATION_DATE,''),IFNULL(CREDIT_EXPIRY_DATE,''),
IFNULL(FIRST_VOICE_CALL_DATE,''),IFNULL(FIRST_SMS_DATE,''),IFNULL(FIRST_MMS_DATE,''),
IFNULL(FIRST_WAP_DATE,''),IFNULL(FIRST_TOPUP_DATE,''),IFNULL(TOTAL_USAGE,''),
IFNULL(TOTAL_TOPUP_AMOUNT,''),IFNULL(TOTAL_TOPUP_COUNT,''),IFNULL(LAST_TOPUP_DATE,''),
IFNULL(ACCOUNT_BALANCE,''),IFNULL(TARIFF_PLAN,''),IFNULL(DATE_OF_BIRTH,''),IFNULL(ADDRESS,''),
IFNULL(COUNTRY,''),IFNULL(CITY,''),IFNULL(POSTCODE,''),IFNULL(IC_NUMBER,''),
GENDER,IFNULL(FIXED_LINE_PHONE,''),IFNULL(EMAIL,''),IFNULL(PROFESSION,''),
IFNULL(EMPLOYER_NAME,''),IFNULL(EMPLOYER_ADDRESS,''),IFNULL(EMPLOYER_PHONE_NUMBER,''),
IFNULL(EMPLOYER_FAX_NUMBER,''),IFNULL(SIM_STATE,''),
IFNULL(ADDITIONAL_INFORMATION,'') INTO OUTFILE '/tmp/SUBSCRIBER_PROFILE.txt'
FIELDS ESCAPED BY '\",\\r,\\015\n,\\n,\\b' TERMINATED BY '|'
OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\n' FROM SUBSCRIBER_PROFILE;
But When I set only one letter in 'escape by' option, it works
Please help me to solve this issue .
Thanks ,
Dhanushka

The escape character in your case should just be "\" because you're using \ to escape the character. It isn't asking for a list of sequences that you're going to escape, just the character that lets it know that it is being escaped, in this case the \.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Working with MySQL 5.0.21
mysql> select ID,MSISDN,IFNULL(TITLE,''),IFNULL(SECONDARY_TITLE,''),IFNULL(NAME,''),IFNULL(ACTIVATION_DATE,''),IFNULL(CREDIT_EXPIRY_DATE,''),
-> IFNULL(FIRST_VOICE_CALL_DATE,'') ,IFNULL(FIRST_SMS_DATE,''),IFNULL(FIRST_MMS_DATE,''),IFNULL(FIRST_WAP_DATE,''),
-> IFNULL(FIRST_TOPUP_DATE,''),IFNULL(TOTAL_USAGE,''),IFNULL(TOTAL_TOPUP_AMOUNT,''),IFNULL(TOTAL_TOPUP_COUNT,''),
-> IFNULL(LAST_TOPUP_DATE,''),IFNULL(ACCOUNT_BALANCE,''),IFNULL(TARIFF_PLAN,''), IFNULL(DATE_OF_BIRTH,''),IFNULL(ADDRESS,''),
-> IFNULL(COUNTRY,''),IFNULL(CITY,''),IFNULL(POSTCODE,''),IFNULL(IC_NUMBER,''), GENDER,IFNULL(FIXED_LINE_PHONE,''),IFNULL(EMAIL,''),
-> IFNULL(PROFESSION,''),IFNULL(EMPLOYER_NAME,''),IFNULL(EMPLOYER_ADDRESS,''), IFNULL(EMPLOYER_PHONE_NUMBER,''),
-> IFNULL(EMPLOYER_FAX_NUMBER,''),IFNULL(SIM_STATE,''),IFNULL(ADDITIONAL_INFORMATION,'') INTO OUTFILE '/tmp/SUBSCRIBER_PROFILE.txt'
-> ***FIELDS ESCAPED BY '\",\\r,\\015\n,\\n,\\b'*** TERMINATED BY '|' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\n' FROM SUBSCRIBER_PROFILE;
*Query OK, 345812 rows affected (7.77 sec*)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Not working with MySQL 5.5.28
mysql> select ID,MSISDN,IFNULL(TITLE,''),IFNULL(SECONDARY_TITLE,''),IFNULL(NAME,''),IFNULL(ACTIVATION_DATE,''),IFNULL(CREDIT_EXPIRY_DATE,''),
-> IFNULL(FIRST_VOICE_CALL_DATE,'') ,IFNULL(FIRST_SMS_DATE,''),IFNULL(FIRST_MMS_DATE,''),IFNULL(FIRST_WAP_DATE,''),
-> IFNULL(FIRST_TOPUP_DATE,''),IFNULL(TOTAL_USAGE,''),IFNULL(TOTAL_TOPUP_AMOUNT,''),IFNULL(TOTAL_TOPUP_COUNT,''),
-> IFNULL(LAST_TOPUP_DATE,''),IFNULL(ACCOUNT_BALANCE,''),IFNULL(TARIFF_PLAN,''),IFNULL(DATE_OF_BIRTH,''),IFNULL(ADDRESS,''),
-> IFNULL(COUNTRY,''),IFNULL(CITY,''),IFNULL(POSTCODE,''),IFNULL(IC_NUMBER,''),GENDER,IFNULL(FIXED_LINE_PHONE,''),IFNULL(EMAIL,''),
-> IFNULL(PROFESSION,''),IFNULL(EMPLOYER_NAME,''),IFNULL(EMPLOYER_ADDRESS,''),IFNULL(EMPLOYER_PHONE_NUMBER,''),
-> IFNULL(EMPLOYER_FAX_NUMBER,''),IFNULL(SIM_STATE,''),IFNULL(ADDITIONAL_INFORMATION,'') INTO OUTFILE '/tmp/SUBSCRIBER_PROFILE.txt'
-> **FIELDS ESCAPED BY '\",\\r,\\015\n,\\n,\\b'** TERMINATED BY '|' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\n' FROM SUBSCRIBER_PROFILE;
*ERROR 1083 (42000): Field separator argument is not what is expected; check the manual*

can I use a variable to specify OUTFILE in mysql

Is there a way to do something like the following ? which doesn't work but shows what I want to do
SET #OutputPath = '/Users/jo/Documents'
SET #fullOutputPath = CONCAT(#OutputPath,'/','filename.csv')
SET #fullOutputPath2 = CONCAT(#OutputPath,'/','filename2.csv')
SELECT * INTO OUTFILE #fullOutputPath
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
FROM database.tableName;
SELECT * INTO OUTFILE #fullOutputPath2
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
FROM database.tableName2;

Edit: Saving data(e.g. a table) into file without using variable (only constant values)
-- folder_path could could be like => c:/users/sami
-- choose the directory/folder already available in system
-- and make sure you have access to write the file there
SELECT * INTO OUTFILE 'folder_path/filename.csv'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
FROM database.tableName;
Now using variable
Whenever you have to use a variable name in sql, you need dynamic sql (which is applicable in stored procedures only, neither in simple sql query nor in triggers or functions)
SET #OutputPath := 'Users/jo/Documents'; //or any folder_path
SET #fullOutputPath := CONCAT(#OutputPath,'/','filename.csv');
SET #fullOutputPath2 := CONCAT(#OutputPath,'/','filename2.csv');
set #q1 := concat("SELECT * INTO OUTFILE ",#fullOutputPath,
" FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
FROM database.tableName");
set #q2 := concat("SELECT * INTO OUTFILE ",#fullOutputPath2,
" FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
FROM database.tableName2");
prepare s1 from #q1;
execute s1;deallocate prepare s1;
prepare s1 from #q2;
execute s1;deallocate prepare s1;
As you had both ' and " in your query already, so I concatenated your query using " and used \ to escape your original " to ensure its use as a literal character and not used for concatenation
I just told the use of variable in sql. First You should make sure if your query works like example at the top (without using variable)
Conclusion: If your above query works fine then my told dynamic sql will work as well given that you are using it in some stored procedure.

I have a low carma so I'm posting an answer that should go as a comment to Sami's post - you need to enclose the file name by quotes (note added ' before and after #fullOutputPath):
set #q1 := concat("SELECT * INTO OUTFILE '",#fullOutputPath,
"' FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
FROM database.tableName");

If you want to do this from bash, i.e. export some data from mysql in csv to a file with dynamic name, it maybe easier and more readable like the following.
The SQL with embedded bash variables:
where (e.timestamp >= ${begin_ts} and e.timestamp < ${end_ts}) order by ed.timestamp ASC ) a
INTO OUTFILE '${export_path}' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
And the bash script that runs the sql file. Notice the envsubst command that evaluates the sql script and substitutes the variables.
#!/bin/bash
mysql_db="dbname"
mysql_user="mysqlpass"
mysql_pass="password"
export_path="./data.csv"
begin_ts="1478278490"
current_ts=$(date +%s -u)
sql=`export_path=${export_path} begin_ts=${last_ts} end_ts=${current_ts} envsubst < export.sql`
mysql $mysql_db -u $mysql_user -p$mysql_pass -e"${sql}"

You cannot do it in mysql CLI but in this way it works
mysql -e "SELECT * FROM database.tableName;" -u user -p database > filename.csv

mysql & INTO OUTFILE - escape or replace new lines in data

I have the following sql statement to out put a sql table to a csv file.
Some data found in the column 'content' includes new line charactors which is causing issues, is there a way to replace all \n with 's?
SELECT id, title, content
INTO OUTFILE '/content.csv'
FIELDS TERMINATED BY ',' ESCAPED BY '\\' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM content

REPLACE( IFNULL(content , ''), '\n' , '<br/>' ) as content

Try:
SELECT id, title, REPLACE(content , CHR(13), ' ') as content
INTO OUTFILE '/content.csv'
FIELDS TERMINATED BY ',' ESCAPED BY '\\' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM content
where CHR(13) represents new line.