I have a csv file which contains a lot of sql keywords. How do i escape the keywords?
sample row from the csv file:
F Aabha Hanjura 115 O/o District Fire Officer Chittor Dist . Chittoor AP 500032 500032 Chittoor AP 9963736976 Ap_Airlt_Pst ANDHRA PRADESH AIRTEL
This is what my import statement looks like:
LOAD DATA INFILE "/home/kannel/2.csv" INTO TABLE number_data COLUMNS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"' LINES TERMINATED BY '\n';
Related
The following code doesn't work on the part of importing dates and i cant figure why. Dates in the csv are like DD/MM/YYYY and the error is it imports all the data but leaves null every date. Also error says:
ER_UNKNOWN_SYSTEM_VARIABLE: Unknown system variable 'FECHA_POSICION'
Lines in the csv file look like:
EDC00001,66600/7089855,21/01/2021,21/01/2021,"DEPOSIT Deposit",4000,4000
EDC00002,66600/7089855,29/01/2021,29/01/2021,CFDs,"-9,94","3990,06"
USE DATA_BASE;
CREATE TABLE ESTADO_DE_CUENTA (
ID_OPERACION VARCHAR(20) NOT NULL PRIMARY KEY,
ID_CUENTA VARCHAR(20),
FECHA_POSICION DATE,
FECHA_VALOR DATE,
CONCEPTO VARCHAR(100),
IMPORTE FLOAT(12, 2),
SALDO_EN_EFECTIVO FLOAT(12, 2)
);
LOAD DATA LOCAL INFILE 'PATH.csv' INTO TABLE ESTADO_DE_CUENTA2
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(ID_OPERACION, ID_CUENTA, #FECHA_POSICION, #FECHA_VALOR, CONCEPTO, IMPORTE,
SALDO_EN_EFECTIVO)
SET FECHA_POSICION = STR_TO_DATE(#FECHA_POSICION, '%d/%m/%Y')
SET FECHA_VALOR = STR_TO_DATE(#FECHA_VALOR, '%d/%m/%Y')
You can call SEt only once and all columsn have to be separated by comma
Like
USE DATA_BASE;
LOAD DATA LOCAL INFILE 'PATH.csv' INTO TABLE ESTADO_DE_CUENTA2
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(ID_OPERACION, ID_CUENTA, #FECHA_POSICION, #FECHA_VALOR, CONCEPTO, IMPORTE,
SALDO_EN_EFECTIVO)
SET `FECHA_POSICION` = STR_TO_DATE(#FECHA_POSICION, '%d/%m/%Y') , `FECHA_VALOR` = STR_TO_DATE(#FECHA_VALOR, '%d/%m/%Y')
I have a table like this:
CREATE TABLE `tblinquiries` (
`UID` varchar(50) DEFAULT NULL,
`ReviewDate` date NOT NULL,
`InquiryId` varchar(50) DEFAULT NULL,
`AuditStatus` varchar(50) DEFAULT NULL,
PRIMARY KEY (`InquiryId`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I have a csv file with data:
UID,ReviewDate,InquiryId,AuditStatus
UID1,2018-07-06,109814969,Check
UID2,2018-07-06,109866072,Check
UID3,2018-07-06,109911408,Check
UID4,2018-07-06,109798278,Check
I use below command to to upload the data:
$location = '../uploads/';
$name = $_FILES["file"]["name"];
$filePath = $location.$name;
$table = 'tblinquiries';
LOAD DATA LOCAL INFILE "'.$filePath.'"
INTO TABLE '.$table.'
FIELDS TERMINATED by \',\' OPTIONALLY ENCLOSED BY \'"\'
LINES TERMINATED BY \'\n\'
IGNORE 1 LINES
It uploads the data but adds extra character "\r" from the second line. I exported the data and got like below:
('UID4', '2018-07-06', '109798278', 'Check'),
('UID1', '2018-07-06', '109814969', 'Check\r'),
('UID2', '2018-07-06', '109866072', 'Check\r'),
('UID3', '2018-07-06', '109911408', 'Check\r');
After running:
SELECT AuditStatus, LENGTH(AuditStatus) FROM `tblinquiries`
got:
AuditStatus LENGTH(AuditStatus)
Check 5
Check 6
Check 6
Check 6
How can I solve this?
I would assume that your source data has those \r control characters, because LOAD DATA doesn't typically add data to the source file (unless you tell it to do so, which does not appear to be the case). We can try running RTRIM on the AuditStatus column:
LOAD DATA LOCAL INFILE "'.$filePath.'"
INTO TABLE '.$table.'
FIELDS TERMINATED by \',\' OPTIONALLY ENCLOSED BY \'"\'
LINES TERMINATED BY \'\n\'
IGNORE 1 LINES
(UID, ReviewDate, InquiryId, #AuditStatus)
SET AuditStatus = RTRIM(#AuditStatus);
As #Sloan suggested I changed the line terminators and that solved the problem.
Here is the final code.
LOAD DATA LOCAL INFILE "'.$filePath.'"
INTO TABLE '.$table.'
FIELDS TERMINATED by \',\' OPTIONALLY ENCLOSED BY \'"\'
LINES TERMINATED BY \'\r\n\'
IGNORE 1 LINES
I am trying to export some data to csv but facing some issue with the "," delemeter.
I am running the below query:
SELECT c.NAME
FROM company c
WHERE COMPANY_ID = 1
INTO OUTFILE 'E:\\ab.csv' FIELDS OPTIONALLY ENCLOSED BY '' TERMINATED BY ',' ESCAPED BY '' LINES TERMINATED BY '\n';
If my name does not contain any "," then the csv is correctly populated but if my "name" field contains "," then data is split into two rows.
For ex:
c.Name = "Google INC" -> Works fine
c.Name = "Google,INC" -> it creates csv with two different column "Google" and "INC".
Kindly suggest what should I do.
Based on the link at the comment you can try something like this
SELECT c.NAME
FROM company c
WHERE COMPANY_ID = 1
INTO OUTFILE 'E:\\ab.csv' FIELDS OPTIONALLY ENCLOSED BY '' TERMINATED BY '",' ESCAPED BY '' LINES TERMINATED BY '\n';
Marking ", instead ,
Suck with the following:
$loaddata = "LOAD DATA INFILE 'filename.csv'
INTO TABLE tb1
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\\r\\n'
IGNORE 1 LINES
(
Entity,
HK,
#Period,
)
SET Period = STR_TO_DATE(#Period,'%C%YY%MM')
";
which gives me and sql syntax error near
) SET Period = STR_TO_DATE(#Period,'%C%YY%MM')
Period is a DATE variable. for the period Oct-13 the cvs will show 11310.
tks in advance!
You have a superfluous comma after #Period:
$loaddata = "LOAD DATA INFILE 'filename.csv'
INTO TABLE tb1
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\\r\\n'
IGNORE 1 LINES
(
Entity,
HK,
#Period -- , removed here
)
SET Period = STR_TO_DATE(#Period,'%C%YY%MM')
";
However, your date format string is almost certainly incorrect. %C, %YY and %MM are invalid specifiers. See DATE_FORMAT().
I do not believe this question is a duplicate.
I want rows in the body to be "OPTIONALLY ENCLOSED BY" double quotes. Numerical values should not be enclosed. That's easy to do without a header. But when you include a header using UNION, MySQL now treats every column as a string type and encloses all the values in quotes.
You can add a header to SELECT INTO OUTFILE like this:
SELECT "id", "numerical_values", "string_values" #header section of csv
UNION ALL
SELECT `id`, `numerical_values`, `string_values` #body section of csv
INTO OUTFILE "/tmp/values.csv"
FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY "\n"
FROM `values_table`
Again, if I omit the header, only string_values will be enclosed by quotes.
With the header, all columns are seen as string columns and will be enclosed.
I tried this:
SELECT '"', "id", ',"', "numerical_values", ',"', "string_values", '"'
#added quotes/commas to header
UNION ALL
SELECT `id`, ',', `numerical_values`, ',"', `string_values`, '"'
#added commas, and add quotes around string_values
INTO OUTFILE "/tmp/values.csv"
FIELDS TERMINATED BY "" ENCLOSED BY '' #empty values for fields terminated and enclosed
# empty because manually selected in query
LINES TERMINATED BY "\n"
FROM `values_table`
I thought that would work, but I ended up with what looks like a bunch of extra whitespace. This is from an file I just created with the 2nd method. The header row ends at "string7". The 1st line of data ends after "207". I just put part of the 3rd line.
" num1"," string1"," num2"," string2"," num3"," string3"," num4"," string4"," num5"," string5"," num6"," string6"," string7""
33.95 ," 1023 ", 7.50 ," 207-1023 ", 26.95 ,"2 1023 ",23.00 ,"3Wx4Hx4D ",19.00 ,"1023 ", 0.00 ,"UPC: 123456789012 "," 207 "
40.95 ," 1058 ", 9.00 ,
SELECT '"id"', '"numerical_values"', '"string_values"'
UNION ALL
SELECT id, numerical_values, CONCAT('"', REPLACE(string_values, '"', '""'), '"')
INTO OUTFILE '/tmp/values.csv'
FIELDS TERMINATED BY ',' ENCLOSED BY '' ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM values_table