I'm trying to import data from a .csv file and I'm getting and error code 1193 unknown system variable. I'm utilizing MySQL 5.5.34.
LOAD DATA LOCAL INFILE 'path to the file/student_2.csv'
INTO TABLE STUDENT
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 2 LINES
(S_ID, S_LAST, S_FIRST, S_MI, S_ADDRESS, S_CITY, S_STATE, S_ZIP, S_PHONE, S_CLASS, #S_DOB, S_PIN, F_ID, #DATE_ENROLLED);
SET S_DOB = STR_TO_DATE(#S_DOB, '%m/%d/%Y'),
DATE_ENROLLED = STR_TO_DATE(#DATE_ENROLLED, '%m/%d/%Y');
The csv file's data is as follows:
S_ID,S_LAST,S_FIRST,S_MI,S_ADDRESS,S_CITY,S_STATE,S_ZIP,S_PHONE,S_CLASS,S_DOB,S_PIN,F_ID,DATE_ENROLLED
Number,String,String,String,String,String,String,String,String,String,Date/Time,String,Number,String
1,Joffs,Tami,R,1817 Eagldge Cle,Houston,TX,74027,356487654,SR,7/14/88,8891,1,1/3/13
2,Petez,Jimmge,C,951 Drainbow Place,Absail,TX,76901,3253945432,SR,18/09/76,1230,1,11/10/02
3,Marks,Johannes,A,1015 Wild St,Dallas,TX,71012,3251454321,JR,08/13/83,1613,1,8/24/03
4,Smyth,Mark,,428 EN 16 Plaza,Arsehole,TX,7012,3221143210,SO,1/14/88,1841,2,8/23/04
I also change the year format from %Y to %y and did not work either.
It is something wrong with the script?
Hm - I can't try it out and I did not dive deep into your script, but are you sure about the ; before the set-commands?
....
(S_ID, S_LAST, S_FIRST, S_MI, S_ADDRESS, S_CITY,
S_STATE, S_ZIP, S_PHONE, S_CLASS, #S_DOB, S_PIN, F_ID, #DATE_ENROLLED);
SET S_DOB = STR_TO_DATE(#S_DOB, '%m/%d/%Y'),
DATE_ENROLLED = STR_TO_DATE(#DATE_ENROLLED, '%m/%d/%Y');
Related
I have been trying to load csv data into Snowflake using COPY INTO command
This is the sample data
4513194677~"DELL - ULTRASHARP 32\" MONITOR 4K U3223QE"~""~""
I have tried using below COPY INTO syntax
file_format =
type = 'csv'
field_delimiter = '~'
skip_header = 1
record_delimiter = '\\n'
field_optionally_enclosed_by = '"'
ESCAPE = 'NONE'
ESCAPE_UNENCLOSED_FIELD = 'NONE'
However, getting this error "Found character 'M' instead of field delimiter '~'"
How can I escape the " and load the columns data as DELL - ULTRASHARP 32 " MONITOR 4K U3223QE
If I try to use ESCAPE, I get below error when running the COPY command
[ERROR] ProgrammingError: 001003 (42000): 01a8e01d-3201-36a9-0050-4502537cfc7f: SQL compilation error:
syntax error line 15 at position 43 unexpected '''.
syntax error line 20 at position 20 unexpected ')'.
file_format =
type = 'csv'
field_delimiter = '~'
skip_header = 1
record_delimiter = '\\n'
field_optionally_enclosed_by = '"'
ESCAPE = '\\'
ESCAPE_UNENCLOSED_FIELD = '\\'
Try using two double quotes in the data instead of one without trying to escape the double quote
Data similar to "sample"
You can have your csv formated like below
"Data similar to ""sample"""
df <- data.frame(category = c("A","B","A","D","E"),
date = c("5/10/2005","6/10/2005","7/10/2005","8/10/2005","9/10/2005"),
col1 = c(1,NA,2,NA,3),
col2 = c(1,2,NA,4,5),
col3 = c(2,3,NA,NA,4))
I have to insert a data frame that is created in R to mysql server.
I have tried these methods(Efficient way to insert data frame from R to SQL). However, my data also has NA which are fails the whole process of exporting.
Is there a way around to faster upload to data.
dbWriteTable(cn,name ="table_name",value = df,overwrite=TRUE, row.names = FALSE)
The above works but is very slow to upload
The method that I have to use is this :
before = Sys.time()
chunksize = 1000000 # arbitrary chunk size
for (i in 1:ceiling(nrow(df)/chunksize)) {
query = paste0('INSERT INTO dashboard_file_new_rohan_testing (',paste0(colnames(df),collapse = ','),') VALUES ')
vals = NULL
for (j in 1:chunksize) {
k = (i-1)*chunksize+j
if (k <= nrow(df)) {
vals[j] = paste0('(', paste0(df[k,],collapse = ','), ')')
}
}
query = paste0(query, paste0(vals,collapse=','))
dbExecute(cn, query)
}
time_chunked = Sys.time() - before
Error Encountered:
Error in .local(conn, statement, ...) :
could not run statement: Unknown column 'NA' in 'field list'
One of the fastest ways to load data into MySQL is to use its LOAD DATA command line tool. You may try first writing your R data frame to a CSV file, then using MySQL's LOAD DATA to load it:
write.csv(df, "output.csv", row.names=FALSE)
Then from your command line, use:
LOAD DATA INFILE 'output.csv' INTO TABLE table_name
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES;
Note that this assumes the CSV file is already on the same machine as MySQL. If not, and you have it still locally, then use LOAD DATA LOCAL INFILE instead.
You may read MYSQL import data from csv using LOAD DATA INFILE for more help using LOAD DATA.
Edit:
To deal with the issue of NA values, which should represent NULL in MySQL, you may take the approach of first casting the entire data frame to text, and then replacing the NA values with empty string. LOAD DATA will interpret a missing value in a CSV column as being NULL. Consider this:
df <- data.frame(lapply(df, as.character), stringsAsFactors=FALSE)
df[is.na(df)] <- ""
Then, use write.csv along with LOAD DATA as described above.
Background: I work with phpMyAdmin (MySQL Workbench) in a mysql DB. I write some PHP code to import data in the DB and execute this with the task scheduler of windows. <= this works fine!
Now I want to export some data into a file in a Windows folder. At first I write the SQL code in phpMyAdmin to see some debug-infos. <= this is where the problem occurs.
My Topic:
I want to export some columns of my DB. My Goal is to put a variable CURRENT_TIMESTAMP in the filename. For this I use the Concat statement.
My code (posted below), gets the result of the following error:
Can't create/write to file 'C:\Temp\Export\2018-08-08 09:21:27.txt' (Errcode: 13 "Permission denied")
Funny thing is, if I replace the variable CURRENT_TIMESTAMP with e.g. "Hello World" there is no error and my file is created in the folder.
Here is my code:
*set #sql = concat("SELECT `LS_ID_Nr`,
`Stk_pro_Krt_DL` * `Krt_DL` + `RB_Stk_pro_Krt_DL` * `RB_Krt_DL`,
`Umstellzeit`,
`Produktionszeit`,
`Teilmeldung`,
`Fertigmeldung` INTO OUTFILE 'C:/Temp/Export/",CURRENT_TIMESTAMP,".txt' fields terminated by ';' lines terminated by '\r\n' From praemie where Proof_P = 0");
prepare s1 from #sql;
execute s1;
DROP PREPARE s1;
UPDATE praemie SET Proof_P = 1 WHERE Proof_P = 0;*
Does anybody have an idea why there is an Permission Error with the use of a variable? Thanks in advance!
Ooooohh....
I get it now!
The problem is, that windows not handle ":" in filenames. So I have to edit the code with the Date_Format statement like this:
set #sql =
concat("SELECT `LS_ID_Nr`, `Stk_pro_Krt_DL` * `Krt_DL` + `RB_Stk_pro_Krt_DL` * `RB_Krt_DL`,
`Umstellzeit`, `Produktionszeit`, `Teilmeldung`, `Fertigmeldung`
INTO OUTFILE 'C:/Temp/Export/Test - ", DATE_FORMAT(NOW(), '%Y%m%d%H%i%s')," - Test.txt'
fields terminated by ';'
lines terminated by '\r\n'
From praemie where Proof_P = 0")
I have this data in my file (sometimes fields could be in double quotes) as shown below:
1, "ala", johnes, 2017-09-01, 100
2, dqwdqdq,, 2017-09-01, 101
Data is loaded to database however quotes are also inserted into column so instead to see ala i see "ala" in the column.
My code:
Using exclecon As New MySqlConnection("Server=Localhost;DataBase=mobily;user=root;password=897;")
Dim uploadQry As String = "Load DATA LOCAL INFILE 'C:/Users/Robert/Desktop/test.csv' INTO TABLE Test FIELDS TERMINATED BY ',' ENCLOSED BY '""' ESCAPED BY '' LINES TERMINATED BY '\r\n' IGNORE 0 LINES;"
Dim myCUpload As New MySqlCommand(uploadQry, exclecon)
exclecon.Open()
myCUpload.ExecuteNonQuery()
End Using
i want to import csv file. my script is :
#echo off
set numid=2015092510524361378197540100
sqlldr USER#db/PSW data=csv\2015092510524361378197540100.csv control=ctl\control.ctl log=log\2015092510524361378197540100.log bad=bad\id.bad
pause
my table is :
CREATE TABLE SV (NO1 VARCHAR2(255),NAMA VARCHAR2(255),ALAMAT VARCHAR2(255),id VARCHAR2(20),JAB VARCHAR2(50),numid VARCHAR(55));
my control.ctl is :
OPTIONS (SKIP=43, errors=12000) LOAD DATA APPEND INTO TABLE sv when NAMA <> '' FIELDS TERMINATED BY ',' optionally enclosed by '"' TRAILING NULLCOLS (no filler,no1 "TRIM (:no1)",nama "TRIM (:nama)", alamat "TRIM (:alamat)",id "TRIM (:id)",jab "TRIM (:jab)",numid "%numid%")
error is :
Record 10: Rejected - Error on table sV, column numid.
ORA-00911: invalid character
please let me know which one is wrong. thanks All
Since numid is a VARCHAR2(55), change the table datatype to be VARCHAR2 and the control file to read:
numid "TRIM(:numid)"