I'm trying to convert timestamps on the fly when importing a csv file into mysql from string to datetime data type. But I am getting a #1411 - Incorrect datetime value: '2007-03-30 16:01:15' for function str_to_date error.
The SQL:
load data infile 'C:/ProgramData/MySQL/MySQL Server 5.5/data/testfile.csv'
into table test
fields terminated by ','
lines terminated by '\n'
(date, col1,col2,col3,col4)
SET
date = str_to_date(date,'%Y.%m.%d %H:%i:%s.%f');
All rows in the .csv are formated like this:
2007.03.30 16:01:15.901,117.53,117.55,35600000,43700000
I've applied
SELECT str_to_date(date,'%Y.%m.%d %H:%i:%s.%f') FROM test
to sample data that was already stored in mysql, it did work.
The target row date is set to DATETIME.
You need to go via a user variable. As the manual says:
The column list can contain either column names or user variables. With user variables, the SET clause enables you to perform transformations on their values before assigning the result to columns.
User variables in the SET clause can be used in several ways. The following example uses the first input column directly for the value of t1.column1, and assigns the second input column to a user variable that is subjected to a division operation before being used for the value of t1.column2:
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, #var1)
SET column2 = #var1/100;
In your case:
LOAD DATA INFILE 'C:/ProgramData/MySQL/MySQL Server 5.5/data/testfile.csv'
INTO TABLE test
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(#date, col1, col2, col3, col4)
SET date = STR_TO_DATE(#date, '%Y.%m.%d %H:%i:%s.%f');
Related
I would like to convert MySQL string to required date format .
I have below lines in a file.
30-06-2017,clarke
31-07-2018,warner
my table is having 2 columns .
Column1 datatype :: date
Column2 datatype :: varchar(30)
I have executed below query
load data local infile 'test.txt' into table sample fields terminated by ',' set column1=str_to_date(#c1,'%d-%m-%Y') ;
Column1 data was not loaded and I got below warnings.
Data wad truncated for column1 at row1
May I know what is wrong in the sql query which I am using ?
You have to include the columns (#c1, #c2) from the file. Following command works fine.
LOAD DATA LOCAL INFILE 'test.txt' INTO TABLE sample FIELDS TERMINATED BY ',' (#c1, #c2) SET column1=STR_TO_DATE(#c1,'%d-%m-%Y'), column2=#c2;
Hello I'm using LOAD DATA INFILE to populate a table in MySQL.
LOAD DATA INFILE 'test.txt'
INTO TABLE myTestTable
FIELDS TERMINATED BY '\t'
IGNORE 1 LINES;
Everything is working peachy except that there is a datetime column in my data that is formatted without any delimiter between the date and time sections. Like so:
SomeDateColumn
20050101081946
When I read this in, MySQL replaces all the dates with dummy values. Is there a way to have MySQL read this in correctly straight from a file?
Thanks!
You may call STR_TO_DATE when you run LOAD DATA, and convert the text date to a bona fide date on the fly:
LOAD DATA INFILE 'test.txt'
INTO TABLE myTestTable
FIELDS TERMINATED BY '\t'
IGNORE 1 LINES
(
col1, col2, #var1 -- list out all columns here
)
SET SomeDateColumn = STR_TO_DATE(#var1, '%Y%m%d%h%i%s');
I need a way to use the UTC_TIMESTAMP() function in a CSV file.
Currently I'm using the following Syntax;
LOAD DATA INFILE '/path/to/file.csv'
INTO TABLE my_table FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n';
I'm not quite sure what to do to keep the UTC_TIMESTAMP() from being enclosed in quotes. When entered in the database, I get the following result
"0000-00-00 00:00:00"
This was the only example I could find on stack overflow or on Google for converting a string to a MYSQL value.
MySQL CSV import: datetime value
I solved the problem by following the MySQL documentation on this page.
https://dev.mysql.com/doc/refman/5.7/en/load-data.html
About halfway down the page there is a section that shows you how to create variables for rows and then set table rows equal to native mysql functions or values assigned to those variables(Whichever you choose).
The example in the documentation that I'm referring to looks like this.
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, column2)
SET column3 = CURRENT_TIMESTAMP;
I fixed my problem by restructuring my code like this...
LOAD DATA LOCAL INFILE '/path/to/file.csv'
INTO TABLE veh_icodes FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(id, vcode, year, make, model, body_style, tran_type, engine_cap, drive_train, doors, trim, created_by, updated_by, #created_at, #updated_at, active)
SET created_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP;"
I hope this helps someone. :)
I am using a Load command to insert all the data in a CSV file to the mysql table. The load query sample is:
LOAD DATA LOCAL INFILE 'C:\\path\\to\\windows\\file.CSV'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(field1, field2, field3, fieldx);
The data in the file has the following format:
FName || LName || num1 || num2 || num3|| num4 || num5 || date
Here all nums are of Float data type.
Here the date format of date in csv file is dd-MM-yyyy.
So when loading the complete file in DB I am storing dates as a varchar, because when I store them in a DATE datatype I get 0000-00-00.
Now after inserting data I have to work on dates but I am not able to get the sorted dates as they are stored as a Varchar.
Is there any way I can specify the default dateformat at the time of table creation. For example:
create table test (
mydates date(date : dd-mm-yyyy));
something like this.
Or could anyone suggest a different approach to tackle this issue?
Use str_to_date to convert the string into a date time object and use set to set the column's value manually. Lets say fieldx is your date field:
LOAD DATA LOCAL INFILE 'C:\\path\\to\\windows\\file.CSV'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(field1, field2, field3, #fieldx)
SET fieldx = str_to_date(#fieldx, "%d-%m-%Y");
Have a look at the manual page for load data for more information; and adjust the format string using this table.
I suggest using the "string to date" function
STR_TO_DATE(table.datestring, '%m-%d-%Y')
I have CSV file having two columns id_a and id_b, but I need to insert 4 more columns; ie. emp_sal_a, emp_sal_b, emp_dept_a, emp_dept_b using sqlldr. So my current control file looks like:
load data
infile '/home/.../employee.txt'
into table employee
fields terminated by ","
( id_a, id_b,
emp_sal_a ":id_a+1000", emp_sal_b "id_b+1000", emp_dept_a "10", emp_dept_b "20")
But I am getting error:
invalid binding variables
From MySQL Load Data Ref
note: search for the "(" character and it's the 35th instance of it on the page
User variables in the SET clause can be used in several ways. The following example uses the first input column directly for the value of t1.column1, and assigns the second input column to a user variable that is subjected to a division operation before being used for the value of t1.column2:
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, #var1)
SET column2 = #var1/100;
#var1 is the name of a variable you want to run an operation on, and what you're doing is calling SET on column2 to be equal to #var1/100.