I have a table with payDate DATETIME in which I'm inserting using load data local infile 'file.txt' into table tableName; dates like 26/04/2012 00:00:00.
This gives warnings like Warning | 1265 | Data truncated for column 'payDate' at row 1 and the date in the table is 0000-00-00 00:00:00.
Is there any way to specify the format of the date?
Try STR_TO_DATE:
load data local infile 'file.txt' into table tableName
SET payDate = str_to_date(#payDate, '%d/%m/%Y');
I worked out without converting from datetime to varchar ,
Below is the working code for it -
mysql query to export csv data to local directory
SELECT organization_id,bank_name,branch_name,account_number,statement_type,parameter_name,
parameter_value,created_by,creation_date,updated_by,updation_date FROM cm_sub_param_values
INTO OUTFILE 'D:\mytable.csv'
FIELDS ESCAPED BY '""'
TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n';
In the csv exported , remove the column names and SAVE.
In the Table , creation_date and updation_date are of type datetime .
Import CSV data into mysql
LOAD DATA INFILE 'D:/mytable.csv' IGNORE
INTO TABLE `cm_sub_param_values`
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\''
LINES TERMINATED BY '\r\n'
(sub_param_id,organization_id,bank_name,branch_name,account_number,
statement_type,parameter_name,parameter_value,created_by,#creation_date,updated_by,#updation_date)
SET creation_date = STR_TO_DATE(#creation_date, '%m/%d/%Y %H:%i'),
updation_date = STR_TO_DATE(#updation_date, '%m/%d/%Y %H:%i') ;
Import Successful !
You would want to modify those variables before loading :
Link
set column type to VARCHAR
LOAD DATA INFILE
UPDATE SET column = DATE_FORMAT( str_to_date(column, '%d/%m/%Y'), '%Y-%m-%d' )
set column type back to DATE
SHOW WARNINGS
Related
I have data with date format 1577234966837.
I uploaded this data in table via command :
load data infile 'C:/file.tsv'
into table table_1
fields terminated by '\t'
lines terminated by'\n'
ignore 1 lines (value, #timestamp)
set timestamp = FROM_UNIXTIME(#timestamp);
Command successful, but value in column timestamp is null. Ho to upload this format?
Your code looks right, but have problem with data type convertation.
In MySQL unixtime is number of second from 1970-01-01 00:00:00.
In your case number looks as JavaScript time in milliseconds, so for right convertion you should to divide the number by 1000
select from_unixtime(1577234966837); -- result is NULL
select from_unixtime(1577234966837/1000); -- result 2019-12-25 00:49:26.8370
DB fiddle link
So right import command should be like:
load data infile 'C:/file.tsv'
into table table_1
fields terminated by '\t'
lines terminated by'\n'
ignore 1 lines (value, #timestamp)
set timestamp = FROM_UNIXTIME(#timestamp/1000);
Hello I'm using LOAD DATA INFILE to populate a table in MySQL.
LOAD DATA INFILE 'test.txt'
INTO TABLE myTestTable
FIELDS TERMINATED BY '\t'
IGNORE 1 LINES;
Everything is working peachy except that there is a datetime column in my data that is formatted without any delimiter between the date and time sections. Like so:
SomeDateColumn
20050101081946
When I read this in, MySQL replaces all the dates with dummy values. Is there a way to have MySQL read this in correctly straight from a file?
Thanks!
You may call STR_TO_DATE when you run LOAD DATA, and convert the text date to a bona fide date on the fly:
LOAD DATA INFILE 'test.txt'
INTO TABLE myTestTable
FIELDS TERMINATED BY '\t'
IGNORE 1 LINES
(
col1, col2, #var1 -- list out all columns here
)
SET SomeDateColumn = STR_TO_DATE(#var1, '%Y%m%d%h%i%s');
I've got a MySQL load script that almost works, it is perfect except for the date columns, which are not in a MySql friendly format.
load data infile '/Users/pfarrell/sandbox/waybase/folklore/Titles_1976.csv'
into table fix76
fields terminated by ','
enclosed by '"'
ignore 1 lines
( patentId, USPatentNum, title, grantDate, filedDate)
The problem is that my dates are in mm/dd/yyyy format. Looks like the str_to_date
function is what I want, but I can't figure out how to use it in the load command.
I'm envisioning something like:
grantDate = STR_TO_DATE(something, '%m/%d/%Y'),
but that doesn't work.
You can load the date strings into user-defined variables, and then use STR_TO_DATE(#date, '%m/%d/%Y') to convert them to MySQL dates.
Try this:
load data infile '/Users/pfarrell/sandbox/waybase/folklore/Titles_1976.csv'
into table fix76
fields terminated by ','
enclosed by '"'
ignore 1 lines
( patentId, USPatentNum, title, #grantDate, #filedDate)
set grantDate = STR_TO_DATE(#grantDate, '%m/%d/%Y'),
filedDate = STR_TO_DATE(#filedDate, '%m/%d/%Y')
i am loading data from CSV file,it contains around 70k records ,in csv file date format for one column is like 10/24/2013 5:27:05 PM and in my table date formate is like(datetime 00-00-0000) ,so when i am uploading data form csv file using below command it was failed due to different time stamp.
LOAD DATA INFILE "/tmp/fulldata.csv" INTO TABLE test.customerorders COLUMNS TERMINATED BY ',' LINES TERMINATED BY '\n';
how can fix this time stamp in my table and how can i upload data in to the table?
You could convert the data like following:
SELECT STR_TO_DATE( '10/24/2013 5:27:05 PM', '%m/%d/%Y %I:%i:%s %p' )
and store it in a datetime field
edit:
LOAD data file can load pretty faster for 70K records.
LOAD DATA LOCAL INFILE '/filelocationfromroot' INTO TABLE `yourtablename`
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(
`tablefieldforfirstcsvcolumn`,`tablefieldforsecondcsvcolumn`,...,#yourdatecolumn,
)
SET `datacolumnname`=STR_TO_DATE( #yourdatecolumn, '%m/%d/%Y %I:%i:%s %p' )
I am using Load data in file query to insert csv into table. I have to format a date column inside the csv,
LOAD DATA INFILE '/invoices/invoice1381301986.csv' INTO TABLE invoice_table FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' (`code`,#var1) set datefield=STR_TO_DATE(#var1, '%m/%d/%Y');
I can format the date using the above query.
But the problem is, I have different formats for the csv date column. Possible formats are, "m/d/Y","m-d-Y", "m/d/y", "m-d-y", "Y-m-d", "Y/m/d".
so my query should be according to date format from the csv, so that I can modify my queries like,
datefield=STR_TO_DATE(#var1, '%m/%d/%Y')
OR
datefield=STR_TO_DATE(#var1, '%m-%d-%Y')..
How can I read the in which format the csv date field is?
you need to use a CASE :
LOAD DATA INFILE '/invoices/invoice1381301986.csv'
INTO TABLE invoice_table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(`code`,#var1)
SET datefield= (
CASE
WHEN #var1 REGEXP '[0-9]{2}/[0-9]{2}/[0-9]{4}' THEN STR_TO_DATE(#var1,'%m/%d/%Y')
...
END
)