This question already has answers here:
mySQL convert varchar to date
(3 answers)
Closed 2 years ago.
i have a table in a csv file with date format MM-DD-YYYY. I will like to import the whole table into mysql. To do so, I believe I will need to create a table first. Since I am unable to import that table because of the date format not being recognized by SQL, I created the field as a varchar field.
Now that I have imported the table in, I will like to convert this varchar column into a date column (YYYY-MM-DD) that I can run operations on (such as using the YEAR() function). Can someone teach me how? I am new to SQL.
Thanks!
If you "import CSV into MySQL" then you use LOAD DATA INFILE.
If your CSV contains date column in a format which is not recognized by MySQL then you must use SET-variant of LOAD DATA INFILE query.
CREATE TABLE imported__from_csv ( .. ,
datetime_column DATETIME,
.. );
LOAD DATA ..
INFILE ..
INTO TABLE imported__from_csv ( .. , #datetime_variable, .. )
..
SET .. ,
datetime_column = STR_TO_DATE( #datetime_variable, '%d-%m-%Y'),
.. ;
I.e. we load the value which has incorrect format into user-defined variable, then we convert the value to correct date literal, and finally we save the value into the table.
Related
I am using load data syntax to load a csv file to a table.The file is same format as hive accepts. But still after load data is issued, Last 2 columns returns null on select.
1750,651,'2013-03-11','2013-03-17'
1751,652,'2013-03-18','2013-03-24'
1752,653,'2013-03-25','2013-03-31'
1753,654,'2013-04-01','2013-04-07'
create table dattable(
DATANUM INT,
ENTRYNUM BIGINT,
START_DATE DATE,
END_DATE DATE )
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ;
LOAD DATA LOCAL INPATH '/path/dtatable.csv' OVERWRITE INTO TABLE dattable ;
Select returns NULL values for the last 2 cols
Other question was what if the date format is different than YYYY-MM-DD. is it possible to make hive identify the format? (Because right now i am modifying the csv file format to accept by hive)
Answer to your 2nd question:
You will need an additional temporary table to read your input file, and then you can do date conversions in your insert select statements.In your temporary table store date fields as string. Ex.
create table dattable_ext(
DATANUM INT,
ENTRYNUM BIGINT,
START_DATE String,
END_DATE String)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
Load data into temporary table
LOAD DATA LOCAL INPATH '/path/dtatable.csv' OVERWRITE INTO TABLE dattable_ext;
Insert from temporary table to the managed table.
insert into table dattable select DATANUM, ENTRYNUM,
from_unixtime(unix_timestamp(START_DATE,'yyyy/MM/dd'),'yyyy-MM-dd'),
from_unixtime(unix_timestamp(END_DATE,'yyyy/MM/dd'),'yyyy-MM-dd') from dattable_ext;
You can replace date format in unix_timestamp function with your input date format.
LasySimpleSerDe (default) does not work with quoted CSV. Use CSVSerDe:
create table dattable(
DATANUM INT,
ENTRYNUM BIGINT,
START_DATE DATE,
END_DATE DATE )
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
WITH SERDEPROPERTIES (
"separatorChar" = ",",
"quoteChar" = "'"
)
STORED AS TEXTFILE;
Also read this: CSVSerDe treats all columns to be of type String
Define you date columns as string and apply conversion in select.
CSV date format is DD/MM/YYYY like this 16/11/2016. All the date become 0000-00-00 in MySQL. How to solve this differences?
As of now I can think of two solutions to you problem:
Create a additional VARCHAR field to insert those values (like 16/11/2016), and create a TRIGGER on INSERT to update the date field by converting the string date to 'YYYY-mm-dd' type.
These links may help
https://dev.mysql.com/doc/refman/5.5/en/trigger-syntax.html
http://www.w3resource.com/mysql/date-and-time-functions/mysql-str_to_date-function.php
In this step also, Create a additional VARCHAR field to insert those values (like 16/11/2016) and after import is done run a UPDATE query to update your date field using SET dateField = convertDate(dateFromSheet)
This question already has an answer here:
mysql YYYY-MM-DDThh:mm:ss
(1 answer)
Closed 6 years ago.
I want to upload my csv file that contains yyyy-mm-ddThh:mm:ss.sssZ data.
When I set DATETIME type in MySQL,
I got error code 1292.
MySQL How can I upload yyyy-mm-ddThh:mm:ss.sssZ type successfully?
First you need to convert your string dates into a valid date time format (yyyy-mm-dd hh:mm:ss).
With the help of STR_TO_DATE() and DATE_FORMAT() functions you can convert those date strings into the above desired format.
Now you can safely change/modify the datatype to timestamp/datetime.
Here's a demonstration:
SQL FIDDLE DEMO
Convert string dates into valid date format:
Create table yourtable(
id INT primary key AUTO_INCREMENT,
start varchar(50)
);
INSERT INTO yourtable(start) VALUES('1901-02-03T05:30:00.000Z');
UPDATE yourtable
SET start = DATE_FORMAT(STR_TO_DATE(start,'%Y-%m-%dT%H:%i:%s.000Z'),'%Y-%m-%d %H:%i:%s');
Change the datatype:
ALTER TABLE yourtable MODIFY COLUMN start datetime;
I am trying to create a table in MySQL, and load the data into it which i have as a txt file
the date format in the txt file is dd/mm/yyyy with few dates as 12/12/1988 and few as 1/2/1988
really confused how to give value for the date column while creating the table ?
Please help, I am a beginner with MySQL.
MySQL Supports Dates in Format 'YYYY-MM-DD' i.e '2014-01-28'
You can load the date strings into user-defined variables, and then use a function to convert them to MySQL dates.
Example
I have a table Like below
CREATE TABLE TestTable
(
patentId INT,
USPatentNum INT,
title CHAR(10),
grantDate DATE,
filedDate DATE
);
Now I need to load string dates into DATE column we can do like
load data local infile '/home/abdul/Test.csv'
into table TestTable
fields terminated by ','
enclosed by '"'
ignore 1 lines
( patentId, USPatentNum, title, #grantDate, #filedDate)
set grantDate = STR_TO_DATE(#grantDate, '%m/%d/%Y'),
filedDate = STR_TO_DATE(#filedDate, '%m/%d/%Y')
Im using LOAD DATA INFILE to import a csv file, the files date format is 29/11/2010 and the database format is 2010-11-29, what can i use to format the date inside the query?
I've tried str_to_date:
SET date_start = STR_TO_DATE(#from_date,'%Y-%m-%d'),
but that only inserts 0000-00-00
MySQL 4.x
LOAD DATA will try to insert your dates as they are. It isn't aware about format and in common case you can not apply some post-processing to your fields (well, different from some format which is allowed inside LOAD DATA syntax itself) - and you can not adjust your values via SET keyword like in MySQL 5.x
Instead you can do following steps:
Declare your table's column as VARCHAR. Let it name be record_date
Do your LOAD DATA query. It will load your dates into record_date column
Add new column to your table, let it be temp_date - with type DATE: ALTER TABLE t ADD temp_date DATE
Update your temp_date column: UPDATE t SET temp_date = STR_TO_DATE(record_date, '%d/%m/%Y')
Drop your VARCHAR date column: ALTER TABLE t DROP record_date
Finally, rename column with correct DATE type to original one: ALTER TABLE t CHANGE temp_date record_date DATE
As result, you'll have your dates loaded into your table as DATE date type. Replace record_date to the name which your original column has.
MySQL 5.x
You can use SET keyword and natively replace procedure, described above. So just do:
LOAD DATA INFILE 'file.csv'
INTO TABLE t
(#date)
SET record_date=STR_TO_DATE(#date, '%d/%m/%Y')
-sample above is for one column and you'll need to add others (if they exist). Date column name is also record_date - so change it to actual name too.
Try something like
update tablename SET date_start = date_format(#from_date,'%Y-%m-%d')
Use DATE_FORMAT() . It will Formats the date value according to the format string. Mysql
date_format(#from_date,'%Y-%m-%d')