Objective
I'm trying to load a .csv file called persondata into DB alldata TABLE persondata and then run a query on it for firstname, lastname and dateofbirth (dob). The .csv has only 4 records in it.
Date format in csv is MM/DD/YYYY, the output should be YYYY/MM/DD, YYYY-MM-DD or YYYYMMDD.
SQL
LOAD DATA LOCAL INFILE 'C:/Users/john.smith/Desktop/persondata.csv'
INTO TABLE alldata.persondata
FIELDS TERMINATED BY ','
(firstname, lastname, dob, apptdate, icd9, cpt)
SET dob = str_to_date(#dob, '%c/%e/%Y')
;
SELECT firstname, lastname, dob
FROM alldata.persondata
Problem and Error Message I'm Getting
firstname and lastname return proper values but dob returns null for all 4 records. In the csv file, the first three colums (A, B, C) are firstname, lastname, dob. So same order as in the table persondata.
Error:
4 row(s) affected, 8 warning(s): 1265 Data truncated for column 'dob' at row 1 1411 Incorrect datetime value: '19850708' for function str_to_date 1265
Help pages I consulted:
(using str_to_date in general)
How to convert csv date format to into mysql db
(using 'SET column = str_to_date...')
MySql load data infile STR_TO_DATE returning blank?
(other)
How to change string date to MySQL date format at time of import of CSV using MySQL's LOAD DATA LOCAL INFILE
Cannot transform mm/dd/yyyy in excel to csv yyyymmdd date format using SSIS
MySQL str_to_date produces NULL despite valid formatting
Additional Information:
I experimented with this query and a bunch of variations of it but no luck:
SET dob = date_format(str_to_date(#dob, '%c/%e/%Y'), '%Y/%c/%e')
I'm not seeing a huge amount of consensus on how to write this. Some people specify the output format of %Y/%d/%m and some don't. But isn't that the only date format that mysql supports? This makes me think I shouldn't have to write it. Not sure if this is even related. I've seen a few syntaxes of the entire thing. I've read through all the support pages and I think that I understand the 'SET' command.
*this is my first post on stackoverflow so please let me know if should present anything differently
You're using "dob" where you should be using "#dob" in your column list... line 4 should be:
(firstname, lastname, #dob, apptdate, icd9, cpt)
^^^^
This is because mysql reads the date into a VARIABLE (indicated by the #), and then the later SET command manipulates the variable to match the actual column (which is identified correctly dob without the #).
Also, I think you're using slashes in your string format when the date doesn't appear to have slashes in it. That is, the error says the date is "19850708" (July 8th, 1985). I believe you want:
%Y%m%d
That is, change line 5 to:
SET dob = str_to_date(#dob, '%Y%m%d')
%m and %d are the 2-digit month and day, which you need since you have '07' instead of just '7' for July, for example. After your comments I'm not sure which date format is correct, but certainly the #dob vs dob variable issue is real. You may want to replace every instance of #dob with something like #original_dob just to alleviate future confusion.
See this page: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_str-to-date
Related
I have customers data translated into DB table in which few cell values in column type Varchar, contains date and few doesnt. The ones that contains Date in certain format, need to be converted into another one which will be further used for data analysis jobs. I tried below query but it shows errors for the cells which doesnt have date. How do I resolve this?
Basically I want to strip off the actual time and append 000000 in place of actual time, along with the date.
Query:
UPDATE Table1
SET C1 = DATE_FORMAT(C1,'%Y%m%d000000')
WHERE DATE_FORMAT(C1,'%Y%m%d%H%i%s') AND C1 IS NOT NULL;
Error:
Code : 1292
Truncated incorrect datetime value: 'EVN'
You need to use STR_TO_DATE() to parse the date that's in the VARCHAR column.
You can use a regular expression to test if the column contains a date in MM/DD/YY format.
UPDATE Table1
SET C1 = DATE_FORMAT(STR_TO_DATE('%m/%d/%y', C1), '%Y%m%d000000')
WHERE C1 RLIKE '^[01][0-9]/[0-3][0-9]/[0-9][0-9]$'
Here is my code:
CREATE TABLE A
(`ID` INT NULL,
`DATE` DATE NULL,
`NUM` INT NULL
);
LOAD DATA LOCAL INFILE "fakepath/file.csv"
INTO TABLE A
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(ID,DATE,NUM)
SET
DATE = str_to_date(#DATE, '%Y%m%d');
The original data in csv file is like this-- 20160101,20160102,20160103 (the date is different). After I execute the code, all the date in the DATE column become one day value such as 2016-01-02 in table A.
Why do this happen? I have other table which used the same code(different column name)
How can I fix it? Thank you!
You have to tell MySQL to load the from-csv data value into a variable first:
LOAD DATA LOCAL INFILE "fakepath/file.csv"
[..snip..]
IGNORE 1 LINES
(ID,DATE,NUM)
^---table field, **NOT** a variable
SET
DATE = str_to_date(#DATE, '%Y%m%d');
^---variable never gets populated
Try
(ID, #DATE, NUM)
^--note this
instead. That'll load the id/num values directly into the table, but puts your date value into the variable, which you can use afterwards in the SET portion of the query.
The fact that you actually get a date value put into the table with a proper date format indicates that somwhere else, in previous code, you did set a #DATE variable, and it's simply being re-used in this query. But since you don't CHANGE that variable's value in this query, you end up using the SAME date value for all records.
I'm trying to import a XML file into a MySQL Table. In the XML file there is a timestamp in <CurrentTime> in the following format:
2016-01-26T09:52:19.3420655+01:00
This timstamp should go into the corresponding DATETIME CurrentTime column in my Table. So I did the following
LOAD XML INFILE 'xxx.xml'
INTO TABLE test.events
ROWS IDENTIFIED BY '<Event>'
SET CurrentTime = str_to_date(CurrentTime, '%Y-%m-%dT%H:%i:%s.%f');
But it quits with the error
Error Code: 1292. Incorrect datetime value: '2016-01-25T16:22:24.1840792+01:00' for column 'CurrentTime' at row 1
So it seems it doesn't convert the string at all. Why?
I think that error is thrown when the string value from the file is loaded directly to the column. The error is thrown before you get to the SET clause.
Here's an abbreviated example of how to use user-defined variables to pass the value of a field down to the SET, bypassing the assignment to the column.
Note that the columns _row and account_number are populated directly from the first two fields in the file. The later fields in the file are assigned to user-defined variables (identifiers beginning with #.
The SET clause evaluates the user-defined variables, and assigns the result of the expression to the actual column in the table.
In this example, the "dates" were formatted YYYYMMDD. I used the STR_TO_DATE() function to have that string converted to a proper DATE.
I abbreviated this sample somewhat, but it demonstrates the approach of reading field values into user-defined variables.
CREATE TABLE _import_water
(`_row` INT
,`account_number` VARCHAR(255)
,`total_due` DECIMAL(18,2)
,`end_date` DATE
,`start_date` DATE
,`ccf` DECIMAL(18,4)
)
LOAD DATA LOCAL INFILE '//server/share$/users/me/mydir/myfile.csv'
INTO TABLE _import_water
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(_row
,account_number
,#total_due
,#end_date
,#start_date
,#ccf
)
SET `total_due` = NULLIF(#total_due,'')
, `end_date` = STR_TO_DATE(#end_date,'%Y%m%d')
, `start_date` = STR_TO_DATE(#start_date,'%Y%m%d')
, `ccf` = NULLIF(#ccf,'')
Also, it doesn't look like there's any problem with your STR_TO_DATE, it seems to evaluate just fine.
testing...
SELECT STR_TO_DATE('2016-01-25T16:22:24.1840792+01:00','%Y-%m-%dT%H:%i:%s.%f') AS mydatetime
returns:
mydatetime
--------------------------
2016-01-25 16:22:24.184079
I have a set of code that I'm coverting from Postgres to work on MySQL (PHPmyAdmin) which already has a section of inserts in the following type of format:
INSERT INTO ORD (TOTAL, SHIPDATE, ORDID, ORDERDATE, CUSTID, COMMPLAN)
VALUES ('101.4', '08-JAN-87', '610', '07-JAN-87', '101', 'A');
Since MySQL doesn't like the date as '08-JAN-87' it just ends up displaying 0000-00-00 in the date columns.
From the searches I've done so far the only solutions given seem to be converting it during the select statement so it displays correctly. I would like to know a method of changing the data itself. (not just a select statement)
This is for a uni assignment and the lecturer could only advise manually changing all the insert statments to a format it will accept. This might work for this case but in the long run or for a larger data dump this won't be possible.
You need to modify the import so that the data string is converted for MySQL
INSERT INTO ORD (TOTAL, SHIPDATE, ORDID, ORDERDATE, CUSTID, COMMPLAN)
VALUES ('101.4', STR_TO_DATE('08-JAN-87', '%d-%b-%y'), '610', STR_TO_DATE('07-JAN-87', '%d-%b-%y'), '101', 'A');
This tells mysql that the string 08-JAN-87 is a date in the format DD-MMM-YY and to convert it to a valid date datatype.
Good Morning All;
I currently have a MySQL table where there are 3 date fields (Columns) that were loaded as strings in this format 20140101 YYYYmmdd. I would like to convert this to a date format 2014/01/01 YYYY/mm/dd. Can someone please provide a simple sql syntax that would alter the table to a date format from a string and change the column to display the dates like this 2014/01/01 and not like 20140101. Thanks to all
Try this:
date_format(str_to_date(datecolumn, '%Y%m%d'),'%Y/%m/%d')
If you just want to reformat the values in the VARCHAR column, assuming that the column with sufficient length e.g. VARCHAR(10), and all the values are eight characters in length...
You could do something like this:
UPDATE mytable t
SET t.mycol = CONCAT( LEFT( t.mycol ,4)
, '/'
, SUBSTR( t.mycol ,5,2)
,'/'
, SUBSTR( t.mycol ,7,2)
)
WHERE CHAR_LENGTH(t.mycol) = 8
We want something in the statement that will prevent the statement from "working" a second time, if it's inadvertently re-run. It doesn't have to be CHAR_LENGTH. We might want to include a check that the value doesn't already contain a slash character AND t.mycol NOT LIKE '%/%'.
But why on earth are "date" values being stored in character columns, rather than in DATE datatype, which is custom designed for storing and working with date values?
ALTER TABLE mytable MODIFY mycol DATE ... ;
(If the column is defined as NOT NULL, has a default value, has a comment, those attributes can be retained, they need to be included in the new column specification, e.g.
ALTER TABLE mytable MODIFY mycol DATE NOT NULL COMMENT 'creation date';
Note that DATE columns do not have a "format" per se. When converting to string, MySQL uses date format '%Y-%m-%d'. And MySQL expects string literals representing date values to be in that same format. To get a value from a DATE column converted to string in format 'yyyy/mm/dd'.
SELECT DATE_FORMAT(date_col,'%Y/%m/%d') AS date_col
To get a string value in that format converted to DATE datatype
SELECT STR_TO_DATE('2015/06/01','%Y/%m/%d')