I am trying to convert a database with all values stored as VARCHAR into the correct column type eg: INT, DATE, DECIMAL etc...
Problem is, that the columns date_order, date_billed and date_paid are stored in different formats
I still get a 1292 Truncated incorrect date value: '05/18/2011' errors and have pretty much no clue what to do since that exact date format is listed as CASE
My code:
SELECT
CAST(`ar_no` AS UNSIGNED),
CAST(`accession_id` AS UNSIGNED),
CAST(`client_id` AS UNSIGNED),
CAST(`insurance_id` AS UNSIGNED),
CAST(`test_id` AS UNSIGNED),
CASE
WHEN `date_paid` = '0' THEN `date_paid` = '00/00/0000'
WHEN LENGTH(DATE(STR_TO_DATE(`date_order`, '%m/%d/%y'))) IS NOT NULL THEN STR_TO_DATE(`date_order`, '%m/%d/%y')
WHEN LENGTH(DATE(STR_TO_DATE(`date_order`, '%m/%d/%Y'))) IS NOT NULL THEN STR_TO_DATE(`date_order`, '%m/%d/%Y')
END,
CASE
WHEN `date_paid` = '0' THEN `date_paid` = '00/00/0000'
WHEN LENGTH(DATE(STR_TO_DATE(`date_billed`, '%m/%d/%y'))) IS NOT NULL THEN STR_TO_DATE(`date_billed`, '%m/%d/%y')
WHEN LENGTH(DATE(STR_TO_DATE(`date_billed`, '%m/%d/%Y'))) IS NOT NULL THEN STR_TO_DATE(`date_billed`, '%m/%d/%Y')
END,
CASE
WHEN `date_paid` = '0' THEN `date_paid` = '00/00/0000'
WHEN LENGTH(DATE(STR_TO_DATE(`date_paid`, '%m/%d/%y'))) IS NOT NULL THEN STR_TO_DATE(`date_paid`, '%m/%d/%y')
WHEN LENGTH(DATE(STR_TO_DATE(`date_paid`, '%m/%d/%Y'))) IS NOT NULL THEN STR_TO_DATE(`date_paid`, '%m/%d/%Y')
END,
CAST(`amount_billed` AS DECIMAL(15,2)),
CAST(`amount_received` AS DECIMAL(15,2)),
CAST(`amount_adjusted` AS DECIMAL(15,2))
FROM `acs`.`billing_unformatted`;
Mysql only excepts dates in YYYY-MM-DD format. That is why you are getting errors such as 1292 Truncated incorrect date value: '05/18/2011'
You can change this in Excel before doing your import.
In order to change the date format in excel: right click on the top cell. Choose format cells from the drop down list. change the local to something like 'Afrikans'. Choose the format that looks like 2001-03-14. Use the top cell to fill down. Then save the document.
Just a quick note: Excel sometimes tries to do too much and will revert this column back to a the English(U.S) default time zone. So, if you plan on doing more editing make sure that the column has not reverted back.
Here is a link to more string literals on dev.mysql.
This stored procedure and post might help you as well: Error code 1292
Related
I am trying to analyze order_Date column and column have multiple date format i want to convert all those date in same format which wull make be easier to analyze the order_date.
I am trying to analyze the order_date however this column have multiple date format 2019/07/15 and 1/13/2014
Howeever, while converting different format date with one format yyyy/mm/dd with query.
select date_format(order_date, '%y/%m/%d'),orderid from superstore;
it shows null values like this.
i have tried to use `CAST as well but it shows every single value as null.
select case when order_date like '%Y' then date_format(order_date, '%Y/%m/%d') else null end as newdate from superstore;
date_format funtion is used to format a date datatype you should use https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_str-to-date any null values returned by str_to_date either failed or started as null. You will need to examine these and adjust the str_to_date parameters appropriately. There is a catch though is 20/2/20 y/m/d or d/m/y (for example) and how can you differentiate month and day where both are <=12?
For example
drop table if exists t;
create table t
(dt varchar(10));
insert into t values
('1/1/2020'),('2020/1/12'),('12/12/12'),(null),('13-14-15');
select dt,
case when length(substring_index(dt,'/',-1)) = 4 then str_to_date(dt,'%d/%m/%Y')
when length(substring_index(dt,'/',1)) = 4 then str_to_date(dt,'%Y/%m/%d')
when length(substring_index(dt,'/',1)) = 2 then str_to_date(dt,'%y/%m/%d')
else str_to_date(dt,'%y/%m/%d')
end dateformatted
from t;
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=553219f33ad9e9a4404fc4c0cb6571c9
note in no case can I identify month and day and sometimes year..
I'm having trouble inserting a field into the database, type conversion keeps timezone
UPDATE order o
SET o.data_envio = CONVERT( '2022-01-05T14:47:00-03:00', DATETIME );
1292 - Truncated incorrect datetime value: '2022-01-05T14:47:00-03:00'
You want to convert the timestamp with the STR_TO_DATE() function:
https://mariadb.com/kb/en/str_to_date/
Like this:
UPDATE order o
SET o.data_envio = STR_TO_DATE( '2022-01-05T14:47:00-03:00', '%Y-%m-%dT%H:%i:%s')
Only drawback here is that you will lose the timezone information.
This requires mysql 8; it will not work on 5.7 or earlier, or any version of mariadb (through at least 10.7.1).
Instead, you can do:
SET o.data_envio = convert_tz('2022-01-05T14:47:00','-03:00','+00:00')
Note that the first parameter cannot have the offset; it has to just be provided as the second parameter. If you have to remove it in your sql, not your code, and your dates are always well formed (no missing - or :, always two digits for each part except four for year), you can do:
UPDATE `order` o SET o.data_envio = CONVERT_TZ(
SUBSTRING('2022-01-05T14:47:00-03:00' FROM 1 FOR 19),
SUBSTRING('2022-01-05T14:47:00-03:00' FROM 20),
'+00:00'
)
I'm using MySQL version: 5.7.22
So I'm trying to have a table that contains a date column from string. The text field contains data in following DateTime format "d/m/YYYY h:m:s" format. e.g. "14/11/2018 20:10:04 +00:00".
I want to alter the table with a new column that is of the following format '%Y-%m-%d'. I get a
Data truncation: Truncated incorrect date value error
when I try to update the table. But I get the result when I just use a select statement to convert from string to date.
UPDATE BIG_DATA SET BIG_DATA.RealDate = ( SELECT x.d
From (SELECT (DATE_FORMAT(STR_TO_DATE(BIG_DATA.Date , '%d/%m/%Y'), '%Y-%m-%d')) as d
FROM BIG_DATA) as x);
Any help would be grateful!
The reason you are getting an error is that the warning for an incorrect date value (one that produces a value with zeros in it) that is produced by STR_TO_DATE on a SELECT is promoted to an error when you attempt to do an UPDATE. For example, if you do
SELECT STR_TO_DATE('14/11/2018 20:10:04 +00:00', '%d/%m/%Y');
SHOW WARNINGS
Your output will be:
2018-11-14
Warning 1292 Truncated incorrect date value: '14/11/2018 20:10:04 +00:00'
You can work around this by only supplying the date part (the leftmost 10 characters) of the string to STR_TO_DATE:
SELECT STR_TO_DATE(LEFT('14/11/2018 20:10:04 +00:00', 10), '%d/%m/%Y');
SHOW WARNINGS
Output is simply 2018-11-14
This then allows you to create your other column and UPDATE it from the date column:
ALTER TABLE big_data ADD
realdate DATE;
UPDATE big_data
SET realdate = STR_TO_DATE(LEFT(date, 10), '%d/%m/%Y');
SELECT * FROM big_data
Another possibility you might want to consider is using a generated column:
ALTER TABLE big_data ADD
realdate DATE AS (STR_TO_DATE(date, '%d/%m/%Y'));
SELECT * FROM big_data
In both cases the output is
date realdate
14/11/2018 20:10:04 +00:00 2018-11-14
Demo on dbfiddle
I have a column called "Time" this column is currently set to Varchar since i have some corrupted Time in some updates.Some Time has non ASCII characters etc. So how can i sort out all these corrupted and non properly formated time fields and set to NULL ? So that i can safely convert the Time column back to DateTime. The normal time fields in the updates are usually in the format of 2013-07-24 14:37:56
I was thinking of sorting it out by doing something like :
SELECT * FROM updates WHERE TIME not LIKE '....-..-.. ..:..:..'
But i don't know if that is the right regex approach and most efficient.
You could try using MySQL STR_TO_DATE on your varchar column:
update updates set `time` = null
where str_to_date(`time`,'%Y-%m-%d %H:%i:%s') is null;
So how can i sort out all these corrupted and non properly formated time fields and set to NULL ?
If CAST of such TIME column returns a NULL, then you can set it to NULL or a desired date time string value.
Example with Select:
mysql> select #ts:=cast( '2013-07╞ƒ~¥14:37:56' as datetime ) ts, #ts is null;
+------+-------------+
| ts | #ts is null |
+------+-------------+
| NULL | 1 |
+------+-------------+
Example for update:
update table_name
set time_coumn = null -- or a valid date time string value
where cast( time_column as datetime ) is null
Once you have set the column value to a NULL, you can then set valid datetime values for those records which have NULL values.
SELECT * FROM updates WHERE DATE_FORMAT(`TIME`, '%Y-%m-%d %H:%i:%s') IS NULL
Function DATE_FORMAT will try to format TIME in 'Y-m-d H:i:s' format, if function is not able to format TIME it will return NULL, so that's how you'll know which one is not in appropriate format and then you can handle them manually or set them to null with UPDATE query
I have a VARCHAR field completion_date that contains a timestamp date (ex 1319193919). Is it possible to run a query on such a field that compares it with NOW() ? I'm trying:
SELECT * FROM (`surveys`) WHERE `active` = '1' AND `archived` = '0' AND `completion_date` > 'NOW()'
But the results are not really what I'm expecting, is this cause of the VARCHAR? If so, what kind of date field am I better off using? The data must remain a Linux timestamp.
Convert NOW() to a timestamp using UNIX_TIMESTAMP()
SELECT *
FROM (`surveys`)
WHERE `active` = '1' AND `archived` = '0' AND `completion_date` > UNIX_TIMESTAMP(NOW())
Also, remove the quotes you had around 'NOW()'
mysql> SELECT UNIX_TIMESTAMP(NOW());
+-----------------------+
| UNIX_TIMESTAMP(NOW()) |
+-----------------------+
| 1319288622 |
+-----------------------+
N.B. In case you need it, the inverse of this function is FROM_UNIXTIME() to convert a timestamp into the default MySQL DATETIME format.
As mentioned in comments below, if you have the ability to make the change, it is recommended to use a real DATETIME type instead of VARCHAR() for this data.
A Linux timestamp can easily be stored in a BIGINT (or an UNSIGNED INT), which would make the type of comparisons you're trying to do possible. A VARCHAR is going to do a lexical, not numeric, comparison and which is NOT what you want. Using a BIGINT in conjunction with converting NOW() with UNIX_TIMESTAMP() should get you what you want.
It might even be better to store it using a DATETIME data type and do the conversion when you select the data. Storing it as a DATETIME future proofs your application in the event that you move to or add a different platform where a Linux timestamp isn't appropriate. Then you only need to modify your code, not convert your data to have it continue to work. I'm working on a project now where dates were stored as character data and it's been no end of problems getting the old data into shape to use with the new application, though you might experience fewer problems than us because you're storing a time stamp, not a formatted date, with its attendant typos.