I am trying to do CDC on pentaho 8.3 version Community edition, Source is Oracle and the target is Mysql.
The date format in Oracle is in the format DD-MON-YY (01-JAN-19), but when I see the data in PDI(Preview)
it appears as 2019/01/01 00:00:00.00000000.
Source query:
Select ColA, ColB
from table
where Last_upd_dt > ? and last_upd_dt <=?
in the table input step and getting the start_date and end_date from the Get System info step (Start_date as start date range, end_date as end date range) and in the Transformation Properties >> Logging>> Transformation gave a Log_table along with connections. But when I execute the transformation I am getting below errors:
Error: Unable to write the log record to log table[Log]
Error: org.pentaho.di.core.exception.KettleDatabaseException:
Error inserting/updating row
Data truncation: Incorrect datetime value: '1900-01-01 03:00:00.0' for Column Start_date' at row 1
Caused by: com.mysql.cj.jdbc.execeptions.MysqlDataTruncation:Data truncation: Incorrect datetime value: '1900-01-01 03:00:00.0' for Column Start_date' at row 1
That's because date time format problem.Place select value between input and output and define date format dd-MMM-yyyy or which is supported by output step in meta-data tab.
Related
I am using Grafana 8.3.4
In the Dashboard Settings->Variables, I am using MySQL query as the data source.
Even though the values are stored in datetime format in database, it is getting converted to unixtime in variables.
For eg: values in the database - 2022-01-20 05:39:34
Values returned by Grafana - 1642657174000
How to get the values as datetime format?
What's wrong?
A new column (in Excel) was made for calculating Duration
Using formula to subtract values from 2 cells
For example: 2021-01-01 07:30 - 2021-01-01 07:00; Duration = 00:30:00
In MySQL table, I set the data type for the column as TIME, but it would return with error when importing
Error message: 'Error Code: 1292. Incorrect datetime value: '00:13:00 ' for column 'ride_length'
What I have tried:
Changing data type to DATETIME, INT, TIME -> None worked
Changing the data format to HH:MM:SS, HHMMSS, 'YYYY-MM-dd HH:MM:SS' in CSV -> None worked
Question
What kind of data type I should set in MySQL?
If it was the CSV file's problem, what kind of 'data format' I need to set?
Thank you very much
Actually, it might be easier for you to just import the two datetime values and then use TIMSTAMPDIFF inside MySQL:
SELECT TIMESTAMPDIFF(MINUTE, '2021-01-01 07:00:00', '2021-01-01 07:30:00')
-- 30
Note that because the difference is actually a computed quantity, it generally makes more sense to compute it when you select. This way, should one of the two values be updated later, you don't have to worry about maintaining the difference column.
MySQL version: 8.0.23-0ubuntu0.20.04.1 - (Ubuntu)
When running sample query:
SELECT * FROM `redacted-tbl`
WHERE `redacted-col` = 'some-invalid-date'
ORDER BY `redacted-col` DESC LIMIT 0, 25
data structure: redacted-col DATE
I'm getting #1525 - Incorrect DATE value: 'some-invalid-date' error.
Now I understand that 'some-invalid-date' is definitely not a valid mysql date format. I understand that the error is expected behavior if it's an INSERT or UPDATE query.
But why do I get such error on SELECT query? Previous version of mysql didn't throw such error for SELECT query (only for INSERT/UPDATE).
Also, how do I turn off this error for SELECT-ing DATE column? Is there any mysql flags to disable such check?
Edit (added from my comment):
In my opinion, there are good reasons to allow comparison of non-valid-date-string with DATE columns:
querying with WHERE mydatecol > '2015' to get all date that is after '2015-01-01'
even better, I can just pass user inputted date as filter (sanitized and parameter-bind-ed of course): WHERE mydatecol > ?,
if user enter 2015 then it will become shorthand for user who cares only to get all records after 2015
if user enter 2015-04, then it will become shorthand for user who want records after 2015 month 04/April)
if user enter 2015-04-15 (normal/valid mysql date string), then app will display records after 2015 month 04/April date 15
without this "non-date-validated comparison", I would have to write more application code just to check if the user inputted valid date or not, e.g.:
if the input format is 2015 then I have to change it into 2015-01-01,
else if the input format is 2015-04 then I have to change it into 2015-04-01,
else if the input format is 2015-04-15 then it's valid,
else it's not valid and throw error (or just output current date/default date or just show 'no entry matched your search criteria')
[The text of this answer was originally written by forpas https://stackoverflow.com/users/10498828/forpas ]
You can cast mydatecol to a string to perform the comparison. An easy way to do it is with CONCAT():
WHERE CONCAT(mydatecol) > '2015'
or with cast:
WHERE CAST(redacted-col AS CHAR) > 2015
MySQL 5.6 - we had a talk with one of the developers and found something very curious - couldn't find anything about it in google:
create temporary table md (dates date);
insert into md select '2000-01-01'; -- result: 2000-01-01
insert into md select '2000-00-01'; -- result: 1999-12-01 ????
insert into md select '2000-00-00'; -- result: 1999-11-30 ????
insert into md select '2000-01-00'; -- result: 1999-12-31 ????
insert into md select '2000-13-01'; -- result: EXPECTED: SQL Error [1292] [22001]: Data truncation: Incorrect date value: '2000-13-01' for column 'dates' at row 1
insert into md select '2000-12-49'; -- result: EXPECTED: SQL Error [1292] [22001]: Data truncation: Incorrect date value: '2000-12-49' for column 'dates' at row 1
Could someone please explain?
Why does it let this happen? It should throw the same Data issue error imho.
Documentation?
What is the exact logic behind this?
a) Month 00 previous month same day?
b) Day 00 previous month last day?
c) both 00-00 previous month's previous' last day?
Thanks!
https://dev.mysql.com/doc/refman/8.0/en/using-date.html
MySQL permits you to store dates where the day or month and day are zero. This is convenient if you want to store a birthdate in a DATE column and you know only part of the date. To disallow zero month or day parts in dates, enable the NO_ZERO_IN_DATE mode.
I'm using MySQL version: 5.7.22
So I'm trying to have a table that contains a date column from string. The text field contains data in following DateTime format "d/m/YYYY h:m:s" format. e.g. "14/11/2018 20:10:04 +00:00".
I want to alter the table with a new column that is of the following format '%Y-%m-%d'. I get a
Data truncation: Truncated incorrect date value error
when I try to update the table. But I get the result when I just use a select statement to convert from string to date.
UPDATE BIG_DATA SET BIG_DATA.RealDate = ( SELECT x.d
From (SELECT (DATE_FORMAT(STR_TO_DATE(BIG_DATA.Date , '%d/%m/%Y'), '%Y-%m-%d')) as d
FROM BIG_DATA) as x);
Any help would be grateful!
The reason you are getting an error is that the warning for an incorrect date value (one that produces a value with zeros in it) that is produced by STR_TO_DATE on a SELECT is promoted to an error when you attempt to do an UPDATE. For example, if you do
SELECT STR_TO_DATE('14/11/2018 20:10:04 +00:00', '%d/%m/%Y');
SHOW WARNINGS
Your output will be:
2018-11-14
Warning 1292 Truncated incorrect date value: '14/11/2018 20:10:04 +00:00'
You can work around this by only supplying the date part (the leftmost 10 characters) of the string to STR_TO_DATE:
SELECT STR_TO_DATE(LEFT('14/11/2018 20:10:04 +00:00', 10), '%d/%m/%Y');
SHOW WARNINGS
Output is simply 2018-11-14
This then allows you to create your other column and UPDATE it from the date column:
ALTER TABLE big_data ADD
realdate DATE;
UPDATE big_data
SET realdate = STR_TO_DATE(LEFT(date, 10), '%d/%m/%Y');
SELECT * FROM big_data
Another possibility you might want to consider is using a generated column:
ALTER TABLE big_data ADD
realdate DATE AS (STR_TO_DATE(date, '%d/%m/%Y'));
SELECT * FROM big_data
In both cases the output is
date realdate
14/11/2018 20:10:04 +00:00 2018-11-14
Demo on dbfiddle