STR_TO_DATE(string_time,'%Y-%m-%d %H:%i:%s') > (select timestamp from table)
this line causes the error 'Function str_to_date not registered' in Athena. is there any way to work around this problem?
All databases have their own set of functions, even though some are common and exist in more than one. STR_TO_DATE is not available in Athena, but there are lots of other date and time functions that can be used to achieve the same goal.
You can find links to all functions supported by Athena here: https://docs.aws.amazon.com/athena/latest/ug/presto-functions.html
In your case I think you can use either parse_datetime, which looks like it works like STR_TO_DATE in your example.
Alternatively I think you could cast the string to a timestamp since the format you are using matches Athena's, try CAST(string_time AS TIMESTAMP)
Related
I'm loading a bunch of semi-structured data (JSON) into my database through Snowflake. The timestamp values in the entries are javascript timestamps that look like this:
"time": 1621447619899
Snowflake automatically converts this into a timestamp variable that looks like this:
53351-08-15 22:04:10.000.
All good so far. However, I think that the new timestamp is wrong. The actual datetime should by May 19, 2021 around 12pm MDT. Am I reading it wrong? Is it dependent on the timezone that my Snowflake instance is in?
When comparing the following options manually in SQL:
with x as (
SELECT parse_json('{time:1621447619899}') as var
)
SELECT var:time,
var:time::number,
var:time::varchar::timestamp,
1621447619899::timestamp,
'1621447619899'::timestamp,
var:time::timestamp
FROM x;
It appears that what you want to do is execute the following:
var:time::varchar::timestamp
Reviewing the documentation it does look like the to_timestamp is looking for the number as a string, so you need to cast to varchar first, and then cast to timestamp, otherwise you get what you are getting.
The question says that Snowflake transforms it to "53351-08-15 22:04:10.000" looks right, but it doesn't look right to me.
When I try the input number in Snowflake I get this:
select '1621447619899'::timestamp;
-- 2021-05-19T18:06:59.899Z
That makes a lot more sense.
You'll need to provide more code or context for further debugging - but if you tell Snowflake to transform that number to a timestamp, you'll get the correct timestamp out.
See the rules that Snowflake uses here:
https://docs.snowflake.com/en/sql-reference/functions/to_timestamp.html#usage-notes
The ::timestamp handles strings and numeric inputs differently. I.e. a string is added to 1970-01-01 as milliseconds (correct) whereas the numeric value is added in seconds which returns a date way in the future "53351-08-18 20:38:19.000".
SELECT TO_VARCHAR(1621447619899::timestamp) AS numeric_input
,'1621447619899'::timestamp AS string_input
numeric_input = 53351-08-18 20:38:19.000
string_input = 2021-05-19 18:06:59.899
Solutions are to convert to a string or divide by 1000:
SELECT TO_TIMESTAMP(time::string)
SELECT TO_TIMESTAMP(time/1000)
Asking for any ideas to convert this kind of date in SQL from May-15-2020 18:03 to 'yyyyMMddHHmiss' or 'yyyyMMdd'.
I am trying this query
select from_unixtime(unix_timestamp('May-15-2020 16:03', 'MM-dd-yyyy
HH:mi'), 'yyyyMMdd') from dual
but it wont work.
Use the right right function STR_TO_DATE, and use a format that matches your date rather than something scraped from a previous answer/blog.
Reference manuals of date and time functions are very useful for solving these basic problems.
I'm trying to convert a date type column into a nice human readable string like so: 25/11/2016 (or any other masks I'd like to use)
Does Big Query supports masks when using dates? When I use the Date() Functions it returns something like "2016-05-05" but that's not the standard pattern in many countries.
I've searched for a lot of different things the closes thing I got is this doc: https://cloud.google.com/bigquery/query-reference but I didn't see anythin that would help me
check STRFTIME_UTC_USEC
SELECT STRFTIME_UTC_USEC(CURRENT_DATE(), '%d/%m/%Y')
I've got this as the select part of my query:
SELECT cast(cast(exp_channel_titles.edit_date as char(14)) as datetime) AS Edit_Date
That takes data from a db in this format 20130501092128 and returns it in this format 2013-05-01 09:21:28
I can only assume it is some kind of magic as i don't fully understand how this works tbh.
But, i need to change the format of the date that it spits out to this format: %d/%m/%Y %k:%i:%s
I can honestly say i have no idea how to do this in that query, i've tried adding it as a param to datetime (is that even a mysql function?!?) but no joy and many other poor attempts that i wont go into.
If anyone can help, i'd be hugely grateful!
MySql automatically converts 20130501092128 to a date and time field, even if it is a VARCHAR or a INT, and you can just use this:
SELECT DATE_FORMAT(exp_channel_titles.edit_date, '%d/%m/%Y %k:%i:%s')
Please see fiddle here.
You can change output format using DATE_FORMAT() function from MySQL. Here is the documentation post about it.
https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_date-format
You can change the output format into whatever format you want, but if you recieve that data into an application, modifies it and return that data to server (editing a row for example). Remember to reformat it into a valid date for MySQL.
If you dont know how to do it, just have to do this into your query:
SELECT DATE_FORMAT(cast(cast(exp_channel_titles.edit_date as char(14))
as datetime), '%e/%m/%Y %k:%i:%s') AS Edit_Date
I am using SQLAlchemy to make database-independent querys.
I am facing one issue with to_char function.
Consider the simple query like:
select to_char(id,'999') from xyz
It's working on Postgres but MySQL doesn't support it.
How can I make this query database-independent using SQLAlchemy?
use the CAST function for simple type conversions. it is SQL standard and SQLAlchemy supports it directly via the cast() function: http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/expressions.html?highlight=cast#sqlalchemy.sql.expression.cast .
for date values, SQLA has an extract() function that produces the SQL EXTRACT function - a translation layer translates common fieldnames like "month", "day", "minute", etc. into the appropriate keyword for the backend in use.
Use the cast function, for example
cast(numberfield as char)
You're looking for format in MySQL:
select format(id, '999') from xyz
To answer your all-in-one question: There's no cross-database way to do this. Every RDBMS is very different, and there's no real standardization past the fairly simple queries. This is especially true with string manipulations.