I am receiving a Datetime field in my JSON which looks like the below screenshot
where first 4 characters are year, next two characters are month, next two characters are date, next two characters are hour, next two characters are minutes and last two characters are seconds.
I extracted the date successfully and tried to convert using to_timestamp function in SNOWFLAKE but returns me the result like
"2606-08-31 03:17:04.416"
code that I am working with to convert into Timestamp
select distinct to_timestamp(a.value) within group(order by to_timestamp(a.value)) from orders a
Could someone please help me with this?
The default format for conversion is AUTO, which will attempt to guess. It appears as though it is guessing incorrectly.
To specify a format, add a second parameter to your call to to_timestamp that is a string of the format. Here are the docs.
to_timestamp(a.value, 'YYYYMMDDHH24MISS')
Related
I'm loading a bunch of semi-structured data (JSON) into my database through Snowflake. The timestamp values in the entries are javascript timestamps that look like this:
"time": 1621447619899
Snowflake automatically converts this into a timestamp variable that looks like this:
53351-08-15 22:04:10.000.
All good so far. However, I think that the new timestamp is wrong. The actual datetime should by May 19, 2021 around 12pm MDT. Am I reading it wrong? Is it dependent on the timezone that my Snowflake instance is in?
When comparing the following options manually in SQL:
with x as (
SELECT parse_json('{time:1621447619899}') as var
)
SELECT var:time,
var:time::number,
var:time::varchar::timestamp,
1621447619899::timestamp,
'1621447619899'::timestamp,
var:time::timestamp
FROM x;
It appears that what you want to do is execute the following:
var:time::varchar::timestamp
Reviewing the documentation it does look like the to_timestamp is looking for the number as a string, so you need to cast to varchar first, and then cast to timestamp, otherwise you get what you are getting.
The question says that Snowflake transforms it to "53351-08-15 22:04:10.000" looks right, but it doesn't look right to me.
When I try the input number in Snowflake I get this:
select '1621447619899'::timestamp;
-- 2021-05-19T18:06:59.899Z
That makes a lot more sense.
You'll need to provide more code or context for further debugging - but if you tell Snowflake to transform that number to a timestamp, you'll get the correct timestamp out.
See the rules that Snowflake uses here:
https://docs.snowflake.com/en/sql-reference/functions/to_timestamp.html#usage-notes
The ::timestamp handles strings and numeric inputs differently. I.e. a string is added to 1970-01-01 as milliseconds (correct) whereas the numeric value is added in seconds which returns a date way in the future "53351-08-18 20:38:19.000".
SELECT TO_VARCHAR(1621447619899::timestamp) AS numeric_input
,'1621447619899'::timestamp AS string_input
numeric_input = 53351-08-18 20:38:19.000
string_input = 2021-05-19 18:06:59.899
Solutions are to convert to a string or divide by 1000:
SELECT TO_TIMESTAMP(time::string)
SELECT TO_TIMESTAMP(time/1000)
I have to send data from csv into SQL DB.
Problem starts when I try to convert data into Int. It wasnt my idea and I really cant do much with this datatype. When I'm trying to achieve this problem pop up:
Data Conversion 2: Data conversion failed while converting column
"pr_czas" (387) to column "C pr_dCz_id" (14). The conversion returned
status value 2 and status text "The value could not be converted
because of a potential loss of data.".
Tried already to ignore this problem but then another problems came up so there is no other way than solving this.
I have to convert this data from csv file which is str 50 into int 4
It must be int4. One of the requirements Dont know what t odo.
This is data I'm trying to put into int4. Look on pr_czas
This is data's datatype
Before I tried to do same thing with just DD.MM.YYYY but got same result...
Given an input column named [pr_czas] that contain string values that look like 31.01.2020 00:00 which appears to be a formatted date time represented in the format "DD.mm.YYYY HH:MM", I would like to express that as a whole number DDMMYYHHMM
Add a derived column to your data flow and call this new_pr_czas
The logic I'm going to use is a series of REPLACE statements and cast the final result to an integer. Replace the period, replace the colon and the space - all with nothing
(DT_I8)REPLACE(REPLACE(REPLACE([pr_czas], ".", ""), ":", ""), " ", "")
This is an easy case but things to note.
An integer/int32/I4 has a maximum value of 2 billion.
310120200000 is too large to fit into that space so you would need to make that an bigint/int64/I8. If I remember your previous question, you were having troubles with a lookup task so this data type mismatch might hurt you there.
The other thing to be aware of is that leading zeros will be dropped when converted to a number because they are not significant. If you need to retain the leading zeros, then you're working with string data type. This is an advantage to working with the ISO standard but if your data expects DD, then far be it for me to say otherwise.
If you need to slice your date into another format, then you'll want to have a few derived columns. The first one will generate a string column for each piece of pr_czas - year, month, day, hour and minute. You'll use the substring method for this and findstring to find the period space and colon.
The next data flow will be used to put those string pieces back into the new format and cast that to I8. Why? Because you can't debug doing it all in one shot but you can put a data viewer between two derived columns to figure out where a slice went awry.
i have a timestamp in my bigquery looking like this: 2017.09.25 10:22:19
i want to convert this string to a date dimension. i tried it with the dropdown menu, calculated fields like datetime, dateparse, date,... and a calculated field where i trimmed the string and took only parts of the date as a sum, but nothing is working. I always get the error that google bigquery could'nt compile my task: "Invalid date: '2017.07.03 10:52:16' "
does anyone have an idea as a solution for my problem?
regards
Date parts need to be separated with dashes, not dots, in order for the cast to work. For example,
'2017-09-25 10:22:19'
As a string, this is valid to cast both to a DATETIME and a TIMESTAMP type. If you want to convert your original string to one of these types, however, you can use PARSE_DATETIME, or similarly PARSE_TIMESTAMP:
SELECT
PARSE_DATETIME('%Y.%m.%d %T', timestamp_string) AS datetime
FROM YourTable;
I'm using Access DB 2007 - 2010; I've tried to import many CSV files but the timestamp column keeps failing to import correctly.
So I linked all of the CSV's to an Access DB and I'm trying to query all of the tables.
I'm trying to extract the year and day of the year from the time stamp (which is currently a string)
I'm trying to combine the Format with datepart functions and it keeps failing. (it just says error in the table)
The format function by itself works but I can't combine it with anything.
I'm basically trying to do this:
select datepart("y", Format(gmt, "dd-mmm-yyyy hh:nn:ss")) as DOY from Table1;
but it fails. I've also tried CDate and DateValue in different combinations but it all fails.
Does anyone know how to get this to work?
UPDATE
The format function isn't doing anything. The text remains the same no matter how I try to format it.
Here's a datetime sample: 05-Dec-2008 13:40:01.955
Access can't cope with the milliseconds in your date strings.
Use Left() to exclude them and feed the resulting substring to CDate().
SELECT CDate(Left(gmt, 20)) AS date_from_string
FROM Table1;
Once you have a valid Date/Time value, you can use Year(<Date/Time value>) or DatePart("yyyy", <Date/Time value>) to extract the year. And DatePart("y", <Date/Time value>) will give you the day of the year.
Just solve this issue, here is my code for your reference:
update tablename
set date=cdate(format(left(gmt,4)&"-"&right(gmt,2),"yyyy-mm"))
I've got a table setup which has populated data. In column "date", I have dates in the following format:
yyyymmdd i.e. 20131110
I have created a new field and called it newdate with the text format.
Then, I open up the SQL window and put the following in
UPDATE wl_daily
SET
newdate = UNIX_TIMESTAMP(date)
For some reason, it is running correctly, however it only outputs NULL to all the rows. Also, the column name is blank for some reason
Any suggestions?
That's because your field in a string and you're trying to add timestamp to it which is not a string. You need to use a valid datetime field like timestamp for this to work.
Advice: don't store dates and times as strings. Store them in their native format. It makes working with dates and times much easier.
While John Cronde's answer is correct - it doesnt help your situtation
UNIX_TIMESTAMP(STR_TO_DATE(`date`, '%Y%m%d'))
will do the conversion for example
SELECT UNIX_TIMESTAMP(STR_TO_DATE('20131111', '%Y%m%d'))
returns
unix_timestamp(STR_TO_DATE('20131111', '%Y%m%d'))
---------------------------------------------------
1384128000
You should only use this to convert your columns to the date specific columns. Converting each time you need a number will add load and slow down the query if used in production