How to convert epoch to timestamp in NiFi within a JSON file? - json

Question

I'm having issues with getting an epoch conversion to Timestamp to work properly. So far my example timestamp looks like the following:
{"createTime": 1510932843000}
What my end goal is to make it look like the following:
2017-11-17 3:34:03.000
The things I've tried so far are the UpdateRecord and JoltTransformation Processor. For the UpdateRecord I have tried various ways but all end in an error. The current code I have for this is:
${field.value:format("yyyy-MM-dd HH:mm:ss.SSS")}
Which results in the following error:
JSON Object due to java.lang.NumberFormatException: For input string: "2017-11-17 15:34:03.000": For input string: "2017-11-17 15:34:03.000"
I have also tried the code without the multiply(1000) to the same effect.
I have also tried a Jolt Transformation of the following code:
{
"createTime": "${createTime:append('000'):format('yyyy-MM-dd HH:mm:ss.SSS')}"
}
This however results in the following:
"createTime": "1970-01-01 00:00:00.000"
Which isn't what I'm looking for as its the incorrect date result. Am I doing something wrong within my code itself or is another factor occurring? I've been working with this and searching all over for different kind of results and have tried multiple different formats with no success. Any help with this would be greatly appreciated!

Answer 1

My preferred solution:
Use a ScriptedTransformRecord processor:
Record Reader: JsonTreeReader
Record Writer: JsonRecordSetWriter
Script Language: Groovy
Script Body:
import java.time.Instant;
import java.time.format.DateTimeFormatter;
import java.time.ZoneId;
def formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS").withZone(ZoneId.of("Europe/Bratislava"))
record.setValue("createTimeFormatted", formatter.format(Instant.ofEpochMilli(record.getAsLong("createTime"))))
return record
Output json:
{
"createTime" : 1510932843000,
"createTimeFormatted" : "2017-11-17 16:34:03.000"
}
Another approach
Use 2 processors: JoltTransformJSON (convert type from Long to String) -> UpdateRecord (convert date).
JoltTransformJSON processor:
Jolt Transformation DSL: Chain
Jolt specification:
[
{
"operation": "modify-overwrite-beta",
"spec": {
"createTime": "=toString"
}
}
]
UpdateRecord processor:
Record Reader: JsonTreeReader
Record Writer: JsonRecordSetWriter
Replacement Value Strategy: Literal Value
/createTime (dynamic property): ${field.value:format("yyyy-MM-dd HH:mm:ss.SSS")}
Output json:
{
"createTime" : "2017-11-17 16:34:03.000"
}