Parsing JSON file into logstash - json

Question

Hi I am trying to send a json file with multiple objects to elasticsearch with the logstash so I can display the data using kibana. I have researched this extensively and simply cannot understand how to make the data formatted correctly to be used in kibana.
I have tried to use different filters such as: json, date, and grok
The issue is probably how I'm going about using these filters as I can't understand it's setup all to well.
Here is a sample line of the input json file:
{"time":"2015-09-20;12:13:24","bug_code":"tr","stacktrace":"543534"},
I want to use this format for displaying the data in kibana and sorting many objects according to their "time"
this following is what my current filter section is:
filter {
date {
match => ["time", "YYYY-MM-dd;HH:mm:ss Z" ]
timezone => "America/New_York"
locale => "en"
target => "#timestamp"
}
grok {
match => ["time", "%{TIMESTAMP_ISO8601:timestamp}"]
}
}
At this point I know the grok is wrong because I get "_grokparsefailure"
but how can I figure out the correct way to use grok or is there a simple way to sort the data using the given timestamp and not the processed timestamp given when sending the data through.
here is what the output currently shows:
"message" => "{\"time\":\"2015-09-20;12:13:24\",\"bug_code\":\"tr\",\"stacktrace\":\"543534\"},\r",
"#version" => "1",
"#timestamp" => "2015-11-23T09:54:50:274Z",
"host" => "<my_computer>",
"path" => "<path_to_.json>",
"type" => "json",
"tags" => [
[0] "_grokparsefailure"
any advice would be very much appreciated

Answer 1

You're almost there, I could get it working with a few tweaks.
First, you need to add the json{} filter in the first position. Then you need to change the date pattern to YYYY-MM-dd;HH:mm:ss and finally you can remove the grok filter at the end. You filter configuration would look like this:
filter {
json {
source => "message"
}
date {
match => ["time", "YYYY-MM-dd;HH:mm:ss" ]
timezone => "America/New_York"
locale => "en"
target => "#timestamp"
}
}
The parsed event for your sample JSON line would then look like this:
{
"message" => "{\"time\":\"2015-09-20;12:13:24\",\"bug_code\":\"tr\",\"stacktrace\":\"543534\"}",
"#version" => "1",
"#timestamp" => "2015-09-20T16:13:24.000Z",
"host" => "iMac.local",
"time" => "2015-09-20;12:13:24",
"bug_code" => "tr",
"stacktrace" => "543534"
}