Logstash 1.4.1 multiline codec not working - multiline

Question

I'm trying to parse multiline data from log file.
I have tried multiline codec and multiline filter.
but it doesn't work for me.
Log data
INFO 2014-06-26 12:34:42,881 [4] [HandleScheduleRequests] Request Entity:
User Name : user
DLR : 04
Text : string
Interface Type : 1
Sender : sdr
DEBUG 2014-06-26 12:34:43,381 [4] [HandleScheduleRequests] Entitis is : 1 System.Exception
and this is configuration file
input {
file {
type => "cs-bulk"
path =>
[
"/logs/bulk/*.*"
]
start_position => "beginning"
sincedb_path => "/logstash-1.4.1/bulk.sincedb"
codec => multiline {
pattern => "^%{LEVEL4NET}"
what => "previous"
negate => true
}
}
}
output {
stdout { codec => rubydebug }
if [type] == "cs-bulk" {
elasticsearch {
host => localhost
index => "cs-bulk"
}
}
}
filter {
if [type] == "cs-bulk" {
grok {
match => { "message" => "%{LEVEL4NET:level} %{TIMESTAMP_ISO8601:time} %{THREAD:thread} %{LOGGER:method} %{MESSAGE:message}" }
overwrite => ["message"]
}
}
}
and this is what I get when logstash parsing the multiline part
It just get the first line, and tag it as multiline.
the other lines not parsed!
{
"#timestamp" => "2014-06-27T16:27:21.678Z",
"message" => "Request Entity:",
"#version" => "1",
"tags" => [
[0] "multiline"
],
"type" => "cs-bulk",
"host" => "lab",
"path" => "/logs/bulk/22.log",
"level" => "INFO",
"time" => "2014-06-26 12:34:42,881",
"thread" => "[4]",
"method" => "[HandleScheduleRequests]"
}

Answer 1

Place a (?m) at the beginning of your grok pattern. That will allow regex to not stop at \n.

Answer 2

Not quite sure what's going on, but using a multiline filter instead of a codec like this:
input {
stdin {
}
}
filter {
multiline {
pattern => "^(WARN|DEBUG|ERROR)"
what => "previous"
negate => true
}
}
Does work in my testing...
{
"message" => "INFO 2014-06-26 12:34:42,881 [4] [HandleScheduleRequests] Request Entity:\nUser Name : user\nDLR : 04\nText : string\nInterface Type : 1\nSender : sdr",
"#version" => "1",
"#timestamp" => "2014-06-27T20:32:05.288Z",
"host" => "HOSTNAME",
"tags" => [
[0] "multiline"
]
}
{
"message" => "DEBUG 2014-06-26 12:34:43,381 [4] [HandleScheduleRequests] Entitis is : 1 System.Exception",
"#version" => "1",
"#timestamp" => "2014-06-27T20:32:05.290Z",
"host" => "HOSTNAME"
}
Except... the test file I used it never prints out the last line (because it's still looking for more to follow)