Nxlog - how to process multiline log with multiple headers - multiline

Question

I would like to pre-process following log structure with nxlog and send it then to graylog.
My custom app log structure:
timestamp;field1;field2; ---- Start of good event ----
timestamp;field3;field4;field5;field6
timestamp;field7;field8;field9;field10
timestamp;field11;field12; --- End of good event ---
timestamp;FAIL;field13;field14
timestamp;FAIL;field15;field16
The GELF output from nxlog should contain full_message with "good event" or "bad event".
"good event" = 1 line as follows:
timestamp;field1;field2; ---- Start of good event ----;timestamp;field3;field4;field5;field6;timestamp;field7;field8;field9;field10;timestamp;field11;field12; --- End of good event ---
"bad event" should contain 1 line as follows:
timestamp;FAIL;field13;field14; timestamp;FAIL;field15;field16
I have no problem to parse "good event" with xm_multiline and define it's HeaderLine and EndLine.
But I have absolutely no idea, how to parse two different multilines. Could you give me any hint, please?
Is it possible to use if-else statement with "InputType"? I mean "if condition1 then InputType good-event and some-actions else InputType bad-event and some-actions". Or it needs totally different approach - e.g. no xm_multiline usage but some kind of regex magic?
Thanks in advance.

Answer 1

You can still use xm_multiline. You just need to define the two different patterns with regex.
Since you didn't provide your configuration I'll use my configuration for a different log format as an example.
I have a java application I need to monitor the logs for that doesn't use consistent time formatting, so messages might look like this:
2019-04-24 00:00:13,952 WARN [SemaphoreArrayListManagedConnectionPool] (QuartzScheduler_quartzScheduler-wildflyapp0201401_ClusterManager) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:336)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:343)
Or like this:
14:00:34,426 INFO [stdout] (default task-73) com.xyz.england.idserver.comp.impl.Service DEBUG [Get][db113034-ecc6-4c0d-86f2-moo3e33942f2] Job Package id.
14:00:34,426 INFO [stdout] (default task-73) [DEBUG 2019-04-24 14:00:34,426] [Get][db113034-ecc6-4c0d-86f2-moo3e33942f2] Job Package id.
14:00:34,427 INFO [stdout] (default task-39) com.xyz.england.idserver.comp.impl.Service DEBUG [Get][0c4d63c0-74d7-4599-bc40-mooa84cf62ea] Job Package id.
14:00:34,427 INFO [stdout] (default task-39) [DEBUG 2019-04-24 14:00:34,425] [Get][0c4d63c0-74d7-4599-bc40-mooa84cf62ea] Job Package id.
If the log used one or the other time formats I could have used one of these two configurations:
<Extension java_multiline>
Module xm_multiline
HeaderLine /^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d /
</Extension>
OR
<Extension java_multiline>
Module xm_multiline
HeaderLine /^\d\d:\d\d:\d\d,\d\d\d/
</Extension>
Since that wasn't the case I had to include them in a single statement using alternation, specifically using the pipe symbol aka the OR operand:
<Extension java_multiline>
Module xm_multiline
HeaderLine /^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d |^\d\d:\d\d:\d\d,\d\d\d /
</Extension>
Using this regex statement either time format will match as my header line.