We are using Swisscom application cloud (based on Cloudfoundry) and the provided Kibana/Logstash/Elasticsearch service. Now we d like to log in JSON format from our applications to logstash.
Thats why we integrated Logstash formatter into our wildfly swarm apps and since then they log in JSON format like:
{"#version":1,"#timestamp":"2018-07-24T18:28:51.291+0200","sequence":15299,"loggerClassName":"org.jboss.as.server.logging.ServerLogger_$logger","loggerName":"org.jboss.as.server.deployment","level":"INFO","threadName":"MSC service thread 1-2","message":"WFLYSRV0027: Starting deployment of \"hospush.war\" (runtime-name: \"hospush.war\")","threadId":31,"mdc":{},"ndc":""}
I also added a filter.conf to the logstash app on swisscom appcloud with the following content:
filter {
json {
source => "message"
}
}
When I check the logs of logstash now I can see that it throws an error and no logs are transferred to Kibana.
2018-07-24 20:33:15 [APP/PROC/WEB/0] OUT [2018-07-24T18:33:15,202][WARN ][logstash.filters.json ] Error parsing json {:source=>"message", :raw=>"<14>1 2018-07-24T18:33:15.018187+00:00 HosPush.demo-test.demo-test 3694b57f-bc05-459a-880a-17c174fc6d7c [APP/PROC/WEB/0] - - {\"#version\":1,\"#timestamp\":\"2018-07-24T20:33:15.017+0200\",\"sequence\":3796,\"loggerClassName\":\"org.slf4j.impl.Slf4jLogger\",\"loggerName\":\"com.hospush.business.escalation.EscalationService\",\"level\":\"INFO\",\"threadName\":\"EJB default - 3\",\"message\":\"Found 0 patientNeeds with no open notification.\",\"threadId\":154,\"mdc\":{},\"ndc\":\"\"}\n", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
2018-07-24 20:33:15 [APP/PROC/WEB/0] OUT at [Source: (byte[])"<14>1 2018-07-24T18:33:15.018187+00:00 HosPush.demo-test.demo-test 3694b57f-bc05-459a-880a-17c174fc6d7c [APP/PROC/WEB/0] - - {"#version":1,"#timestamp":"2018-07-24T20:33:15.017+0200","sequence":3796,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"com.hospush.business.escalation.EscalationService","level":"INFO","threadName":"EJB default - 3","message":"Found 0 patientNeeds with no open notification.","threadId":154,"mdc":{},"ndc":""}
2018-07-24 20:33:30 [APP/PROC/WEB/0] OUT [2018-07-24T18:33:30,117][WARN ][logstash.filters.json ] Error parsing json {:source=>"message", :raw=>"<14>1 2018-07-24T18:33:30.002777+00:00 HosPush.demo-test.demo-test 3694b57f-bc05-459a-880a-17c174fc6d7c [APP/PROC/WEB/0] - - {\"#version\":1,\"#timestamp\":\"2018-07-24T20:33:30.001+0200\",\"sequence\":3797,\"loggerClassName\":\"org.slf4j.impl.Slf4jLogger\",\"loggerName\":\"com.hospush.business.escalation.OrphanEscalationScheduler\",\"level\":\"INFO\",\"threadName\":\"EJB default - 4\",\"message\":\"orphan escalation scheduler called...\",\"threadId\":155,\"mdc\":{},\"ndc\":\"\"}\n", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
2018-07-24 20:33:30 [APP/PROC/WEB/0] OUT at [Source: (byte[])"<14>1 2018-07-24T18:33:30.002777+00:00 HosPush.demo-test.demo-test 3694b57f-bc05-459a-880a-17c174fc6d7c [APP/PROC/WEB/0] - - {"#version":1,"#timestamp":"2018-07-24T20:33:30.001+0200","sequence":3797,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"com.hospush.business.escalation.OrphanEscalationScheduler","level":"INFO","threadName":"EJB default - 4","message":"orphan escalation scheduler called...","threadId":155,"mdc":{},"ndc":""}
My guess is, that because of source => "message" logstash parses the message property as JSON which failes. What it should do is parsing the whole "root object" as json instead of only the message property.
Could that be and if yes, how do I need to adjust the filter.conf to make it work?
Thx a lot for your help guys.
Probably not relevant, but I guess the filter here is incorrect.
This says that their is a JSON structure in the message field that should be parsed as JSON. As I see, I do not see any message field, so this will not work.
I'm using JSON Validation Schema component and I have notice, that it logs all the errors to the console.
I would like to avoid this error message being displayed in the console.
Even though I have chosen a special exception strategy which has catch exception strategy with JSONValidation Exception and has custom logic implemented and no loggers at all in it, I still see the following error message:
org.mule.api.MessagingException: Json content is not compliant with schema
com.github.fge.jsonschema.core.report.ListProcessingReport: failure
--- BEGIN MESSAGES ---
error: string "blah" is too long (length: 4, maximum allowed: 3)
level: "error"
schema: {"loadingURI":"file:/...}
instance: {"pointer":"/blah_blah_code"}
domain: "validation"
keyword: "maxLength"
value: "blah"
found: 4
maxLength: 3
--- END MESSAGES ---
How could I make mule omit this error message? I don't want these errors to be logged to the console.
You can set the logException attribute of the catch-exception-strategy element to false, forcing mule not to log errors to the console:
<catch-exception-strategy logException="false">
set below logger to false in log4j2.xml
<AsyncLogger name="org.mule.module.apikit.validation.RestJsonSchemaValidator" level="OFF"/>
Help me with multiple response body with single status code in yaml format.
Ex::
'401':
description: Not Found
headers:
Content-Type :
type: string
default: application/json;charset=UTF-8
examples:
errorCode: '48'
errorLabel: RONotAllowed
errorDescription: Unable to execute the remote operation as it is not allowed for the vehicle
'401':
description: Not Found
headers:
Content-Type :
type: string
default: application/json;charset=UTF-8
examples:
errorCode: '45'
errorLabel: VehicleNotFound
errorDescription: Vehicle could not be found using VIN provided in request
'401':
description: Not Found
headers:
Content-Type :
type: string
default: application/json;charset=UTF-8
examples:
errorCode: '49'
errorLabel: VehicleAlreadyOnline
errorDescription: Unable to execute the remote operation since vehicle is already online
Using above code i found error
✖ YAML Syntax Error
Duplicated mapping key at line 608, column 9: '401'
If I understand this correctly you are trying to provide a list of examples for the error code. For this you need:
'401':
description: Not Found
headers:
Content-Type :
type: string
default: application/json;charset=UTF-8
examples:
- errorCode: '48'
errorLabel: RONotAllowed
errorDescription: 'Unable to execute the remote operation as it is not allowed for the vehicle'
- errorCode: '45'
errorLabel: VehicleNotFound
errorDescription: 'Vehicle could not be found using VIN provided in request'
- errorCode: '49'
errorLabel: VehicleAlreadyOnline
errorDescription: 'Unable to execute the remote operation since vehicle is already online'
I have a problem with this code:
GraphHopperAPI gh = new GraphHopperWeb();
gh.load("http://localhost:8989/api/route");
GHResponse ph = gh.route(new GHRequest(45.104546,7.69043,45.104546,7.69043));
It gives me this error:
2014-03-29 09:33:00,036 [main] INFO graphhopper.http.GraphHopperWeb - Full request took:0.037406, API took:0.0
Exception in thread "main" java.lang.RuntimeException: Problem while fetching path 45.104546, 7.69043->45.104546, 7.69043
at com.graphhopper.http.GraphHopperWeb.route(GraphHopperWeb.java:119)
at provaMain.main(provaMain.java:23)
Caused by: org.json.JSONException: A JSONObject text must begin with '{' at character 0
at org.json.JSONTokener.syntaxError(JSONTokener.java:410)
at org.json.JSONObject.<init>(JSONObject.java:179)
at org.json.JSONObject.<init>(JSONObject.java:402)
at com.graphhopper.http.GraphHopperWeb.route(GraphHopperWeb.java:95)
... 1 more
The documentation currently undergoes a change (moving it from wiki to source). Where did you find that snippet? Please try gh.load("http://localhost:8989/"); for the latest branch and gh.load("http://localhost:8989/api"); before.
Please anyone has an idea what is causing this error while checking in files on tfs:
"The type initializer for 'Microsoft.TeamFoundation.Framework.Server.ByteArray' threw an exception."
this is the stack trace from the event Viewer:
**TF53010: The following error has occurred in a Team Foundation component or extension:
Date (UTC): 4/7/2014 5:57:14 AM
Machine: TFSSRV
Application Domain: /LM/W3SVC/8080/ROOT/tfs-14-130413234339367965
Assembly: Microsoft.TeamFoundation.Framework.Server, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; v2.0.50727
Service Host: 61e34d8a-3995-492a-b0b1-b1ca43304366 (SABIS)
Process Details:
Process Name: w3wp
Process Id: 3896
Thread Id: 4844
Account name: INTEREDLB\eghanem
Detailed Message: TF30065: An unhandled exception occurred.
Web Request Details
Url: tfssrv:8080/tfs/_tfs_resources/VersionControl/v1.0/upload.ashx [method: POST]
User Agent: Team Foundation (devenv.exe, 10.0.30319.1)
Headers: Content-Length=125836&Content-Type=multipart%2fform-data%3b+boundary%3d--------------------------8e5m2D6l5Q4h6&Accept-Language=en-US&Authorization=NTLM+TlRMTVNTUAADAAAAGAAYAIQAAABcAVwBnAAAABIAEgBYAAAADgAOAGoAAAAMAAwAeAAAAAAAAAD4AQAABYKIogYC8CMAAAAPEamXFnqofvmag9Tv7MumY0kATgBUAEUAUgBFAEQATABCAGUAZwBoAGEAbgBlAG0ASQBUAFEAQwAwADIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgPJ7o8t9UpE1fWPVvOiQkwEBAAAAAAAAOSm6OSZSzwGLmc3OzEbnlAAAAAACABIASQBOAFQARQBSAEUARABMAEIAAQAMAFQARgBTAFMAUgBWAAQAJgBpAG4AdABlAHIAZQBkAGwAYgAuAHMAYQBiAGkAcwAuAG4AZQB0AAMANAB0AGYAcwBzAHIAdgAuAGkAbgB0AGUAcgBlAGQAbABiAC4AcwBhAGIAaQBzAC4AbgBlAHQABQASAHMAYQBiAGkAcwAuAG4AZQB0AAgAMAAwAAAAAAAAAAEAAAAAIAAAPo4lTXR7%2fzg5WQsBAME%2f5oXFLmv2XYbJUNBj1fLeBfkKABAAAAAAAAAAAAAAAAAAAAAAAAkAPgBIAFQAVABQAC8AdABmAHMAcwByAHYALgBpAG4AdABlAHIAZQBkAGwAYgAuAHMAYQBiAGkAcwAuAG4AZQB0AAAAAAAAAAAA&Expect=100-continue&Host=tfssrv%3a8080&User-Agent=Team+Foundation+(devenv.exe%2c+10.0.30319.1)&X-TFS-Version=1.0.0.0&X-TFS-Session=f705371b-1a30-4e4a-861c-62d652add5e3&TF-Instance=f705371b-1a30-4e4a-861c-62d652add5e3
Path: /tfs/_tfs_resources/VersionControl/v1.0/upload.ashx
Local Request: False
Host Address: 192.168.200.234
User: INTEREDLB\eghanem [authentication type: NTLM]
Exception Message: The type initializer for 'Microsoft.TeamFoundation.Framework.Server.ByteArray' threw an exception. (type TypeInitializationException)
Exception Stack Trace: at Microsoft.TeamFoundation.Framework.Server.ByteArray..ctor(Int32 sizeRequested)
at Microsoft.TeamFoundation.VersionControl.Server.UploadHandler.UploadFile(VersionControlRequestContext versionControlRequestContext, String workspaceName, String workspaceOwner, String serverItem, Byte[] hash, Stream fileStream, Int64 fileLength, Int64 compressedLength, Int64 offsetFrom, CompressionType compressionType)
at Microsoft.TeamFoundation.VersionControl.Server.TeamFoundationVersionControlService.UploadFile(TeamFoundationRequestContext requestContext, String workspaceName, String workspaceOwner, String serverItem, Byte[] hash, Stream fileStream, Int64 fileLength, Int64 compressedLength, Int64 offsetFrom, CompressionType compressionType)
at Microsoft.TeamFoundation.VersionControl.Server.UploadHandler.Execute()
Inner Exception Details:
Exception Message: Exception of type 'System.OutOfMemoryException' was thrown. (type OutOfMemoryException)
Exception Stack Trace: at Microsoft.TeamFoundation.Framework.Server.BufferPool..ctor(Int32 bufferSize, Int32 initialNumberOfEntries)
at Microsoft.TeamFoundation.Framework.Server.ByteArray..cctor()
For more information, see Help and Support Center at**
I was encountering the same issue just today. For the sake of others that may hit this page, since it's the first hit when googling the exception. I looked at the MS Social question here, but the key appears to be something in the App Tier Framework, I executed IISRESET.exe (as Administrator) on the TFS app tier server and everything started working perfectly again.