Splunk Enterprise: large json events not indexed - json

Question

I have a sourcetype defined like this (system\local\props.conf):
[my_json]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = json
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = Timestamp
category = Structured
description = json
disabled = false
pulldown_type = 1
TIME_FORMAT = HH:mm:ss.fff
LINE_BREAKER = ([\r\n]+)
limits.conf:
[spath]
# Number of characters to read from an XML or JSON event when
# auto extracting.
extraction_cutoff = 5000
extract_all = true
If I try to index following json (no line breaks, I just formated it here):
{
"Timestamp": "19:51:27.757",
"Level": "INFO",
"EventType": "Audit",
"EventId": "ApiServiceInvocationResponse",
"ThreadId": "19",
"Method": "TXXXX1234.Common.WCF.ParameterInspector.AfterCall",
"Context": {
"PhoneNumber": "48600000000",
"ApplicationId": "7C217CF0CC45E0292623203E56AD87EC",
"ApiType": "android",
"ApiVersion": "6.0",
"AppVersion": "1.0.debug",
"UserId": 25714,
"SessionId": 1440538,
"CorrelationId": "98ccaec5-4d23-4c5f-b5da-7ce0e440f2e3"
},
"Payload": {
"Operation": "Initialize",
"Response": {
"Message": null,
"CanRun": true,
"PhoneNumber": null,
"DefaultPhoneNumber": "48600000000",
"DriverPhoneNumber": null,
"RegisterationPhoneNumber": null,
"Favourites": null,
"SessionId": "4DC24EB6E4B0261DD03CDD4F6A7C7DC8",
"IsFriendlyCustomer": true,
"OptionsAvailable": [],
"MaxOrderDate": null,
"FavouriteDriverNumber": null,
"ShareMessage": null,
"PaymentInstruments": [],
"InAppPaymentAvailable": true,
"HasActiveOrders": false,
"UserName": "some name",
"UserPhone": "48600000000",
"ApplicationId": "",
"KioskInfo": null,
"CallResult": {
"Code": "SSREA",
"Message": null
}
},
"truncate": false
},
"Message": null,
"Exception": null
}
it gets properly indexed. But the following one does NOT get indexed:
{
"Timestamp": "16:31:27.074",
"Level": "INFO",
"EventType": "Audit",
"EventId": "ApiServiceInvocationResponse",
"ThreadId": "5",
"Method": "TXXXX1234.Common.WCF.ParameterInspector.AfterCall",
"Context": {
"PhoneNumber": "48600000000",
"ApplicationId": "A70BAFD855CE7120A8E331E27D39E645",
"ApiType": "MOCK",
"ApiVersion": "1.0",
"AppVersion": null,
"UserId": 11852,
"SessionId": 448107,
"CorrelationId": "28d9cc6f-c207-4199-9c24-ac6c4b4cfc8e"
},
"Payload": {
"Operation": "Initialize",
"Response": {
"Message": "message",
"CanRun": false,
"PhoneNumber": "48600000000",
"DefaultPhoneNumber": "48600000000",
"DriverPhoneNumber": "",
"RegisterationPhoneNumber": null,
"Favourites": [],
"SessionId": "0778662D04444C9456694B3FAB44F8C6",
"IsFriendlyCustomer": true,
"OptionsAvailable": [
"PaymentCard",
"Combi",
"SevenSeats",
"Animal",
"AirContition"
],
"MaxOrderDate": "2019-01-30 16:31",
"FavouriteDriverNumber": null,
"ShareMessage": "some long share message. http://www.sharing.net.pl/",
"PaymentInstruments": [],
"InAppPaymentAvailable": false,
"HasActiveOrders": false,
"UserName": "some name",
"UserPhone": "48600000000",
"ApplicationId": "",
"KioskInfo": null,
"CallResult": {
"Code": "SSREA",
"Message": "Zwr󣮯 klucz sesji dla zarejestrowanego uzytkownika"
}
},
"truncate": false
},
"Message": null,
"Exception": null
}
UPDATE:
This is what I have found in logs:
01-02-2019 20:40:31.780 +0100 ERROR JsonLineBreaker - JSON StreamId:9928927958268928125 had parsing error:Unexpected character while parsing backslash escape: 'x' - data_source="C:\Logs\Txxx.log", data_host="WIN-BP2MBISNI04", data_sourcetype="my_json"

Answer 1

Try adding CHARSET = UTF-8 to the props.conf stanza.