Dynamic logstash json mapping in elasticsearch - json

Question

I need to implement rest service that will take any typed data in request and put it to elasticsearch through logstash.
Spring controller receives request body:
public class CustomData {
private String component;
private Object data;
}
Data is any custom json from PUT request.
I try to utilize net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder
Logstash config is as follows:
input {
tcp {
port => 5000
codec => json
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
index => "%{indexName}-%{+YYYY.MM.dd}"
}
}
As you can see there is a indexName parameter - I set it with MDC. But after message goes through logstash, elastic says there is no mapping for my object:
{
"appId1-2016.03.15" : {
"aliases" : { },
"mappings" : {
"logs" : {
"properties" : {
"#timestamp" : {
"type" : "date",
"format" : "strict_date_optional_time||epoch_millis"
},
"#version" : {
"type" : "long"
},
"host" : {
"type" : "string"
},
"port" : {
"type" : "long"
}
}
},
"e1" : {
"properties" : {
"#timestamp" : {
"type" : "date",
"format" : "strict_date_optional_time||epoch_millis"
},
"#version" : {
"type" : "long"
},
"applicationId" : {
"type" : "string"
},
"host" : {
"type" : "string"
},
"level" : {
"type" : "string"
},
"level_value" : {
"type" : "long"
},
"logger_name" : {
"type" : "string"
},
"message" : {
"type" : "string"
},
"port" : {
"type" : "long"
},
"thread_name" : {
"type" : "string"
},
"type" : {
"type" : "string"
},
"userId" : {
"type" : "string"
}
}
}
},
"settings" : {
"index" : {
"creation_date" : "1458053563829",
"number_of_shards" : "5",
"number_of_replicas" : "1",
"uuid" : "w5y7GPd-Sk65gdpBQ_IKow",
"version" : {
"created" : "2020099"
}
}
},
"warmers" : { }
}
}
There's only message with string type.
stdout from logstash is:
elasticsearch_1 | [2016-03-16 04:38:49,018][INFO ][cluster.metadata ] [Ripfire] [json_encoder-events-2016.03.16] create_mapping [ef]
elasticsearch_1 | [2016-03-16 04:38:49,108][INFO ][cluster.metadata ] [Ripfire] [json_encoder-events-2016.03.16] update_mapping [logs]
elasticsearch_1 | [2016-03-16 04:38:49,174][INFO ][cluster.metadata ] [Ripfire] [json_encoder-events-2016.03.16] update_mapping [ef]
logstash_1 | 2016-03-16T04:38:48.197Z 10.27.13.228 Initializing Spring FrameworkServlet 'dispatcherServlet'
logstash_1 | 2016-03-16T04:38:48.197Z 10.27.13.228 FrameworkServlet 'dispatcherServlet': initialization started
logstash_1 | 2016-03-16T04:38:48.219Z 10.27.13.228 FrameworkServlet 'dispatcherServlet': initialization completed in 22 ms
logstash_1 | 2016-03-16T04:38:48.412Z 10.27.13.228 {"field1":"value","field2":40000}
logstash_1 | 2016-03-16T04:38:48.423Z 10.27.13.228 {"field1":"value","field2":40000}
logstash_1 | 2016-03-16T04:38:48.457Z 10.27.13.228
logstash_1 | request payload={
logstash_1 | "type": "ef",
logstash_1 | "userId": "ASD",
logstash_1 | "data": {
logstash_1 | "field1": "value",
logstash_1 | "field2": 40000
logstash_1 | }
logstash_1 | }
logstash_1 |
logstash_1 | response payload=null
Is there any way to get data field mapped according to it's structure? Thank you so much for suggestions...