How to tail multiple files in fluentd - json

Question

I have setup fluentd logger and I am able to monitor a file by using fluentd tail input plugin. All the data is received by fluentd is later published to elasticsearch cluster. Below is the configuration file for fluentd:
<source>
#type tail
path /home/user/Documents/log_data.json
format json
tag myfile
</source>
<match *myfile*>
#type elasticsearch
hosts 192.168.48.118:9200
user <username>
password <password>
index_name fluentd
type_name fluentd
</match>
As you can see I am monitoring log_data.json file by using tail. I also have a file in the same directory log_user.json, I want to monitor it also and publish it logs to elasticsearch. To do this, I thought of creating another <source> & <match> with different tag but it started showing errors.
How can I monitor multiple files in fluentd and publish them to elasticsearch. I see when we start fluentd its worker is started. Is it possible to start multiple worker so that each one of them is monitoring different files, or any other way of doing it. Can anyone point me to some good links/tutorials.
Thanks.

Answer 1

You can use multiple source+match tags.
Label can help you to bind them.
Here an example:
<source>
#label #mainstream
#type tail /home/user/Documents/log_data.json
format json
tag myfile
</source>
<label #mainstream>
<match **>
#type copy
<store>
#type elasticsearch
host elasticsearch
port 9200
logstash_format true
logstash_prefix fluentd
logstash_dateformat %Y%m%d
include_tag_key true
type_name access_log
tag_key #log_name
<buffer>
flush_mode interval
flush_interval 1s
retry_type exponential_backoff
flush_thread_count 2
retry_forever true
retry_max_interval 30
chunk_limit_size 2M
queue_limit_length 8
overflow_action block
</buffer>
</store>
</match>
</label>