AllenNLP) Is there a way to set config for evaluation, epeicially for reader? - allennlp

Question

I am new to allenNLP library.
In order to set args for dataset_reader, I want to set config for evaluation, like train (https://github.com/allenai/allennlp-template-config-files/blob/master/training_config/my_model_trained_on_my_dataset.jsonnet)
But I am not sure if there is a config file template for evaluation, like train, and the config file below works (where train_data_path and trainer parts are deleted.)
{
"dataset_reader" : {
// This name needs to match the name that you used to register your dataset reader, with
// the call to `#DatasetReader.register()`.
"type": "classification-tsv",
// These other parameters exactly match the constructor parameters of your dataset reader class.
"token_indexers": {
"tokens": {
"type": "single_id"
}
}
},
"validation_data_path": "/path/to/your/validation/data/here.tsv",
"model": {
// This name needs to match the name that you used to register your model, with
// the call to `#Model.register()`.
"type": "simple_classifier",
// These other parameters exactly match the constructor parameters of your model class.
"embedder": {
"token_embedders": {
"tokens": {
"type": "embedding",
"embedding_dim": 10
}
}
},
"encoder": {
"type": "bag_of_embeddings",
"embedding_dim": 10
}
},
"data_loader": {
// See http://docs.allennlp.org/master/api/data/dataloader/ for more info on acceptable
// parameters here.
"batch_size": 8,
"shuffle": true
},
}
Thanks in advance.

Answer 1

#petew's answer is correct. allennlp evaluate does not read a configuration file. It uses the configuration file that is stored with the model. Run allennlp evaluate -h to get more info.
If you need the dataset reader to behave differently at evaluation time, use the validation_dataset_reader field in the configuration file. This will take effect both during training (for evaluations at the end of the epoch), and later, when you run allennlp evaluate.