@inproceedings{garneau-lamontagne-2021-shared,
    title = "Shared Task in Evaluating Accuracy: Leveraging Pre-Annotations in the Validation Process",
    author = "Garneau, Nicolas  and
      Lamontagne, Luc",
    editor = "Belz, Anya  and
      Fan, Angela  and
      Reiter, Ehud  and
      Sripada, Yaji",
    booktitle = "Proceedings of the 14th International Conference on Natural Language Generation",
    month = aug,
    year = "2021",
    address = "Aberdeen, Scotland, UK",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2021.inlg-1.26/",
    doi = "10.18653/v1/2021.inlg-1.26",
    pages = "266--270",
    abstract = "We hereby present our submission to the Shared Task in Evaluating Accuracy at the INLG 2021 Conference. Our evaluation protocol relies on three main components; rules and text classifiers that pre-annotate the dataset, a human annotator that validates the pre-annotations, and a web interface that facilitates this validation. Our submission consists in fact of two submissions; we first analyze solely the performance of the rules and classifiers (pre-annotations), and then the human evaluation aided by the former pre-annotations using the web interface (hybrid). The code for the web interface and the classifiers is publicly available."
}