@inproceedings{rodriguez-boyd-graber-2021-evaluation,
    title = "Evaluation Paradigms in Question Answering",
    author = "Rodriguez, Pedro  and
      Boyd-Graber, Jordan",
    editor = "Moens, Marie-Francine  and
      Huang, Xuanjing  and
      Specia, Lucia  and
      Yih, Scott Wen-tau",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    address = "Online and Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2021.emnlp-main.758/",
    doi = "10.18653/v1/2021.emnlp-main.758",
    pages = "9630--9642",
    abstract = "Question answering (QA) primarily descends from two branches of research: (1) Alan Turing`s investigation of machine intelligence at Manchester University and (2) Cyril Cleverdon`s comparison of library card catalog indices at Cranfield University. This position paper names and distinguishes these paradigms. Despite substantial overlap, subtle but significant distinctions exert an outsize influence on research. While one evaluation paradigm values creating more intelligent QA systems, the other paradigm values building QA systems that appeal to users. By better understanding the epistemic heritage of QA, researchers, academia, and industry can more effectively accelerate QA research."
}