@inproceedings{shukla-etal-2021-noobs,
    title = "Noobs at {S}emeval-2021 Task 4: Masked Language Modeling for abstract answer prediction",
    author = "Shukla, Shikhar  and
      Sarthak, Sarthak  and
      Arya, Karm Veer",
    editor = "Palmer, Alexis  and
      Schneider, Nathan  and
      Schluter, Natalie  and
      Emerson, Guy  and
      Herbelot, Aurelie  and
      Zhu, Xiaodan",
    booktitle = "Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2021.semeval-1.107/",
    doi = "10.18653/v1/2021.semeval-1.107",
    pages = "805--809",
    abstract = "This paper presents the system developed by our team for Semeval 2021 Task 4: Reading Comprehension of Abstract Meaning. The aim of the task was to benchmark the NLP techniques in understanding the abstract concepts present in a passage, and then predict the missing word in a human written summary of the passage. We trained a Roberta-Large model trained with a masked language modeling objective. In cases where this model failed to predict one of the available options, another Roberta-Large model trained as a binary classifier was used to predict correct and incorrect options. We used passage summary generated by Pegasus model and question as inputs. Our best solution was an ensemble of these 2 systems. We achieved an accuracy of 86.22{\%} on subtask 1 and 87.10{\%} on subtask 2."
}