@inproceedings{zhang-etal-2020-reasoning,
    title = "Reasoning about Goals, Steps, and Temporal Ordering with {W}iki{H}ow",
    author = "Zhang, Li  and
      Lyu, Qing  and
      Callison-Burch, Chris",
    editor = "Webber, Bonnie  and
      Cohn, Trevor  and
      He, Yulan  and
      Liu, Yang",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2020.emnlp-main.374/",
    doi = "10.18653/v1/2020.emnlp-main.374",
    pages = "4630--4639",
    abstract = "We propose a suite of reasoning tasks on two types of relations between procedural events: goal-step relations ({\textquotedblleft}learn poses{\textquotedblright} is a step in the larger goal of {\textquotedblleft}doing yoga{\textquotedblright}) and step-step temporal relations ({\textquotedblleft}buy a yoga mat{\textquotedblright} typically precedes {\textquotedblleft}learn poses{\textquotedblright}). We introduce a dataset targeting these two relations based on wikiHow, a website of instructional how-to articles. Our human-validated test set serves as a reliable benchmark for common-sense inference, with a gap of about 10{\%} to 20{\%} between the performance of state-of-the-art transformer models and human performance. Our automatically-generated training set allows models to effectively transfer to out-of-domain tasks requiring knowledge of procedural events, with greatly improved performances on SWAG, Snips, and Story Cloze Test in zero- and few-shot settings."
}