@inproceedings{hosseini-etal-2022-knowledge,
    title = "Knowledge-Augmented Language Models for Cause-Effect Relation Classification",
    author = "Hosseini, Pedram  and
      Broniatowski, David A.  and
      Diab, Mona",
    editor = "Bosselut, Antoine  and
      Li, Xiang  and
      Lin, Bill Yuchen  and
      Shwartz, Vered  and
      Majumder, Bodhisattwa Prasad  and
      Lal, Yash Kumar  and
      Rudinger, Rachel  and
      Ren, Xiang  and
      Tandon, Niket  and
      Zouhar, Vil{\'e}m",
    booktitle = "Proceedings of the First Workshop on Commonsense Representation and Reasoning (CSRR 2022)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2022.csrr-1.6/",
    doi = "10.18653/v1/2022.csrr-1.6",
    pages = "43--48",
    abstract = "Previous studies have shown the efficacy of knowledge augmentation methods in pretrained language models. However, these methods behave differently across domains and downstream tasks. In this work, we investigate the augmentation of pretrained language models with knowledge graph data in the cause-effect relation classification and commonsense causal reasoning tasks. After automatically verbalizing triples in ATOMIC2020, a wide coverage commonsense reasoning knowledge graph, we continually pretrain BERT and evaluate the resulting model on cause-effect pair classification and answering commonsense causal reasoning questions. Our results show that a continually pretrained language model augmented with commonsense reasoning knowledge outperforms our baselines on two commonsense causal reasoning benchmarks, COPA and BCOPA-CE, and a Temporal and Causal Reasoning (TCR) dataset, without additional improvement in model architecture or using quality-enhanced data for fine-tuning."
}