Abstract
With their Discovery of Inference Rules from Text (DIRT) algorithm, Lin and Pantel (2001) made a seminal contribution to the field of rule acquisition from text, by adapting the distributional hypothesis of Harris (1954) to rules that model binary relations such as X treat Y. DIRT’s relevance is renewed in today’s neural era given the recent focus on interpretability in the field of natural language processing. We propose a novel take on the DIRT algorithm, where we implement the distributional hypothesis using the contextualized embeddings provided by BERT, a transformer-network-based language model (Vaswani et al. 2017; Devlin et al. 2018). In particular, we change the similarity measure between pairs of slots (i.e., the set of words matched by a rule) from the original formula that relies on lexical items to a formula computed using contextualized embeddings. We empirically demonstrate that this new similarity method yields a better implementation of the distributional hypothesis, and this, in turn, yields rules that outperform the original algorithm in the question answering-based evaluation proposed by Lin and Pantel (2001).- Anthology ID:
- 2022.lrec-1.395
- Volume:
- Proceedings of the Thirteenth Language Resources and Evaluation Conference
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Editors:
- Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 3706–3714
- Language:
- URL:
- https://aclanthology.org/2022.lrec-1.395
- DOI:
- Cite (ACL):
- Mahdi Rahimi and Mihai Surdeanu. 2022. Do Transformer Networks Improve the Discovery of Rules from Text?. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3706–3714, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Do Transformer Networks Improve the Discovery of Rules from Text? (Rahimi & Surdeanu, LREC 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2022.lrec-1.395.pdf