Abstract
Predicting the key explanation concept is essential for generating commonsense explanations. This paper introduces a method to predict the concept from pre-trained language models for commonsense explanation generation. Our experiment found that adopting a language model as the concept extractor and fine-tuning it with 20% training data can improve the quality and accuracy of the generated explanations over multiple evaluation metrics. Compared with conventional methods that search concepts over knowledge graphs, our method does not require the preparation and training models to search through knowledge graphs. To better understand the results from pre-trained language models, we also designed a metric to evaluate the retrieved concepts. Through analysis and experiments, we show the correlation between this metric and the performance of the generators, and we also show the importance of attaching concepts for generating high-quality sentences.- Anthology ID:
- 2022.findings-emnlp.433
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5883–5893
- Language:
- URL:
- https://aclanthology.org/2022.findings-emnlp.433
- DOI:
- Cite (ACL):
- Yanbo Fang and Yongfeng Zhang. 2022. Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5883–5893, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation (Fang & Zhang, Findings 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.findings-emnlp.433.pdf