Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation

Yanbo Fang, Yongfeng Zhang


Abstract
Predicting the key explanation concept is essential for generating commonsense explanations. This paper introduces a method to predict the concept from pre-trained language models for commonsense explanation generation. Our experiment found that adopting a language model as the concept extractor and fine-tuning it with 20% training data can improve the quality and accuracy of the generated explanations over multiple evaluation metrics. Compared with conventional methods that search concepts over knowledge graphs, our method does not require the preparation and training models to search through knowledge graphs. To better understand the results from pre-trained language models, we also designed a metric to evaluate the retrieved concepts. Through analysis and experiments, we show the correlation between this metric and the performance of the generators, and we also show the importance of attaching concepts for generating high-quality sentences.
Anthology ID:
2022.findings-emnlp.433
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5883–5893
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.433
DOI:
Bibkey:
Cite (ACL):
Yanbo Fang and Yongfeng Zhang. 2022. Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5883–5893, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation (Fang & Zhang, Findings 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.findings-emnlp.433.pdf