Abstract
The ability of common sense validation and explanation is very important for most models. Most obviously, this will directly affect the rationality of the generated model output. The large amount and diversity of common sense poses great challenges to this task. In addition, many common sense expressions are obscure, thus we need to understand the meaning contained in the vocabulary in order to judge correctly, which further increases the model’s requirements for the accuracy of word representation. The current neural network models are often data-driven, while the annotated data is often limited and requires a lot of manual labeling. In such case, we proposed transfer learning to handle this challenge. From our experiments, we can draw the following three main conclusions: a) Neural language model fully qualified for commonsense validation and explanation. We attribute this to the powerful word and sentence representation capabilities of language models. b) The inconsistency of task of pre-training and fine-tuning will badly hurt the performance. c) A larger amount of corpus and more parameters will enhance the common sense of the model. At the same time, the content of the corpus is equally important.- Anthology ID:
- 2020.semeval-1.50
- Volume:
- Proceedings of the Fourteenth Workshop on Semantic Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona (online)
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- International Committee for Computational Linguistics
- Note:
- Pages:
- 415–421
- Language:
- URL:
- https://aclanthology.org/2020.semeval-1.50
- DOI:
- 10.18653/v1/2020.semeval-1.50
- Cite (ACL):
- Liu Pai. 2020. QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation System Based on Ensemble of Language Model. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 415–421, Barcelona (online). International Committee for Computational Linguistics.
- Cite (Informal):
- QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation System Based on Ensemble of Language Model (Pai, SemEval 2020)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2020.semeval-1.50.pdf