Abstract
This paper describes the masked reasoner system that participated in SemEval-2020 Task 4: Commonsense Validation and Explanation. The system participated in the subtask B.We proposes a novel method to fine-tune RoBERTa by masking the most important word in the statement. We believe that the confidence of the system in recovering that word is positively correlated to the score the masked language model gives to the current statement-explanation pair. We evaluate the importance of each word using InferSent and do the masked fine-tuning on RoBERTa. Then we use the fine-tuned model to predict the most plausible explanation. Our system is fast in training and achieved 73.5% accuracy.- Anthology ID:
- 2020.semeval-1.49
- Volume:
- Proceedings of the Fourteenth Workshop on Semantic Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona (online)
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- International Committee for Computational Linguistics
- Note:
- Pages:
- 411–414
- Language:
- URL:
- https://aclanthology.org/2020.semeval-1.49
- DOI:
- 10.18653/v1/2020.semeval-1.49
- Cite (ACL):
- Daming Lu. 2020. Masked Reasoner at SemEval-2020 Task 4: Fine-Tuning RoBERTa for Commonsense Reasoning. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 411–414, Barcelona (online). International Committee for Computational Linguistics.
- Cite (Informal):
- Masked Reasoner at SemEval-2020 Task 4: Fine-Tuning RoBERTa for Commonsense Reasoning (Lu, SemEval 2020)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/2020.semeval-1.49.pdf