Abstract
In this paper, we describe our team’s (JUSTers) effort in the Commonsense Validation and Explanation (ComVE) task, which is part of SemEval2020. We evaluate five pre-trained Transformer-based language models with various sizes against the three proposed subtasks. For the first two subtasks, the best accuracy levels achieved by our models are 92.90% and 92.30%, respectively, placing our team in the 12th and 9th places, respectively. As for the last subtask, our models reach 16.10 BLEU score and 1.94 human evaluation score placing our team in the 5th and 3rd places according to these two metrics, respectively. The latter is only 0.16 away from the 1st place human evaluation score.- Anthology ID:
- 2020.semeval-1.66
- Volume:
- Proceedings of the Fourteenth Workshop on Semantic Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona (online)
- Editors:
- Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- International Committee for Computational Linguistics
- Note:
- Pages:
- 535–542
- Language:
- URL:
- https://aclanthology.org/2020.semeval-1.66
- DOI:
- 10.18653/v1/2020.semeval-1.66
- Cite (ACL):
- Ali Fadel, Mahmoud Al-Ayyoub, and Erik Cambria. 2020. JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models against Commonsense Validation and Explanation. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 535–542, Barcelona (online). International Committee for Computational Linguistics.
- Cite (Informal):
- JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models against Commonsense Validation and Explanation (Fadel et al., SemEval 2020)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2020.semeval-1.66.pdf