Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models
Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, Toshinori Miyoshi
Abstract
This paper shows our system for SemEval-2020 task 10, Emphasis Selection for Written Text in Visual Media. Our strategy is two-fold. First, we propose fine-tuning many pre-trained language models, predicting an emphasis probability distribution over tokens. Then, we propose stacking a trainable distribution fusion DistFuse system to fuse the predictions of the fine-tuned models. Experimental results show tha DistFuse is comparable or better when compared with a naive average ensemble. As a result, we were ranked 2nd amongst 31 teams.- Anthology ID:
- 2020.semeval-1.216
- Volume:
- Proceedings of the Fourteenth Workshop on Semantic Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona (online)
- Venue:
- SemEval
- SIGs:
- SIGLEX | SIGSEM
- Publisher:
- International Committee for Computational Linguistics
- Note:
- Pages:
- 1658–1664
- Language:
- URL:
- https://aclanthology.org/2020.semeval-1.216
- DOI:
- 10.18653/v1/2020.semeval-1.216
- Cite (ACL):
- Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, and Toshinori Miyoshi. 2020. Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1658–1664, Barcelona (online). International Committee for Computational Linguistics.
- Cite (Informal):
- Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models (Morio et al., SemEval 2020)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2020.semeval-1.216.pdf