Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models

Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, Toshinori Miyoshi


Abstract
This paper shows our system for SemEval-2020 task 10, Emphasis Selection for Written Text in Visual Media. Our strategy is two-fold. First, we propose fine-tuning many pre-trained language models, predicting an emphasis probability distribution over tokens. Then, we propose stacking a trainable distribution fusion DistFuse system to fuse the predictions of the fine-tuned models. Experimental results show tha DistFuse is comparable or better when compared with a naive average ensemble. As a result, we were ranked 2nd amongst 31 teams.
Anthology ID:
2020.semeval-1.216
Volume:
Proceedings of the Fourteenth Workshop on Semantic Evaluation
Month:
December
Year:
2020
Address:
Barcelona (online)
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
International Committee for Computational Linguistics
Note:
Pages:
1658–1664
Language:
URL:
https://aclanthology.org/2020.semeval-1.216
DOI:
10.18653/v1/2020.semeval-1.216
Bibkey:
Cite (ACL):
Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, and Toshinori Miyoshi. 2020. Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1658–1664, Barcelona (online). International Committee for Computational Linguistics.
Cite (Informal):
Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models (Morio et al., SemEval 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2020.semeval-1.216.pdf