Abstract
This paper reports our approach and the results of our experiments for W-NUT task 2: Identification of Informative COVID-19 English Tweets. In this paper, we test out the effectiveness of transfer learning method with state of the art language models as RoBERTa on this text classification task. Moreover, we examine the benefit of applying additional fine-tuning and training techniques including fine-tuning discrimination, gradual unfreezing as well as our custom head for the classifier. Our best model results in a high F1-score of 89.89 on the task’s private test dataset and that of 90.96 on public test set without ensembling multiple models and additional data.- Anthology ID:
- 2020.wnut-1.71
- Volume:
- Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
- Venue:
- WNUT
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 475–479
- Language:
- URL:
- https://aclanthology.org/2020.wnut-1.71
- DOI:
- 10.18653/v1/2020.wnut-1.71
- Cite (ACL):
- Huy Dao Quang and Tam Nguyen Minh. 2020. UET at WNUT-2020 Task 2: A Study of Combining Transfer Learning Methods for Text Classification with RoBERTa. In Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020), pages 475–479, Online. Association for Computational Linguistics.
- Cite (Informal):
- UET at WNUT-2020 Task 2: A Study of Combining Transfer Learning Methods for Text Classification with RoBERTa (Dao Quang & Nguyen Minh, WNUT 2020)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2020.wnut-1.71.pdf
- Data
- WNUT-2020 Task 2