Abstract
By sharing parameters and providing task-independent shared features, multi-task deep neural networks are considered one of the most interesting ways for parallel learning from different tasks and domains. However, fine-tuning on one task may compromise the performance of other tasks or restrict the generalization of the shared learned features. To address this issue, we propose to use task uncertainty to gauge the effect of the shared feature changes on other tasks and prevent the model from overfitting or over-generalizing. We conducted an experiment on 16 text classification tasks, and findings showed that the proposed method consistently improves the performance of the baseline, facilitates the knowledge transfer of learned features to unseen data, and provides explicit control over the generalization of the shared model.- Anthology ID:
- 2022.wassa-1.8
- Volume:
- Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Jeremy Barnes, Orphée De Clercq, Valentin Barriere, Shabnam Tafreshi, Sawsan Alqahtani, João Sedoc, Roman Klinger, Alexandra Balahur
- Venue:
- WASSA
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 78–88
- Language:
- URL:
- https://aclanthology.org/2022.wassa-1.8
- DOI:
- 10.18653/v1/2022.wassa-1.8
- Cite (ACL):
- Kourosh Meshgi, Maryam Sadat Mirzaei, and Satoshi Sekine. 2022. Uncertainty Regularized Multi-Task Learning. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 78–88, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Uncertainty Regularized Multi-Task Learning (Meshgi et al., WASSA 2022)
- PDF:
- https://preview.aclanthology.org/fix-dup-bibkey/2022.wassa-1.8.pdf
- Data
- IMDb Movie Reviews