Q-Learning Scheduler for Multi Task Learning Through the use of Histogram of Task Uncertainty

Kourosh Meshgi, Maryam Sadat Mirzaei, Satoshi Sekine


Abstract
Simultaneous training of a multi-task learning network on different domains or tasks is not always straightforward. It could lead to inferior performance or generalization compared to the corresponding single-task networks. An effective training scheduling method is deemed necessary to maximize the benefits of multi-task learning. Traditional schedulers follow a heuristic or prefixed strategy, ignoring the relation of the tasks, their sample complexities, and the state of the emergent shared features. We proposed a deep Q-Learning Scheduler (QLS) that monitors the state of the tasks and the shared features using a novel histogram of task uncertainty, and through trial-and-error, learns an optimal policy for task scheduling. Extensive experiments on multi-domain and multi-task settings with various task difficulty profiles have been conducted, the proposed method is benchmarked against other schedulers, its superior performance has been demonstrated, and results are discussed.
Anthology ID:
2022.repl4nlp-1.2
Volume:
Proceedings of the 7th Workshop on Representation Learning for NLP
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Spandana Gella, He He, Bodhisattwa Prasad Majumder, Burcu Can, Eleonora Giunchiglia, Samuel Cahyawijaya, Sewon Min, Maximilian Mozes, Xiang Lorraine Li, Isabelle Augenstein, Anna Rogers, Kyunghyun Cho, Edward Grefenstette, Laura Rimell, Chris Dyer
Venue:
RepL4NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9–19
Language:
URL:
https://aclanthology.org/2022.repl4nlp-1.2
DOI:
10.18653/v1/2022.repl4nlp-1.2
Bibkey:
Cite (ACL):
Kourosh Meshgi, Maryam Sadat Mirzaei, and Satoshi Sekine. 2022. Q-Learning Scheduler for Multi Task Learning Through the use of Histogram of Task Uncertainty. In Proceedings of the 7th Workshop on Representation Learning for NLP, pages 9–19, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Q-Learning Scheduler for Multi Task Learning Through the use of Histogram of Task Uncertainty (Meshgi et al., RepL4NLP 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-1/2022.repl4nlp-1.2.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-1/2022.repl4nlp-1.2.mp4
Data
IMDb Movie ReviewsPenn Treebank