Domain Specific Sub-network for Multi-Domain Neural Machine Translation
Amr Hendy, Mohamed Abdelghaffar, Mohamed Afify, Ahmed Y. Tawfik
Abstract
This paper presents Domain-Specific Sub-network (DoSS). It uses a set of masks obtained through pruning to define a sub-network for each domain and finetunes the sub-network parameters on domain data. This performs very closely and drastically reduces the number of parameters compared to finetuning the whole network on each domain. Also a method to make masks unique per domain is proposed and shown to greatly improve the generalization to unseen domains. In our experiments on German to English machine translation the proposed method outperforms the strong baseline of continue training on multi-domain (medical, tech and religion) data by 1.47 BLEU points. Also continue training DoSS on new domain (legal) outperforms the multi-domain (medical, tech, religion, legal) baseline by 1.52 BLEU points.- Anthology ID:
- 2022.aacl-short.43
- Volume:
- Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
- Month:
- November
- Year:
- 2022
- Address:
- Online only
- Editors:
- Yulan He, Heng Ji, Sujian Li, Yang Liu, Chua-Hui Chang
- Venues:
- AACL | IJCNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 351–356
- Language:
- URL:
- https://aclanthology.org/2022.aacl-short.43
- DOI:
- Cite (ACL):
- Amr Hendy, Mohamed Abdelghaffar, Mohamed Afify, and Ahmed Y. Tawfik. 2022. Domain Specific Sub-network for Multi-Domain Neural Machine Translation. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 351–356, Online only. Association for Computational Linguistics.
- Cite (Informal):
- Domain Specific Sub-network for Multi-Domain Neural Machine Translation (Hendy et al., AACL-IJCNLP 2022)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2022.aacl-short.43.pdf