Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training
Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, Gholamreza Haffari
Abstract
Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.- Anthology ID:
- 2022.findings-acl.49
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2022
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 582–588
- Language:
- URL:
- https://aclanthology.org/2022.findings-acl.49
- DOI:
- 10.18653/v1/2022.findings-acl.49
- Cite (ACL):
- Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, and Gholamreza Haffari. 2022. Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training. In Findings of the Association for Computational Linguistics: ACL 2022, pages 582–588, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training (Vu et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2022.findings-acl.49.pdf
- Code
- trangvu/lodo-nmt