DMix: Adaptive Distance-aware Interpolative Mixup
Ramit Sawhney, Megh Thakkar, Shrey Pandit, Ritesh Soun, Di Jin, Diyi Yang, Lucie Flek
Abstract
Interpolation-based regularisation methods such as Mixup, which generate virtual training samples, have proven to be effective for various tasks and modalities. We extend Mixup and propose DMix, an adaptive distance-aware interpolative Mixup that selects samples based on their diversity in the embedding space. DMix leverages the hyperbolic space as a similarity measure among input samples for a richer encoded representation.DMix achieves state-of-the-art results on sentence classification over existing data augmentation methods on 8 benchmark datasets across English, Arabic, Turkish, and Hindi languages while achieving benchmark F1 scores in 3 times less number of iterations. We probe the effectiveness of DMix in conjunction with various similarity measures and qualitatively analyze the different components.DMix being generalizable, can be applied to various tasks, models and modalities.- Anthology ID:
- 2022.acl-short.67
- Volume:
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 606–612
- Language:
- URL:
- https://aclanthology.org/2022.acl-short.67
- DOI:
- 10.18653/v1/2022.acl-short.67
- Cite (ACL):
- Ramit Sawhney, Megh Thakkar, Shrey Pandit, Ritesh Soun, Di Jin, Diyi Yang, and Lucie Flek. 2022. DMix: Adaptive Distance-aware Interpolative Mixup. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 606–612, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- DMix: Adaptive Distance-aware Interpolative Mixup (Sawhney et al., ACL 2022)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2022.acl-short.67.pdf
- Code
- caisa-lab/DMix-ACL
- Data
- CoLA, GLUE, SST, SST-2