DMix: Adaptive Distance-aware Interpolative Mixup

Ramit Sawhney, Megh Thakkar, Shrey Pandit, Ritesh Soun, Di Jin, Diyi Yang, Lucie Flek


Abstract
Interpolation-based regularisation methods such as Mixup, which generate virtual training samples, have proven to be effective for various tasks and modalities. We extend Mixup and propose DMix, an adaptive distance-aware interpolative Mixup that selects samples based on their diversity in the embedding space. DMix leverages the hyperbolic space as a similarity measure among input samples for a richer encoded representation.DMix achieves state-of-the-art results on sentence classification over existing data augmentation methods on 8 benchmark datasets across English, Arabic, Turkish, and Hindi languages while achieving benchmark F1 scores in 3 times less number of iterations. We probe the effectiveness of DMix in conjunction with various similarity measures and qualitatively analyze the different components.DMix being generalizable, can be applied to various tasks, models and modalities.
Anthology ID:
2022.acl-short.67
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
606–612
Language:
URL:
https://aclanthology.org/2022.acl-short.67
DOI:
10.18653/v1/2022.acl-short.67
Bibkey:
Cite (ACL):
Ramit Sawhney, Megh Thakkar, Shrey Pandit, Ritesh Soun, Di Jin, Diyi Yang, and Lucie Flek. 2022. DMix: Adaptive Distance-aware Interpolative Mixup. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 606–612, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
DMix: Adaptive Distance-aware Interpolative Mixup (Sawhney et al., ACL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/2022.acl-short.67.pdf
Software:
 2022.acl-short.67.software.zip
Video:
 https://preview.aclanthology.org/naacl24-info/2022.acl-short.67.mp4
Code
 caisa-lab/DMix-ACL
Data
CoLAGLUESSTSST-2