@inproceedings{zhang-etal-2025-dynamic,
    title = "Dynamic Feature Fusion for Sign Language Translation Using {H}yper{N}etworks",
    author = "Zhang, Ruiquan  and
      Zhao, Rui  and
      Wu, Zhicong  and
      Zhang, Liang  and
      Zhang, Haoqi  and
      Chen, Yidong",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.348/",
    pages = "6227--6239",
    ISBN = "979-8-89176-195-7",
    abstract = "This paper presents an efficient dual-stream early fusion method for sign language translation. Inspired by the brain{'}s ability to process color, shape, and motion simultaneously, the method explores complex dependencies between RGB and keypoint streams, improving speed and efficiency. A key challenge is extracting complementary features from both streams while ensuring global semantic consistency to avoid conflicts and improve generalization. To address this issue, we propose a hypernetwork-based fusion strategy that effectively extracts salient features from RGB and keypoint streams, alongside a partial shortcut connection training method to strengthen the complementary information between the dual streams. Additionally, we introduce self-distillation and SST contrastive learning to maintain feature advantages while aligning the global semantic space. Experiments show that our method achieves state-of-the-art performance on two public sign language datasets, reducing model parameters by about two-thirds."
}