@inproceedings{hussain-etal-2026-soundbreak,
    title = "{S}ound{B}reak: A Systematic Study of Audio-Only Adversarial Attacks on Trimodal Models",
    author = "Hussain, Aafiya Shamshad  and
      Srivastava, Gaurav  and
      Ishmam, Alvi Md  and
      Hakim, Zaber Ibn Abdul  and
      Thomas, Chris",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1275/",
    pages = "27635--27663",
    ISBN = "979-8-89176-390-6",
    abstract = "Multimodal foundation models that integrate audio, vision, and language achieve strong performance on reasoning and generation tasks, yet their robustness to adversarial manipulation remains poorly understood. We study a realistic and underexplored threat model: **untargeted, audio-only adversarial attacks** on trimodal audio{--}video{--}language models. We analyze six complementary attack objectives that target different stages of multimodal processing, including audio encoder representations, cross-modal attention, hidden states, and output likelihoods. Across four state-of-the-art models and multiple benchmarks, we show that audio-only perturbations can induce severe multimodal failures, achieving up to **96{\%} attack success rate.** We further show that attacks can be successful at low perceptual distortions (LPIPS $\leq 0.08$, SI-SNR $\geq 0${~}dB) and benefit more from extended optimization than increased data scale. We evaluate the feasibility of these attacks under physically realistic conditions by incorporating room impulse response (RIR) modeling, showing that audio-only perturbations remain effective under environmental transformations and thus highlight the practical risk of single-modality attacks in real-world multimodal systems. Transferability across models and encoders remains limited, while speech recognition systems such as Whisper primarily respond to perturbation magnitude, achieving **{\ensuremath{>}}97{\%} attack success** under severe distortion. These results expose a previously overlooked single-modality attack surface in multimodal systems and motivate defenses that enforce cross-modal consistency. Our project website is available at https://aafiya-h.github.io/soundbreak/."
}