@inproceedings{hu-etal-2025-proactive,
    title = "Proactive Hearing Assistants that Isolate Egocentric Conversations",
    author = "Hu, Guilin  and
      Itani, Malek  and
      Chen, Tuochao  and
      Gollakota, Shyamnath",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.emnlp-main.1289/",
    doi = "10.18653/v1/2025.emnlp-main.1289",
    pages = "25377--25394",
    ISBN = "979-8-89176-332-6",
    abstract = "We introduce proactive hearing assistants that automatically identify and separate the wearer{'}s conversation partners, without requiring explicit prompts. Our system operates on egocentric binaural audio and uses the wearer{'}s self-speech as an anchor, leveraging turn-taking behavior and dialogue dynamics to infer conversational partners and suppress others. To enable real-time, on-device operation, we propose a dual-model architecture: a lightweight streaming model runs every 12.5 ms for low-latency extraction of the conversation partners, while a slower model runs less frequently to capture longer-range conversational dynamics. Results on real-world 2- and 3-speaker conversation test sets, collected with binaural egocentric hardware from 11 participants totaling 6.8 hours, show generalization in identifying and isolating conversational partners in multi-conversation settings. Our work marks a step toward hearing assistants that adapt proactively to conversational dynamics and engagement."
}