Heng Dong
2026
ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding
Tian Xueyun | Wei Li | Bingbing Xu | Heng Dong | Yuanzhuo Wang | Huawei Shen
Findings of the Association for Computational Linguistics: ACL 2026
Tian Xueyun | Wei Li | Bingbing Xu | Heng Dong | Yuanzhuo Wang | Huawei Shen
Findings of the Association for Computational Linguistics: ACL 2026
Recent Omni-multimodal Large Language Models show promise in unified audio, vision, and text modeling. However, streaming audio-video understanding remains challenging, as existing approaches suffer from disjointed capabilities: they typically exhibit incomplete modality support or lack autonomous proactive monitoring. To address this, we present ROMA, **a real-time omni-multimodal assistant for unified reactive and proactive interaction**. ROMA processes continuous inputs as synchronized multimodal units, aligning dense audio with discrete video frames to handle granularity mismatches. For online decision-making, we introduce a lightweight *speak head* that decouples response initiation from generation to ensure precise triggering without task conflict. We train ROMA with a curated streaming dataset and a two-stage curriculum that progressively optimizes for streaming format adaptation and proactive responsiveness. To standardize the fragmented evaluation landscape, we reorganize diverse benchmarks into a unified suite covering both proactive (alert, narration) and reactive (QA) settings. Extensive experiments across 12 benchmarks demonstrate ROMA achieves state-of-the-art performance on proactive tasks while competitive in reactive settings, validating its robustness in unified real-time omni-multimodal understanding. Code and benchmark are available [here](https://eureka-maggie.github.io/ROMA_show/).
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization
Tian Xueyun | MingHua Ma | Bingbing Xu | Nuoyan Lyu | Wei Li | Heng Dong | Zheng Chu | Yuanzhuo Wang | Huawei Shen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Tian Xueyun | MingHua Ma | Bingbing Xu | Nuoyan Lyu | Wei Li | Heng Dong | Zheng Chu | Yuanzhuo Wang | Huawei Shen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Supervised fine-tuning (SFT) on chain-of-thought (CoT) trajectories demonstrations is a common approach for enabling reasoning in large language models. Standard practices typically only retain trajectories with correct final answers (*positives*) while ignoring the rest (*negatives*). We argue that this paradigm discards substantial supervision and exacerbates overfitting, limiting out-of-domain (OOD) generalization. Specifically, we surprisingly find that incorporating *negative* trajectories into SFT yields substantial OOD generalization gains over *positive-only* training, as these trajectories often retain valid intermediate reasoning despite incorrect final answers. To understand this effect in depth, we systematically analyze data, training dynamics, and inference behavior, identifying 22 recurring patterns in negative chains that serve a dual role: they moderate loss descent to mitigate overfitting during training and boost policy entropy by 35.67% during inference to facilitate exploration. Motivated by these observations, we further propose **Gain-based LOss Weighting** (GLOW), an adaptive, sample-aware scheme that exploits such distinctive training dynamics by rescaling per-sample loss based on inter-epoch progress. Empirically, GLOW efficiently leverages unfiltered trajectories, yielding a 5.51% OOD gain over positive-only SFT on Qwen2.5-7B and boosting MMLU from 72.82% to 76.47% as an RL initialization. Code is available at [Github](https://github.com/Eureka-Maggie/GLOW).