Jingtao Xu
2026
GUI0: Self-Evolving Foundational GUI Agents in Super App Ecosystems
Xinyi Wang | Wei Dai | Kyle Qiao | Ke Wang | Peng Chen | Gang Cao | Kangqin | Zhongpu Wang | Xiaode Zhang | Yanming Liu | Jihao Gu | Jingtao Xu | Gong Zhi
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Xinyi Wang | Wei Dai | Kyle Qiao | Ke Wang | Peng Chen | Gang Cao | Kangqin | Zhongpu Wang | Xiaode Zhang | Yanming Liu | Jihao Gu | Jingtao Xu | Gong Zhi
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Automated interaction with graphical user interfaces (GUIs) is central to General Artificial Intelligence yet remains challenging within Super App ecosystems, characterized by non-standard rendering and absent accessibility metadata. While GUI agents often rely on explicit accessibility trees or static imitation, they are less explored for dynamic environments marked by sparse feedback and implicit visual cues. We present GUI0, a framework synergizing autonomous data synthesis with dual-agent co-evolution. GUI0 establishes a domain-aware foundation model via synthesized corpora and employs curriculum-driven reinforcement learning, where a curriculum agent generates boundary tasks to optimize an actor agent.Empirical results demonstrate three key advantages: (1) State-of-the-art performance on the SuperAPP benchmark, outperforming Gemini-2.5-Pro and Claude-4-Sonnet; (2) universal efficacy across diverse base models, consistently yielding substantial improvements on both Qwen2.5-VL and GUI-Owl variants; and (3) robust zero-shot generalization to standard GUIs (e.g., +62.7% on ScreenSpot Pro).
2025
Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided Role-playing Agents
Xueqiao Zhang | Chao Zhang | Jingtao Xu | Yifan Zhu | Xin Shi | Yi Yang | Yawei Luo
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Xueqiao Zhang | Chao Zhang | Jingtao Xu | Yifan Zhu | Xin Shi | Yi Yang | Yawei Luo
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Role-playing agents (RPAs) have attracted growing interest for their ability to simulate immersive and interactive characters. However, existing approaches primarily focus on static role profiles, overlooking the dynamic perceptual abilities inherent to humans. To bridge this gap, we introduce the concept of dynamic role profiles by incorporating video modality into RPAs. To support this, we construct Role-playing-Video60k, a large-scale, high-quality dataset comprising 60k videos and 700k corresponding dialogues. Based on this dataset, we develop a comprehensive RPA framework that combines adaptive temporal sampling with both dynamic and static role profile representations. Specifically, the dynamic profile is created by adaptively sampling video frames and feeding them to the LLM in temporal order, while the static profile consists of (1) character dialogues from training videos during fine-tuning, and (2) a summary context from the input video during inference. This joint integration enables RPAs to generate greater responses. Furthermore, we propose a robust evaluation method covering eight metrics. Experimental results demonstrate the effectiveness of our framework, highlighting the importance of dynamic role profiles in developing RPAs.