MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution

Libo Sun, Jiwen Zhang, Siyuan Wang, Zhongyu Wei


Abstract
Mobile GUI agents powered by large foundation models enable autonomous task execution in applications, but frequent updates that alter UI appearance and reorganize workflows cause agents trained on historical data to fail. Despite these surface changes, we observe that functional semantics and task intents remain fundamentally stable. Building on this insight, we introduce MAGNET, a memory-driven adaptive agent framework with dual-level memory: stationary memory that links diverse visual features to stable functional semantics for robust action grounding and procedural memory that captures stable task intents across varying workflows. Furthermore, we propose a dynamic memory evolution mechanism that continuously refines both memories by prioritizing frequently accessed knowledge. Evaluations on the online benchmark AndroidWorld demonstrate substantial improvements over memory-augmented baselines, while offline benchmarks confirm consistent gains under distribution shifts. These results validate that leveraging stable structures across interface changes improves agent performance and generalization in evolving software environments.
Anthology ID:
2026.acl-long.1299
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
28181–28206
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.1299/
DOI:
Bibkey:
Cite (ACL):
Libo Sun, Jiwen Zhang, Siyuan Wang, and Zhongyu Wei. 2026. MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 28181–28206, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolution (Sun et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.1299.pdf
Checklist:
 2026.acl-long.1299.checklist.pdf