Lijiao Xu
2026
Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents
Yifei Li | Weidong Guo | Lingling Zhang | Rongman Xu | Muye Huang | Hui Liu | Lijiao Xu | Yu Xu | Jun Liu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yifei Li | Weidong Guo | Lingling Zhang | Rongman Xu | Muye Huang | Hui Liu | Lijiao Xu | Yu Xu | Jun Liu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Long-term conversational memory is a core capability for LLM-baseddialogue systems, yet existing benchmarks and evaluation protocolsprimarily focus on surface-level factual recall.In realistic interactions, appropriate responses often depend onimplicit constraints such as user state, goals, or values that are notexplicitly queried later.To evaluate this setting, we introduce LoCoMo-Plus, a benchmarkfor assessing cognitive memory under cue–trigger semantic disconnect,where models must retain and apply latent constraints across longconversational contexts.We further show that conventional string-matching metrics and explicittask-type prompting are misaligned with such scenarios, and propose aunified evaluation framework based on constraint consistency.Experiments across diverse backbone models, retrieval-based methods, andmemory systems demonstrate that cognitive memory remains challenging andreveals failures not captured by existing benchmarks.Our code and evaluation framework are publicly available at https://github.com/xjtuleeyf/Locomo-Plus.