Did You Forget What I Asked? Prospective Memory Failures in Large Language Models

Avni Mittal


Abstract
Large language models often fail to satisfy formatting instructions when they must simultaneously perform demanding tasks. We study this behavior through a prospective memory-inspired lens from cognitive psychology, using a controlled paradigm that combines verifiable formatting constraints with benchmark tasks of increasing complexity. Across three model families and over 8,000 prompts, compliance drops by 2–21% under concurrent task load. Vulnerability is highly type-dependent: terminal constraints (requiring action at the response boundary) degrade most, with drops up to 50%, while avoidance constraints remain comparatively robust. A salience-enhanced format (explicit instruction framing plus a trailing reminder) recovers much of the lost compliance, restoring performance to 90–100% in many settings. Interference is bidirectional: formatting constraints can also reduce task accuracy, with one model’s GSM8K accuracy dropping from 93% to 27%. In additional stacking experiments, joint compliance declines sharply as constraints accumulate. All results use deterministic programmatic checkers, with no LLM-as-judge component, on publicly available datasets.
Anthology ID:
2026.trustnlp-main.33
Volume:
Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)
Month:
July
Year:
2026
Address:
San Diego, California
Editors:
Kai-Wei Chang, Ninareh Mehrabi, Satyapriya Krishna, Anubrata Das, Jwala Dhamala, Yang Trista Cao, Tharindu Kumarage, Anil Ramakrishna, Christos Christodoulopoulos, Yixin Wan, Aram Galystan, Anoop Kumar, Rahul Gupta
Venues:
TrustNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
471–488
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.trustnlp-main.33/
DOI:
Bibkey:
Cite (ACL):
Avni Mittal. 2026. Did You Forget What I Asked? Prospective Memory Failures in Large Language Models. In Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026), pages 471–488, San Diego, California. Association for Computational Linguistics.
Cite (Informal):
Did You Forget What I Asked? Prospective Memory Failures in Large Language Models (Mittal, TrustNLP 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.trustnlp-main.33.pdf