What Brain Data Adds to Language Model Training

Gabriele Merlin, Omer Moussa, Mariya Toneva


Abstract
Brain-tuning language models (LMs)—fine-tuning LMs to predict brain recordings elicited by linguistic stimuli—has been proposed as a promising way to align LMs closer to the human brain, with recent work reporting gains on a small number of downstream tasks. However, it remains unclear what benefits brain data provide beyond those obtainable from further training on the same underlying linguistic input, and whether such benefits generalize across tasks. Here, we present a comprehensive evaluation of jointly-tuned LMs, trained on both brain recordings and text-based stimuli, brain-tuned LMs and LMs tuned only on text-based stimuli (i.e., stimulus-tuned LMs). We compare models across a diverse suite of downstream linguistic tasks. We find that jointly-tuned LMs outperform other fine-tuned and pretrained models, and that brain-tuned LMs outperform stimulus-tuned LMs, demonstrating the richness of brain data as an additional training signal for LMs.
Anthology ID:
2026.conll-main.12
Volume:
Proceedings of the 30th Conference on Computational Natural Language Learning
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Claire Bonial, Yevgeni Berzak
Venues:
CoNLL | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
178–212
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.conll-main.12/
DOI:
Bibkey:
Cite (ACL):
Gabriele Merlin, Omer Moussa, and Mariya Toneva. 2026. What Brain Data Adds to Language Model Training. In Proceedings of the 30th Conference on Computational Natural Language Learning, pages 178–212, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
What Brain Data Adds to Language Model Training (Merlin et al., CoNLL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.conll-main.12.pdf