@inproceedings{strohmaier-etal-2025-dlu,
    title = "{DLU}: Dictionary Look-Up Data and Prediction",
    author = "Strohmaier, David  and
      Tyen, Gladys  and
      Gu, Hongyi  and
      Nicholls, Diane  and
      Yuan, Zheng  and
      Buttery, Paula",
    editor = "Boleda, Gemma  and
      Roth, Michael",
    booktitle = "Proceedings of the 29th Conference on Computational Natural Language Learning",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/acl25-workshop-ingestion/2025.conll-1.32/",
    pages = "481--501",
    ISBN = "979-8-89176-271-8",
    abstract = "Knowing which words language learners struggle with is crucial for developing personalised education technologies. In this paper, we advocate for the novel task of ``dictionary look-up prediction'' as a means for evaluating the complexity of words in reading tasks. We release the Dictionary Look-Up development dataset (DLU-dev) and the Dialogue Dictionary Look-Up dataset (D-DLU), which is based on chatbot dialogues. We demonstrate that dictionary look-up is a challenging task for LLMs (results are presented for LLaMA, Gemma, and Longformer models). We explore finetuning with the ROC* loss function as a more appropriate loss for this task than the commonly used Binary Cross Entropy (BCE). We show that a feature-based model outperforms the LLMs. Finally, we investigate the transfer between DLU and the related tasks of Complex Word Identification (CWI) and Semantic Error Prediction (SEP), establishing new state-of-the-art results for SEP."
}