@inproceedings{nohejl-etal-2026-sakura,
    title = "Sakura at {BEA} 2026 Shared Task 1: What Makes Vocabulary Difficult?",
    author = "Nohejl, Adam  and
      Wu, Xuanxin  and
      Ide, Yusuke  and
      Riera Machin, Maria  and
      Chang, Yi-Ning",
    editor = "Kochmar, Ekaterina  and
      Alhafni, Bashar  and
      Bann{\`o}, Stefano  and
      Bexte, Marie  and
      Burstein, Jill  and
      Horbach, Andrea  and
      Laarmann-Quante, Ronja  and
      Tack, Anais  and
      Yaneva, Victoria  and
      Yuan, Zheng",
    booktitle = "Proceedings of the 21st Workshop on Innovative Use of {NLP} for Building Educational Applications ({BEA} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.bea-1.84/",
    pages = "1161--1178",
    ISBN = "979-8-89176-409-5",
    abstract = "We describe two types of models for vocabulary difficulty prediction: a high-accuracy black-box model, which achieved the top shared task result in the open track, and an explainable model, which outperforms a fine-tuned encoder baseline. As the black-box model, we fine-tuned an LLM using a soft-target loss function for effective application to the rating task, achieving r {\ensuremath{>}} 0.91. The explainable model provides insights into what impacts the difficulty of each item while maintaining a strong correlation (r {\ensuremath{>}} 0.77). We further analyze the results, demonstrating that the difficulty of items in the British Council{'}s Knowledge-based Vocabulary Lists (KVL) is often affected by spelling difficulty or the construction of the test items, in addition to the genuine production difficulty of the words. We make our code available online."
}