@inproceedings{kucharavy-etal-2025-llms,
    title = "{LLM}s Prot{\'e}g{\'e}s: Tutoring {LLM}s with Knowledge Gaps Improves Student Learning Outcome",
    author = "Kucharavy, Andrei  and
      Vallez, Cyril  and
      Percia David, Dimitri",
    editor = {Kochmar, Ekaterina  and
      Alhafni, Bashar  and
      Bexte, Marie  and
      Burstein, Jill  and
      Horbach, Andrea  and
      Laarmann-Quante, Ronja  and
      Tack, Ana{\"i}s  and
      Yaneva, Victoria  and
      Yuan, Zheng},
    booktitle = "Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.bea-1.19/",
    pages = "248--257",
    ISBN = "979-8-89176-270-1",
    abstract = "Since the release of ChatGPT, Large Langauge Models (LLMs) have been proposed as potential tutors to students in the education outcomes. Such an LLM-as-tutors metaphor is problematic, notably due to the counterfactual generation, perception of learned skills as mastered by an automated system and hence non-valuable, and learning LLM over-reliance.We propose instead the LLM-as-mentee tutoring schema, leveraging the Learning-by-Teaching prot{\'e}g{\'e} effect in peer tutoring - LLM Prot{\'e}g{\'e}s. In this configuration, counterfactual generation is desirable, allowing students to operationalize the learning material and better understand the limitations of LLM-based systems, both a skill in itself and an additional learning motivation. Our preliminary results suggest that LLM Prot{\'e}g{\'e}s are effective. Students in an introductory algorithms class who successfully diagnosed an LLM teachable agent system prompted to err on a course material gained an average of 0.72 points on a 1-6 scale. Remarkably, if fully adopted, this approach would reduce the failure rate in the second midterm from 28{\%} to 8{\%}, mitigating 72{\%} of midterm failure.We publish code for on-premises deployment of LLM Prot{\'e}g{\'e}s on https://github.com/Reliable-Information-Lab-HEVS/LLM{\_}Proteges."
}