@article{tu-etal-2026-reason,
    title = "Reason-to-Learn ({R}2{L}): Multi-Agent Knowledge Distillation for Lightweight {LLM}s in Sentiment Analysis",
    author = "Tu, Le-Huy  and
      Nguyen, Quan  and
      NGUYEN, Vincent  and
      Bjorklund, Johanna  and
      Vu, Xuan-Son",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    journal = "International Conference on Language Resources and Evaluation",
    volume = "main",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.809/",
    pages = "10301--10312",
    abstract = "Large Language Models (LLMs) boast remarkable capabilities but face deployment challenges due to computational demands. We introduce Reason-to-Learn (R2L), a novel multi-agent collaborative knowledge distillation framework enabling small LLMs to learn from a distributed system of specialized agent models. Our architecture employs multiple autonomous teacher agents, each with distinct expertise and reasoning capabilities, coordinated by a meta-agent that orchestrates knowledge synthesis and conflict resolution. Unlike prior methods, our flexible four-phase process (Detection, Processing, Rationale Generation, Aggregation) leverages agent-based communication protocols and consensus mechanisms for cross-architecture knowledge transfer, demonstrated primarily on Vietnamese sentiment analysis. Experimental results are definitive: our lightweight R2L-Students (1-1.5B) consistently outperform the individual specialized agents (Qwen32B, Llama70B) and the GPT-4o meta-agent coordinator, especially on complex ABSA tasks. Ablation studies confirm our multi-agent collaborative approach outperformed traditional fine-tuning and single-agent distillation. Furthermore, R2L enhance generalizability of lightweight LLMs: our Vietnamese-trained student achieves strong zero-shot cross-lingual performance on Swedish ABSA (Svensk ABSAbank-Imm), with Krippendorff{'}s Alpha scores competitive with the specialized agents. R2L offers an efficient path to compact, high-performing specialist models through coordinated multi-agent learning."
}