@inproceedings{shukla-etal-2026-adaptive,
    title = "Adaptive Data Flywheel: Applying {MAPE} Control Loops to {AI} Agent Improvement",
    author = "Shukla, Aaditya  and
      Knowles, Sidney  and
      Madugula, Meenakshi  and
      Farris, David  and
      Angilly, Ryan  and
      Pombo, Santiago  and
      An, Lu  and
      Xu, Anbang  and
      Balasubramanian, Abhinav  and
      Yu, Tan  and
      Ren, Jiaxiang  and
      Akkiraju, Rama",
    editor = {Matusevych, Yevgen  and
      Eryi{\u{g}}it, G{\"u}l{\c{s}}en  and
      Aletras, Nikolaos},
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 5: Industry Track)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-industry.33/",
    pages = "438--454",
    ISBN = "979-8-89176-384-5",
    abstract = "Enterprise AI agents must continuously adapt to maintain accuracy, reduce latency, and remain aligned with user needs. We present a practical implementation of a data flywheel in NVInfo AI, NVIDIA{'}s Mixture-of-Experts (MoE) Knowledge Assistant serving over 30,000 employees. By operationalizing a MAPE-driven data flywheel, we built a closed-loop system that systematically addresses failures in retrieval-augmented generation (RAG) pipelines and enables continuous learning.Over a 3-month post-deployment period, we monitored feedback and collected 495 negative samples. Analysis revealed two major failure modes: routing errors (5.25{\%}) and query rephrasal errors (3.2{\%}). Using NVIDIA NeMo Microservices, we implemented targeted improvements through fine-tuning. For routing, we replaced a Llama 3.1 70B model with a fine-tuned 8B variant, achieving 96{\%} accuracy, a 10{\texttimes} reduction in model size, and 70{\%} latency improvement. For query rephrasal, fine-tuning yielded a 3.7{\%} gain in accuracy and a 40{\%} latency reduction.Our approach demonstrates how human-in-the-loop (HITL) feedback, when structured within a data flywheel, transforms enterprise AI agents into self-improving systems. Key learnings include approaches to ensure agent robustness despite limited user feedback, navigating privacy constraints, and executing staged rollouts in production. This work offers a repeatable blueprint for building robust, adaptive enterprise AI agents capable of learning from real-world usage at scale."
}