@inproceedings{vejendla-2025-teaching,
    title = "Teaching by Failure: Counter-Example{--}Driven Curricula for Transformer Self-Improvement",
    author = "Vejendla, Harshil",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.findings-ijcnlp.24/",
    pages = "423--431",
    ISBN = "979-8-89176-303-6",
    abstract = "Transformer models often exhibit brittle extrapolation, failing on inputs that are longer or structurally more complex than those seen during training. We introduce Counter-Example{--}Driven Curricula (CEDC), an automated framework that improves model robustness by iteratively focusing on its own failures. At each step, CEDC uses the current model to generate a diverse set of candidate problems, employs a fast, executable verifier to identify incorrect predictions (counter{-}examples), and then fine{-}tunes the model on a dataset enriched with these discovered failures. We evaluate CEDC on a suite of algorithmic and natural language tasks, including integer addition, sorting, Dyck{-}2 language recognition, and three text classification benchmarks. Compared to static training and standard curriculum learning baselines, CEDC achieves up to 30{\texttimes} greater length extrapolation, is 3.75{\texttimes} more computationally efficient than uniform data augmentation, and requires no manual difficulty heuristics. We provide a detailed analysis of the counter{-}examples, showing how the curriculum naturally adapts to target progressively more complex error modes. Our findings establish verifier{-}guided, failure{-}driven learning as a simple, powerful, and efficient paradigm for enhancing the generalization capabilities of Transformer models."
}