Avinash Anand
2026
Better and Worse with Scale: How Contextual Entrainment Diverges with Model Size
Dikshant Kukreja | Kshitij Sah | Gautam Gupta | Avinash Anand | Rajiv Ratn Shah | Zhengkui Wang | Aik Beng Ng | Erik Cambria
Findings of the Association for Computational Linguistics: ACL 2026
Dikshant Kukreja | Kshitij Sah | Gautam Gupta | Avinash Anand | Rajiv Ratn Shah | Zhengkui Wang | Aik Beng Ng | Erik Cambria
Findings of the Association for Computational Linguistics: ACL 2026
Larger language models become simultaneously better and worse at handling contextual information—better at ignoring false claims, worse at ignoring irrelevant tokens. We formalize this apparent paradox through the first scaling laws for contextual entrainment, the tendency of models to favor tokens that appeared in context regardless of relevance. Analyzing the Cerebras-GPT (111M–13B) and Pythia (14M–12B) model families, we find entrainment follows predictable power-law scaling, but with opposite trends depending on context type: semantic contexts show decreasing entrainment with scale, while non-semantic contexts show increasing entrainment. Concretely, the largest models are four times more resistant to counterfactual misinformation than the smallest, yet simultaneously twice as prone to copying arbitrary tokens. These diverging trends, which replicate across model families, suggest that semantic filtering and mechanical copying are functionally distinct behaviors that scale in opposition. These opposing trends suggest that scaling alone does not resolve context sensitivity—it reshapes it.
IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning
Navya Gupta | Rishitej Reddy Vyalla | Avinash Anand | Chhavi Kirtani | Erik Cambria | Zhengchen Zhang | Zhengkui Wang | Timothy Liu | Aik Beng Ng | Simon See | Rajiv Ratn Shah
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Navya Gupta | Rishitej Reddy Vyalla | Avinash Anand | Chhavi Kirtani | Erik Cambria | Zhengchen Zhang | Zhengkui Wang | Timothy Liu | Aik Beng Ng | Simon See | Rajiv Ratn Shah
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Curriculum learning helps language models tackle complex reasoning by gradually increasing task difficulty. However, it often fails to generate consistent step-by-step reasoning, especially in multilingual and low-resource settings where cross-lingual transfer from English to Indian languages remains limited.We propose IRIS: Interleaved Reinforcement with Incremental Staged Curriculum, a two-axis framework that combines Supervised Fine-Tuning on progressively harder problems (vertical axis) with Reverse Curriculum Reinforcement Learning to reduce reliance on step-by-step guidance (horizontal axis). We design a composite reward combining correctness, step-wise alignment, continuity, and numeric incentives, optimized via Group Relative Policy Optimization (GRPO). We release CL-Math, a dataset of 29k problems with step-level annotations in English, Hindi, and Marathi.Across standard benchmarks and curated multilingual test sets, IRIS consistently improves performance, with strong results on math reasoning tasks and substantial gains in low-resource and bilingual settings, alongside modest improvements in high-resource languages. Our code and dataset will be publicly available at https://github.com/avinanand/IRIS-Interleaved-Reinforcement-