@inproceedings{kapadia-etal-2026-leap,
    title = "{LEAP}: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference",
    author = "Kapadia, Shashank  and
      Mishra, Deep Narayan  and
      Alugubelli, Sujal Reddy  and
      Wang, Haoan  and
      Vabbilisetty, Saipraveen  and
      Bhatia, Rishi  and
      Sharma, Anupriya",
    editor = "Li, Yunyao  and
      Rehm, Georg  and
      Tu, Mei",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics ({ACL} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-industry.52/",
    pages = "761--774",
    ISBN = "979-8-89176-394-4",
    abstract = "Layer-aligned distillation and convergence-based early exit represent two predominant computational efficiency paradigms for transformer inference; yet we establish that they exhibit fundamental incompatibility under standard deployment conditions for convergence-based early exit. Distillation objectives that align intermediate student layers to teacher representations suppress the representational convergence that early-exit mechanisms exploit, rendering such mechanisms ineffective on distilled models.We introduce LEAP (Layer-wise Exit-Aware Pretraining), an auxiliary training objective that reconciles this incompatibility. LEAP requires no architectural modifications; it augments standard distillation with a single constraint ensuring intermediate layers approximate final-layer representations. LEAP-MiniLM achieves $1.61\times$ measured wall-clock speedup (batch $= 1$, NVIDIA L4) at $\theta = 0.95$, with 91.9{\%} of samples exiting by layer 7 and $1.80\times$ theoretical layer reduction, where standard distilled models achieve zero effective speedup. We validate across sentence similarity (STS-B: $0.760 \pm 0.006$) and retrieval benchmarks (BEIR), providing operational guidance including latency measurements, decision thresholds, and deployment criteria."
}