@inproceedings{xin-etal-2021-berxit,
    title = "{BER}xi{T}: Early Exiting for {BERT} with Better Fine-Tuning and Extension to Regression",
    author = "Xin, Ji  and
      Tang, Raphael  and
      Yu, Yaoliang  and
      Lin, Jimmy",
    editor = "Merlo, Paola  and
      Tiedemann, Jorg  and
      Tsarfaty, Reut",
    booktitle = "Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume",
    month = apr,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2021.eacl-main.8/",
    doi = "10.18653/v1/2021.eacl-main.8",
    pages = "91--104",
    abstract = "The slow speed of BERT has motivated much research on accelerating its inference, and the early exiting idea has been proposed to make trade-offs between model quality and efficiency. This paper aims to address two weaknesses of previous work: (1) existing fine-tuning strategies for early exiting models fail to take full advantage of BERT; (2) methods to make exiting decisions are limited to classification tasks. We propose a more advanced fine-tuning strategy and a learning-to-exit module that extends early exiting to tasks other than classification. Experiments demonstrate improved early exiting for BERT, with better trade-offs obtained by the proposed fine-tuning strategy, successful application to regression tasks, and the possibility to combine it with other acceleration methods. Source code can be found at \url{https://github.com/castorini/berxit}."
}