@inproceedings{zhang-etal-2023-slowbert,
    title = "{S}low{BERT}: Slow-down Attacks on Input-adaptive Multi-exit {BERT}",
    author = "Zhang, Shengyao  and
      Pan, Xudong  and
      Zhang, Mi  and
      Yang, Min",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2023.findings-acl.634/",
    doi = "10.18653/v1/2023.findings-acl.634",
    pages = "9992--10007",
    abstract = "For pretrained language models such as Google`s BERT, recent research designs several input-adaptive inference mechanisms to improve the efficiency on cloud and edge devices. In this paper, we reveal a new attack surface on input-adaptive multi-exit BERT, where the adversary imperceptibly modifies the input texts to drastically increase the average inference cost. Our proposed slow-down attack called \textit{SlowBERT} integrates a new rank-and-substitute adversarial text generation algorithm to efficiently search for the perturbation which maximally delays the exiting time. With no direct access to the model internals, we further devise a \textit{time-based approximation algorithm} to infer the exit position as the loss oracle. Our extensive evaluation on two popular instances of multi-exit BERT for GLUE classification tasks validates the effectiveness of SlowBERT. In the worst case, SlowBERT increases the inference cost by $4.57\times$, which would strongly hurt the service quality of multi-exit BERT in practice, e.g., increasing the real-time cloud services' response times for online users."
}