@inproceedings{le-etal-2022-efficient,
    title = "Efficient Two-Stage Progressive Quantization of {BERT}",
    author = "Le, Charles  and
      Ardakani, Arash  and
      Ardakani, Amir  and
      Zhang, Hang  and
      Chen, Yuyan  and
      Clark, James  and
      Meyer, Brett  and
      Gross, Warren",
    editor = {Fan, Angela  and
      Gurevych, Iryna  and
      Hou, Yufang  and
      Kozareva, Zornitsa  and
      Luccioni, Sasha  and
      Sadat Moosavi, Nafise  and
      Ravi, Sujith  and
      Kim, Gyuwan  and
      Schwartz, Roy  and
      R{\"u}ckl{\'e}, Andreas},
    booktitle = "Proceedings of The Third Workshop on Simple and Efficient Natural Language Processing (SustaiNLP)",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates (Hybrid)",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.sustainlp-1.2",
    doi = "10.18653/v1/2022.sustainlp-1.2",
    pages = "1--9",
    abstract = "The success of large BERT models has raised the demand for model compression methods to reduce model size and computational cost. Quantization can reduce the model size and inference latency, making inference more efficient, without changing its stucture, but it comes at the cost of performance degradation. Due to the complex loss landscape of ternarized/binarized BERT, we present an efficient two-stage progressive quantization method in which we fine tune the model with quantized weights and progressively lower its bits, and then we fine tune the model with quantized weights and activations. At the same time, we strategically choose which bitwidth to fine-tune on and to initialize from, and which bitwidth to fine-tune under augmented data to outperform the existing BERT binarization methods without adding an extra module, compressing the binary model 18{\%} more than previous binarization methods or compressing BERT by 31x w.r.t. to the full-precision model. Our method without data augmentation can outperform existing BERT ternarization methods.",
}