@inproceedings{gulati-etal-2024-distribution,
    title = "Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression",
    author = "Gulati, Aryan  and
      Dong, Xingjian  and
      Hurtado, Carlos  and
      Shekkizhar, Sarath  and
      Swayamdipta, Swabha  and
      Ortega, Antonio",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.findings-emnlp.758/",
    doi = "10.18653/v1/2024.findings-emnlp.758",
    pages = "12943--12959",
    abstract = "As language models become more general purpose, increased attention needs to be paid to detecting out-of-distribution (OOD) instances, i.e., those not belonging to any of the distributions seen during training. Existing methods for detecting OOD data are computationally complex and storage-intensive. We propose a novel soft clustering approach for OOD detection based on non-negative kernel regression. Our approach greatly reduces computational and space complexities (up to $11\times$ improvement in inference time and 87{\%} reduction in storage requirements). It outperforms existing approaches by up to 4 AUROC points on four benchmarks. We also introduce an entropy-constrained version of our algorithm, leading to further reductions in storage requirements (up to 97{\%} lower than comparable approaches) while retaining competitive performance. Our soft clustering approach for OOD detection highlights its potential for detecting tail-end phenomena in extreme-scale data settings. Our source code is available on Github."
}