@inproceedings{amigo-etal-2025-evaluating,
    title = "Evaluating Sequence Labeling on the basis of Information Theory",
    author = "Amigo, Enrique  and
      {\'A}lvarez-Mellado, Elena  and
      Gonzalo, Julio  and
      Carrillo-de-Albornoz, Jorge",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1351/",
    pages = "27849--27860",
    ISBN = "979-8-89176-251-0",
    abstract = "Various metrics exist for evaluating sequence labeling problems (strict span matching, token oriented metrics, token concurrence in sequences, etc.), each of them focusing on certain aspects of the task. In this paper, we define a comprehensive set of formal properties that captures the strengths and weaknesses of the existing metric families and prove that none of them is able to satisfy all properties simultaneously. We argue that it is necessary to measure how much information (correct or noisy) each token in the sequence contributes depending on different aspects such as sequence length, number of tokens annotated by the system, token specificity, etc. On this basis, we introduce the \textbf{S}equence \textbf{L}abelling \textbf{I}nformation \textbf{C}ontrast \textbf{M}odel (SL-ICM), a novel metric based on information theory for evaluating sequence labeling tasks. Our formal analysis and experimentation show that the proposed metric satisfies all properties simultaneously"
}