Enrique Amigó

Also published as: Enrique Amigo


2024

pdf
A Web Portal about the State of the Art of NLP Tasks in Spanish
Enrique Amigó | Jorge Carrillo-de-Albornoz | Andrés Fernández | Julio Gonzalo | Guillermo Marco | Roser Morante | Laura Plaza | Jacobo Pedrosa
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

This paper presents a new web portal with information about the state of the art of natural language processing tasks in Spanish. It provides information about forums, competitions, tasks and datasets in Spanish, that would otherwise be spread in multiple articles and web sites. The portal consists of overview pages where information can be searched for and filtered by several criteria and individual pages with detailed information and hyperlinks to facilitate navigation. Information has been manually curated from publications that describe competitions and NLP tasks from 2013 until 2023 and will be updated as new tasks appear. A total of 185 tasks and 128 datasets from 94 competitions have been introduced.

2022

pdf
Information Theory–based Compositional Distributional Semantics
Enrique Amigó | Alejandro Ariza-Casabona | Victor Fresno | M. Antònia Martí
Computational Linguistics, Volume 48, Issue 4 - December 2022

In the context of text representation, Compositional Distributional Semantics models aim to fuse the Distributional Hypothesis and the Principle of Compositionality. Text embedding is based on co-ocurrence distributions and the representations are in turn combined by compositional functions taking into account the text structure. However, the theoretical basis of compositional functions is still an open issue. In this article we define and study the notion of Information Theory–based Compositional Distributional Semantics (ICDS): (i) We first establish formal properties for embedding, composition, and similarity functions based on Shannon’s Information Theory; (ii) we analyze the existing approaches under this prism, checking whether or not they comply with the established desirable properties; (iii) we propose two parameterizable composition and similarity functions that generalize traditional approaches while fulfilling the formal properties; and finally (iv) we perform an empirical study on several textual similarity datasets that include sentences with a high and low lexical overlap, and on the similarity between words and their description. Our theoretical analysis and empirical results show that fulfilling formal properties affects positively the accuracy of text representation models in terms of correspondence (isometry) between the embedding and meaning spaces.

pdf
Evaluating Extreme Hierarchical Multi-label Classification
Enrique Amigo | Agustín Delgado
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Several natural language processing (NLP) tasks are defined as a classification problem in its most complex form: Multi-label Hierarchical Extreme classification, in which items may be associated with multiple classes from a set of thousands of possible classes organized in a hierarchy and with a highly unbalanced distribution both in terms of class frequency and the number of labels per item. We analyze the state of the art of evaluation metrics based on a set of formal properties and we define an information theoretic based metric inspired by the Information Contrast Model (ICM). Experiments on synthetic data and a case study on real data show the suitability of the ICM for such scenarios.

2020

pdf
An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results
Enrique Amigo | Julio Gonzalo | Stefano Mizzaro | Jorge Carrillo-de-Albornoz
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

In Ordinal Classification tasks, items have to be assigned to classes that have a relative ordering, such as “positive”, “neutral”, “negative” in sentiment analysis. Remarkably, the most popular evaluation metrics for ordinal classification tasks either ignore relevant information (for instance, precision/recall on each of the classes ignores their relative ordering) or assume additional information (for instance, Mean Average Error assumes absolute distances between classes). In this paper we propose a new metric for Ordinal Classification, Closeness Evaluation Measure, that is rooted on Measurement Theory and Information Theory. Our theoretical analysis and experimental results over both synthetic data and data from NLP shared tasks indicate that the proposed metric captures quality aspects from different traditional tasks simultaneously. In addition, it generalizes some popular classification (nominal scale) and error minimization (interval scale) metrics, depending on the measurement scale in which it is instantiated.

2012

pdf
The Heterogeneity Principle in Evaluation Measures for Automatic Summarization
Enrique Amigó | Julio Gonzalo | Felisa Verdejo
Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization

pdf
UNED: Improving Text Similarity Measures without Human Assessments
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Felisa Verdejo
*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)

2011

pdf
Corroborating Text Evaluation Results with Heterogeneous Measures
Enrique Amigó | Julio Gonzalo | Jesús Giménez | Felisa Verdejo
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

2009

pdf
The Contribution of Linguistic Features to Automatic Machine Translation Evaluation
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Felisa Verdejo
Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP

pdf
The Impact of Query Refinement in the Web People Search Task
Javier Artiles | Julio Gonzalo | Enrique Amigó
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers

pdf
The role of named entities in Web People Search
Javier Artiles | Enrique Amigó | Julio Gonzalo
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2006

pdf
MT Evaluation: Human-Like vs. Human Acceptable
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Lluís Màrquez
Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions

pdf
Iqmt: A Framework for Automatic Machine Translation Evaluation
Jesús Giménez | Enrique Amigó
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

We present the IQMT Framework for Machine Translation Evaluation Inside QARLA. IQMT offers a common workbench in which existing evaluation metrics can be utilized and combined. It provides i) a measure to evaluate the quality of any set of similarity metrics (KING), ii) a measure to evaluate the quality of a translation using a set of similarity metrics (QUEEN), and iii) a measure to evaluate the reliability of a test set (JACK). The first release of the IQMT package is freely available for public use. Current version includes a set of 26 metrics from 7 different well-known metric families, and allows the user to supply its own metrics. For future releases, we are working on the design of new metrics that are able to capture linguistic aspects of translation beyond lexical ones.

2005

pdf
Evaluating DUC 2004 Tasks with the QARLA Framework
Enrique Amigó | Julio Gonzalo | Anselmo Peñas | Felisa Verdejo
Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

pdf
QARLA: A Framework for the Evaluation of Text Summarization Systems
Enrique Amigó | Julio Gonzalo | Anselmo Peñas | Felisa Verdejo
Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)

2004

pdf
An Empirical Study of Information Synthesis Task
Enrique Amigo | Julio Gonzalo | Victor Peinado | Anselmo Peñas | Felisa Verdejo
Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)

pdf
Using syntactic information to extract relevant terms for multi-document summarization
Enrique Amigó | Julio Gonzalo | Víctor Peinado | Anselmo Peñas | Felisa Verdejo
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics