Julio Gonzalo


2020

pdf
An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results
Enrique Amigo | Julio Gonzalo | Stefano Mizzaro | Jorge Carrillo-de-Albornoz
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

In Ordinal Classification tasks, items have to be assigned to classes that have a relative ordering, such as “positive”, “neutral”, “negative” in sentiment analysis. Remarkably, the most popular evaluation metrics for ordinal classification tasks either ignore relevant information (for instance, precision/recall on each of the classes ignores their relative ordering) or assume additional information (for instance, Mean Average Error assumes absolute distances between classes). In this paper we propose a new metric for Ordinal Classification, Closeness Evaluation Measure, that is rooted on Measurement Theory and Information Theory. Our theoretical analysis and experimental results over both synthetic data and data from NLP shared tasks indicate that the proposed metric captures quality aspects from different traditional tasks simultaneously. In addition, it generalizes some popular classification (nominal scale) and error minimization (interval scale) metrics, depending on the measurement scale in which it is instantiated.

2012

pdf
UNED: Improving Text Similarity Measures without Human Assessments
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Felisa Verdejo
*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)

pdf
Automatic Extraction of Polar Adjectives for the Creation of Polarity Lexicons
Silvia Vázquez | Muntsa Padró | Núria Bel | Julio Gonzalo
Proceedings of COLING 2012: Posters

pdf
The Heterogeneity Principle in Evaluation Measures for Automatic Summarization
Enrique Amigó | Julio Gonzalo | Felisa Verdejo
Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization

2011

pdf
Corroborating Text Evaluation Results with Heterogeneous Measures
Enrique Amigó | Julio Gonzalo | Jesús Giménez | Felisa Verdejo
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

2010

pdf
Wikipedia as Sense Inventory to Improve Diversity in Web Search Results
Celina Santamaría | Julio Gonzalo | Javier Artiles
Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics

2009

pdf
The Contribution of Linguistic Features to Automatic Machine Translation Evaluation
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Felisa Verdejo
Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP

pdf
The Impact of Query Refinement in the Web People Search Task
Javier Artiles | Julio Gonzalo | Enrique Amigó
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers

pdf
The role of named entities in Web People Search
Javier Artiles | Enrique Amigó | Julio Gonzalo
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2008

pdf
From Research to Application in Multilingual Information Access: the Contribution of Evaluation
Carol Peters | Martin Braschler | Giorgio Di Nunzio | Nicola Ferro | Julio Gonzalo | Mark Sanderson
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

The importance of evaluation in promoting research and development in the information retrieval and natural language processing domains has long been recognised but is this sufficient? In many areas there is still a considerable gap between the results achieved by the research community and their implementation in commercial applications. This is particularly true for the cross-language or multilingual retrieval areas. Despite the strong demand for and interest in multilingual IR functionality, there are still very few operational systems on offer. The Cross Language Evaluation Forum (CLEF) is now taking steps aimed at changing this situation. The paper provides a critical assessment of the main results achieved by CLEF so far and discusses plans now underway to extend its activities in order to have a more direct impact on the application sector.

2007

pdf
The SemEval-2007 WePS Evaluation: Establishing a benchmark for the Web People Search Task
Javier Artiles | Julio Gonzalo | Satoshi Sekine
Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007)

2006

pdf
MT Evaluation: Human-Like vs. Human Acceptable
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Lluís Màrquez
Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions

2005

pdf
Evaluating DUC 2004 Tasks with the QARLA Framework
Enrique Amigó | Julio Gonzalo | Anselmo Peñas | Felisa Verdejo
Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

pdf
QARLA: A Framework for the Evaluation of Text Summarization Systems
Enrique Amigó | Julio Gonzalo | Anselmo Peñas | Felisa Verdejo
Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)

2004

pdf
Using syntactic information to extract relevant terms for multi-document summarization
Enrique Amigó | Julio Gonzalo | Víctor Peinado | Anselmo Peñas | Felisa Verdejo
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

pdf
The Future of Evaluation for Cross-Language Information Retrieval Systems
Carol Peters | Martin Braschler | Khalid Choukri | Julio Gonzalo | Michael Kluck
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

The objective of the Cross-Language Evaluation Forum (CLEF) is to promote research in the multilingual information access domain. In this short paper, we list the achievements of CLEF during its first four years of activity and describe how the range of tasks has been considerably expanded during this period. The aim of the paper is to demonstrate the importance of evaluation initiatives with respect to system research and development and to show how essential it is for such initiatives to keep abreast of and even anticipate the emerging needs of both system developers and application communities if they are to have a future.

pdf
An Empirical Study of Information Synthesis Task
Enrique Amigo | Julio Gonzalo | Victor Peinado | Anselmo Peñas | Felisa Verdejo
Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)

2003

pdf
Automatic Association of Web Directories with Word Senses
Celina Santamaría | Julio Gonzalo | Felisa Verdejo
Computational Linguistics, Volume 29, Number 3, September 2003: Special Issue on the Web as Corpus

2002

pdf
A Study of Polysemy and Sense Proximity in the Senseval-2 Test Suite
Irina Chugur | Julio Gonzalo | Felisa Verdejo
Proceedings of the ACL-02 Workshop on Word Sense Disambiguation: Recent Successes and Future Directions

2001

pdf
Framework and Results for the Spanish SENSEVAL
German Rigau | Mariona Taulé | Ana Fernandez | Julio Gonzalo
Proceedings of SENSEVAL-2 Second International Workshop on Evaluating Word Sense Disambiguation Systems

pdf
The UNED Systems at SENSEVAL-2
David Fernández-Amorós | Julio Gonzalo | Felisa Verdejo
Proceedings of SENSEVAL-2 Second International Workshop on Evaluating Word Sense Disambiguation Systems

2000

pdf
Evaluating Wordnets in Cross-language Information Retrieval: the ITEM Search Engine
Felisa Verdejo | Julio Gonzalo | Anselmo Peñas | Fernando López | David Fernández
Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00)

pdf bib
Sense clusters for Information Retrieval: Evidence from Semcor and the EuroWordNet InterLingual Index
Julio Gonzalo | Irina Chugur | Felisa Verdejo
ACL-2000 Workshop on Word Senses and Multi-linguality

1999

pdf
Towards a Universal Index of Meaning
Piek Vossen | Wim Peters | Julio Gonzalo
SIGLEX99: Standardizing Lexical Resources

pdf
Lexical ambiguity and Information Retrieval revisited
Julio Gonzalo | Anselmo Penas | Felisa Verdejo
1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora

pdf
An Open Distance Learning Web-Course for NLP in IR
Felisa Verdejo | Julio Gonzalo | Anselmo Penas
EACL 1999: Computer and Internet Supported Education in Language and Speech Technology

1998

pdf
Indexing with WordNet synsets can improve text retrieval
Julio Gonzalo | Felisa Verdejo | Irina Chugur | Juan Cigarran
Usage of WordNet in Natural Language Processing Systems

1995

pdf
Generic Rules and Non-Constituent Coordination
Julio Gonzalo | Teresa Solías
Proceedings of the Fourth International Workshop on Parsing Technologies

We present a metagrammatical formalism, generic rules, to give a default interpretation to grammar rules. Our formalism introduces a process of dynamic binding interfacing the level of pure grammatical knowledge representation and the parsing level. We present an approach to non-constituent coordination within categorial grammars, and reformulate it as a generic rule. This reformulation is context-free parsable and reduces drastically the search space associated to the parsing task for such phenomena.