Suilan Estevez-Velarde


2021

pdf
Active Learning for Assisted Corpus Construction: A Case Study in Knowledge Discovery from Biomedical Text
Hian Cañizares-Díaz | Alejandro Piad-Morffis | Suilan Estevez-Velarde | Yoan Gutiérrez | Yudivián Almeida Cruz | Andres Montoyo | Rafael Muñoz-Guillena
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)

This paper presents an active learning approach that aims to reduce the human effort required during the annotation of natural language corpora composed of entities and semantic relations. Our approach assists human annotators by intelligently selecting the most informative sentences to annotate and then pre-annotating them with a few highly accurate entities and semantic relations. We define an uncertainty-based query strategy with a weighted density factor, using similarity metrics based on sentence embeddings. As a case study, we evaluate our approach via simulation in a biomedical corpus and estimate the potential reduction in total annotation time. Experimental results suggest that the query strategy reduces by between 35% and 40% the number of sentences that must be manually annotated to develop systems able to reach a target F1 score, while the pre-annotation strategy produces an additional 24% reduction in the total annotation time. Overall, our preliminary experiments suggest that as much as 60% of the annotation time could be saved while producing corpora that have the same usefulness for training machine learning algorithms. An open-source computational tool that implements the aforementioned strategies is presented and published online for the research community.

pdf
Knowledge Discovery in COVID-19 Research Literature
Ernesto L. Estevanell-Valladares | Suilan Estevez-Velarde | Alejandro Piad-Morffis | Yoan Gutierrez | Andres Montoyo | Rafael Muñoz | Yudivián Almeida Cruz
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)

This paper presents the preliminary results of an ongoing project that analyzes the growing body of scientific research published around the COVID-19 pandemic. In this research, a general-purpose semantic model is used to double annotate a batch of 500 sentences that were manually selected from the CORD-19 corpus. Afterwards, a baseline text-mining pipeline is designed and evaluated via a large batch of 100,959 sentences. We present a qualitative analysis of the most interesting facts automatically extracted and highlight possible future lines of development. The preliminary results show that general-purpose semantic models are a useful tool for discovering fine-grained knowledge in large corpora of scientific documents.

2020

pdf
Knowledge Discovery in COVID-19 Research Literature
Alejandro Piad-Morffis | Suilan Estevez-Velarde | Ernesto Luis Estevanell-Valladares | Yoan Gutiérrez | Andrés Montoyo | Rafael Muñoz | Yudivián Almeida-Cruz
Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020

This paper presents the preliminary results of an ongoing project that analyzes the growing body of scientific research published around the COVID-19 pandemic. In this research, a general-purpose semantic model is used to double annotate a batch of $500$ sentences that were manually selected by the researchers from the CORD-19 corpus. Afterwards, a baseline text-mining pipeline is designed and evaluated via a large batch of $100,959$ sentences. We present a qualitative analysis of the most interesting facts automatically extracted and highlight possible future lines of development. The preliminary results show that general-purpose semantic models are a useful tool for discovering fine-grained knowledge in large corpora of scientific documents.

pdf
Automatic Discovery of Heterogeneous Machine Learning Pipelines: An Application to Natural Language Processing
Suilan Estevez-Velarde | Yoan Gutiérrez | Andres Montoyo | Yudivián Almeida Cruz
Proceedings of the 28th International Conference on Computational Linguistics

This paper presents AutoGOAL, a system for automatic machine learning (AutoML) that uses heterogeneous techniques. In contrast with existing AutoML approaches, our contribution can automatically build machine learning pipelines that combine techniques and algorithms from different frameworks, including shallow classifiers, natural language processing tools, and neural networks. We define the heterogeneous AutoML optimization problem as the search for the best sequence of algorithms that transforms specific input data into the desired output. This provides a novel theoretical and practical approach to AutoML. Our proposal is experimentally evaluated in diverse machine learning problems and compared with alternative approaches, showing that it is competitive with other AutoML alternatives in standard benchmarks. Furthermore, it can be applied to novel scenarios, such as several NLP tasks, where existing alternatives cannot be directly deployed. The system is freely available and includes in-built compatibility with a large number of popular machine learning frameworks, which makes our approach useful for solving practical problems with relative ease and effort.

pdf
Demo Application for the AutoGOAL Framework
Suilan Estevez-Velarde | Alejandro Piad-Morffis | Yoan Gutiérrez | Andres Montoyo | Rafael Muñoz-Guillena | Yudivián Almeida Cruz
Proceedings of the 28th International Conference on Computational Linguistics: System Demonstrations

This paper introduces a web demo that showcases the main characteristics of the AutoGOAL framework. AutoGOAL is a framework in Python for automatically finding the best way to solve a given task. It has been designed mainly for automatic machine learning(AutoML) but it can be used in any scenario where several possible strategies are available to solve a given computational task. In contrast with alternative frameworks, AutoGOAL can be applied seamlessly to Natural Language Processing as well as structured classification problems. This paper presents an overview of the framework’s design and experimental evaluation in several machine learning problems, including two recent NLP challenges. The accompanying software demo is available online (https://autogoal.github.io/demo) and full source code is provided under the MIT open-source license (https://autogoal.github.io).

2019

pdf
Demo Application for LETO: Learning Engine Through Ontologies
Suilan Estevez-Velarde | Andrés Montoyo | Yudivian Almeida-Cruz | Yoan Gutiérrez | Alejandro Piad-Morffis | Rafael Muñoz
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)

The massive amount of multi-formatted information available on the Web necessitates the design of software systems that leverage this information to obtain knowledge that is valid and useful. The main challenge is to discover relevant information and continuously update, enrich and integrate knowledge from various sources of structured and unstructured data. This paper presents the Learning Engine Through Ontologies(LETO) framework, an architecture for the continuous and incremental discovery of knowledge from multiple sources of unstructured and structured data. We justify the main design decision behind LETO’s architecture and evaluate the framework’s feasibility using the Internet Movie Data Base(IMDB) and Twitter as a practical application.

pdf
A Neural Network Component for Knowledge-Based Semantic Representations of Text
Alejandro Piad-Morffis | Rafael Muñoz | Yoan Gutiérrez | Yudivian Almeida-Cruz | Suilan Estevez-Velarde | Andrés Montoyo
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)

This paper presents Semantic Neural Networks (SNNs), a knowledge-aware component based on deep learning. SNNs can be trained to encode explicit semantic knowledge from an arbitrary knowledge base, and can subsequently be combined with other deep learning architectures. At prediction time, SNNs provide a semantic encoding extracted from the input data, which can be exploited by other neural network components to build extended representation models that can face alternative problems. The SNN architecture is defined in terms of the concepts and relations present in a knowledge base. Based on this architecture, a training procedure is developed. Finally, an experimental setup is presented to illustrate the behaviour and performance of a SNN for a specific NLP problem, in this case, opinion mining for the classification of movie reviews.

pdf
A General-Purpose Annotation Model for Knowledge Discovery: Case Study in Spanish Clinical Text
Alejandro Piad-Morffis | Yoan Guitérrez | Suilan Estevez-Velarde | Rafael Muñoz
Proceedings of the 2nd Clinical Natural Language Processing Workshop

Knowledge discovery from text in natural language is a task usually aided by the manual construction of annotated corpora. Specifically in the clinical domain, several annotation models are used depending on the characteristics of the task to solve (e.g., named entity recognition, relation extraction, etc.). However, few general-purpose annotation models exist, that can support a broad range of knowledge extraction tasks. This paper presents an annotation model designed to capture a large portion of the semantics of natural language text. The structure of the annotation model is presented, with examples of annotated sentences and a brief description of each semantic role and relation defined. This research focuses on an application to clinical texts in the Spanish language. Nevertheless, the presented annotation model is extensible to other domains and languages. An example of annotated sentences, guidelines, and suitable configuration files for an annotation tool are also provided for the research community.

pdf
AutoML Strategy Based on Grammatical Evolution: A Case Study about Knowledge Discovery from Text
Suilan Estevez-Velarde | Yoan Gutiérrez | Andrés Montoyo | Yudivián Almeida-Cruz
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

The process of extracting knowledge from natural language text poses a complex problem that requires both a combination of machine learning techniques and proper feature selection. Recent advances in Automatic Machine Learning (AutoML) provide effective tools to explore large sets of algorithms, hyper-parameters and features to find out the most suitable combination of them. This paper proposes a novel AutoML strategy based on probabilistic grammatical evolution, which is evaluated on the health domain by facing the knowledge discovery challenge in Spanish text documents. Our approach achieves state-of-the-art results and provides interesting insights into the best combination of parameters and algorithms to use when dealing with this challenge. Source code is provided for the research community.