Question Similarity in Community Question Answering: A Systematic Exploration of Preprocessing Methods and Models
Florian Kunneman, Thiago Castro Ferreira, Emiel Krahmer, Antal van den Bosch
Abstract
Community Question Answering forums are popular among Internet users, and a basic problem they encounter is trying to find out if their question has already been posed before. To address this issue, NLP researchers have developed methods to automatically detect question-similarity, which was one of the shared tasks in SemEval. The best performing systems for this task made use of Syntactic Tree Kernels or the SoftCosine metric. However, it remains unclear why these methods seem to work, whether their performance can be improved by better preprocessing methods and what kinds of errors they (and other methods) make. In this paper, we therefore systematically combine and compare these two approaches with the more traditional BM25 and translation-based models. Moreover, we analyze the impact of preprocessing steps (lowercasing, suppression of punctuation and stop words removal) and word meaning similarity based on different distributions (word translation probability, Word2Vec, fastText and ELMo) on the performance of the task. We conduct an error analysis to gain insight into the differences in performance between the system set-ups. The implementation is made publicly available from https://github.com/fkunneman/DiscoSumo/tree/master/ranlp.- Anthology ID:
- R19-1070
- Volume:
- Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)
- Month:
- September
- Year:
- 2019
- Address:
- Varna, Bulgaria
- Editors:
- Ruslan Mitkov, Galia Angelova
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd.
- Note:
- Pages:
- 593–601
- Language:
- URL:
- https://aclanthology.org/R19-1070
- DOI:
- 10.26615/978-954-452-056-4_070
- Cite (ACL):
- Florian Kunneman, Thiago Castro Ferreira, Emiel Krahmer, and Antal van den Bosch. 2019. Question Similarity in Community Question Answering: A Systematic Exploration of Preprocessing Methods and Models. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 593–601, Varna, Bulgaria. INCOMA Ltd..
- Cite (Informal):
- Question Similarity in Community Question Answering: A Systematic Exploration of Preprocessing Methods and Models (Kunneman et al., RANLP 2019)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/R19-1070.pdf
- Code
- fkunneman/DiscoSumo