Comparing Similarity Measures for Distributional Thesauri
Muntsa Padró, Marco Idiart, Aline Villavicencio, Carlos Ramisch
Abstract
Distributional thesauri have been applied for a variety of tasks involving semantic relatedness. In this paper, we investigate the impact of three parameters: similarity measures, frequency thresholds and association scores. We focus on the robustness and stability of the resulting thesauri, measuring inter-thesaurus agreement when testing different parameter values. The results obtained show that low-frequency thresholds affect thesaurus quality more than similarity measures, with more agreement found for increasing thresholds. These results indicate the sensitivity of distributional thesauri to frequency. Nonetheless, the observed differences do not transpose over extrinsic evaluation using TOEFL-like questions. While this may be specific to the task, we argue that a careful examination of the stability of distributional resources prior to application is needed.- Anthology ID:
- L14-1496
- Volume:
- Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
- Month:
- May
- Year:
- 2014
- Address:
- Reykjavik, Iceland
- Editors:
- Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association (ELRA)
- Note:
- Pages:
- 2964–2971
- Language:
- URL:
- http://www.lrec-conf.org/proceedings/lrec2014/pdf/619_Paper.pdf
- DOI:
- Cite (ACL):
- Muntsa Padró, Marco Idiart, Aline Villavicencio, and Carlos Ramisch. 2014. Comparing Similarity Measures for Distributional Thesauri. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2964–2971, Reykjavik, Iceland. European Language Resources Association (ELRA).
- Cite (Informal):
- Comparing Similarity Measures for Distributional Thesauri (Padró et al., LREC 2014)
- PDF:
- http://www.lrec-conf.org/proceedings/lrec2014/pdf/619_Paper.pdf