Corpus-Based methods for Short Text Similarity

Prajol Shrestha


Abstract
This paper presents corpus-based methods to find similarity between short text (sentences, paragraphs, ...) which has many applications in the field of NLP. Previous works on this problem have been based on supervised methods or have used external resources such as WordNet, British National Corpus etc. Our methods are focused on unsupervised corpus-based methods. We present a new method, based on Vector Space Model, to capture the contextual behavior, senses and correlation, of terms and show that this method performs better than the baseline method that uses vector based cosine similarity measure. The performance of existing document similarity measures, Dice and Resemblance, are also evaluated which in our knowledge have not been used for short text similarity. We also show that the performance of the vector-based baseline method is improved when using stems instead of words and using the candidate sentences for computing the parameters rather than some external resource.
Anthology ID:
2011.jeptalnrecital-recitalcourt.1
Volume:
Actes de la 18e conférence sur le Traitement Automatique des Langues Naturelles. REncontres jeunes Chercheurs en Informatique pour le Traitement Automatique des Langues (articles courts)
Month:
June
Year:
2011
Address:
Montpellier, France
Venue:
JEP/TALN/RECITAL
SIG:
Publisher:
ATALA
Note:
Pages:
1–6
Language:
URL:
https://aclanthology.org/2011.jeptalnrecital-recitalcourt.1
DOI:
Bibkey:
Cite (ACL):
Prajol Shrestha. 2011. Corpus-Based methods for Short Text Similarity. In Actes de la 18e conférence sur le Traitement Automatique des Langues Naturelles. REncontres jeunes Chercheurs en Informatique pour le Traitement Automatique des Langues (articles courts), pages 1–6, Montpellier, France. ATALA.
Cite (Informal):
Corpus-Based methods for Short Text Similarity (Shrestha, JEP/TALN/RECITAL 2011)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2011.jeptalnrecital-recitalcourt.1.pdf