Semantic Relatedness Based Re-ranker for Text Spotting

Ahmed Sabir, Francesc Moreno, Lluís Padró

Abstract
Applications such as textual entailment, plagiarism detection or document clustering rely on the notion of semantic similarity, and are usually approached with dimension reduction techniques like LDA or with embedding-based neural approaches. We present a scenario where semantic similarity is not enough, and we devise a neural approach to learn semantic relatedness. The scenario is text spotting in the wild, where a text in an image (e.g. street sign, advertisement or bus destination) must be identified and recognized. Our goal is to improve the performance of vision systems by leveraging semantic information. Our rationale is that the text to be spotted is often related to the image context in which it appears (word pairs such as Delta-airplane, or quarters-parking are not similar, but are clearly related). We show how learning a word-to-word or word-to-sentence relatedness score can improve the performance of text spotting systems up to 2.9 points, outperforming other measures in a benchmark dataset.
Anthology ID:
D19-1346
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3451–3457
Language:
URL:
https://aclanthology.org/D19-1346
DOI:
10.18653/v1/D19-1346
Bibkey:
Cite (ACL):
Ahmed Sabir, Francesc Moreno, and Lluís Padró. 2019. Semantic Relatedness Based Re-ranker for Text Spotting. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3451–3457, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Semantic Relatedness Based Re-ranker for Text Spotting (Sabir et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/D19-1346.pdf
Code
 ahmedssabir/Semantic-Relatedness-Based-Reranker-for-Text-Spotting +  additional community code
PDF Search Code