G3: Geolocation via Guidebook Grounding
Grace Luo, Giscard Biamby, Trevor Darrell, Daniel Fried, Anna Rohrbach
Abstract
We demonstrate how language can improve geolocation: the task of predicting the location where an image was taken. Here we study explicit knowledge from human-written guidebooks that describe the salient and class-discriminative visual features humans use for geolocation. We propose the task of Geolocation via Guidebook Grounding that uses a dataset of StreetView images from a diverse set of locations and an associated textual guidebook for GeoGuessr, a popular interactive geolocation game. Our approach predicts a country for each image by attending over the clues automatically extracted from the guidebook. Supervising attention with country-level pseudo labels achieves the best performance. Our approach substantially outperforms a state-of-the-art image-only geolocation method, with an improvement of over 5% in Top-1 accuracy. Our dataset and code can be found at https://github.com/g-luo/geolocation_via_guidebook_grounding.- Anthology ID:
- 2022.findings-emnlp.430
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5841–5853
- Language:
- URL:
- https://aclanthology.org/2022.findings-emnlp.430
- DOI:
- 10.18653/v1/2022.findings-emnlp.430
- Cite (ACL):
- Grace Luo, Giscard Biamby, Trevor Darrell, Daniel Fried, and Anna Rohrbach. 2022. G3: Geolocation via Guidebook Grounding. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5841–5853, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- G3: Geolocation via Guidebook Grounding (Luo et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/2022.findings-emnlp.430.pdf