RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation

Tzuf Paz-Argaman, Reut Tsarfaty

Abstract
Following navigation instructions in natural language (NL) requires a composition of language, action, and knowledge of the environment. Knowledge of the environment may be provided via visual sensors or as a symbolic world representation referred to as a map. Previous work on map-based NL navigation relied on small artificial worlds with a fixed set of entities known in advance. Here we introduce the Realistic Urban Navigation (RUN) task, aimed at interpreting NL navigation instructions based on a real, dense, urban map. Using Amazon Mechanical Turk, we collected a dataset of 2515 instructions aligned with actual routes over three regions of Manhattan. We then empirically study which aspects of a neural architecture are important for the RUN success, and empirically show that entity abstraction, attention over words and worlds, and a constantly updating world-state, significantly contribute to task accuracy.
Anthology ID:
D19-1681
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
6449–6455
Language:
URL:
https://aclanthology.org/D19-1681
DOI:
10.18653/v1/D19-1681
Bibkey:
Cite (ACL):
Tzuf Paz-Argaman and Reut Tsarfaty. 2019. RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6449–6455, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation (Paz-Argaman & Tsarfaty, EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp22-frontmatter/D19-1681.pdf
Attachment:
 D19-1681.Attachment.zip
Code
 OnlpLab/RUN
Data
RUN
PDF Search Code Attachment