Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
Sonia Raychaudhuri, Saim Wani, Shivansh Patel, Unnat Jain, Angel Chang
Abstract
In the Vision-and-Language Navigation (VLN) task an embodied agent navigates a 3D environment, following natural language instructions. A challenge in this task is how to handle ‘off the path’ scenarios where an agent veers from a reference path. Prior work supervises the agent with actions based on the shortest path from the agent’s location to the goal, but such goal-oriented supervision is often not in alignment with the instruction. Furthermore, the evaluation metrics employed by prior work do not measure how much of a language instruction the agent is able to follow. In this work, we propose a simple and effective language-aligned supervision scheme, and a new metric that measures the number of sub-instructions the agent has completed during navigation.- Anthology ID:
- 2021.emnlp-main.328
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4018–4028
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.328
- DOI:
- 10.18653/v1/2021.emnlp-main.328
- Cite (ACL):
- Sonia Raychaudhuri, Saim Wani, Shivansh Patel, Unnat Jain, and Angel Chang. 2021. Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4018–4028, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments (Raychaudhuri et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2021.emnlp-main.328.pdf