Emergence of Abstract State Representations in Embodied Sequence Modeling
Tian Yun, Zilai Zeng, Kunal Handa, Ashish Thapliyal, Bo Pang, Ellie Pavlick, Chen Sun
Abstract
Decision making via sequence modeling aims to mimic the success of language models, where actions taken by an embodied agent are modeled as tokens to predict. Despite their promising performance, it remains unclear if embodied sequence modeling leads to the emergence of internal representations that represent the environmental state information. A model that lacks abstract state representations would be liable to make decisions based on surface statistics which fail to generalize. We take the BabyAI environment, a grid world in which language-conditioned navigation tasks are performed, and build a sequence modeling Transformer, which takes a language instruction, a sequence of actions, and environmental observations as its inputs. In order to investigate the emergence of abstract state representations, we design a “blindfolded” navigation task, where only the initial environmental layout, the language instruction, and the action sequence to complete the task are available for training. Our probing results show that intermediate environmental layouts can be reasonably reconstructed from the internal activations of a trained model, and that language instructions play a role in the reconstruction accuracy. Our results suggest that many key features of state representations can emerge via embodied sequence modeling, supporting an optimistic outlook for applications of sequence modeling objectives to more complex embodied decision-making domains.- Anthology ID:
- 2023.emnlp-main.749
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 12190–12205
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.749
- DOI:
- 10.18653/v1/2023.emnlp-main.749
- Cite (ACL):
- Tian Yun, Zilai Zeng, Kunal Handa, Ashish Thapliyal, Bo Pang, Ellie Pavlick, and Chen Sun. 2023. Emergence of Abstract State Representations in Embodied Sequence Modeling. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12190–12205, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Emergence of Abstract State Representations in Embodied Sequence Modeling (Yun et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2023.emnlp-main.749.pdf