MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces
Tianyu Zheng, Ge Zhang, Xingwei Qu, Ming Kuang, Wenhao Huang, Zhaofeng He
Abstract
Drawing upon the intuition that aligning different modalities to the same semantic embedding space would allow models to understand states and actions more easily, we propose a new perspective to the offline reinforcement learning (RL) challenge. More concretely, we transform it into a supervised learning task by integrating multimodal and pre-trained language models. Our approach incorporates state information derived from images and action-related data obtained from text, thereby bolstering RL training performance and promoting long-term strategic thinking. We emphasize the contextual understanding of language and demonstrate how decision-making in RL can benefit from aligning states’ and actions’ representation with languages’ representation. Our method significantly outperforms current baselines as evidenced by evaluations conducted on Atari and OpenAI Gym environments. This contributes to advancing offline RL performance and efficiency while providing a novel perspective on offline RL.- Anthology ID:
- 2024.lrec-main.1013
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 11593–11604
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.1013
- DOI:
- Cite (ACL):
- Tianyu Zheng, Ge Zhang, Xingwei Qu, Ming Kuang, Wenhao Huang, and Zhaofeng He. 2024. MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 11593–11604, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces (Zheng et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2024.lrec-main.1013.pdf