Multimodal Learning for Accurate Visual Question Answering: An Attention-Based Approach
Jishnu Bhardwaj, Anurag Balakrishnan, Satyam Pathak, Ishan Unnarkar, Aniruddha Gawande, Benyamin Ahmadnia
Abstract
This paper proposes an open-ended task for Visual Question Answering (VQA) that leverages the InceptionV3 Object Detection model and an attention-based Long Short-Term Memory (LSTM) network for question answering. Our proposed model provides accurate natural language answers to questions about an image, including those that require understanding contextual information and background details. Our findings demonstrate that the proposed approach can achieve high accuracy, even with complex and varied visual information. The proposed method can contribute to developing more advanced vision systems that can process and interpret visual information like humans.- Anthology ID:
- 2023.ranlp-1.20
- Volume:
- Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing
- Month:
- September
- Year:
- 2023
- Address:
- Varna, Bulgaria
- Editors:
- Ruslan Mitkov, Galia Angelova
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd., Shoumen, Bulgaria
- Note:
- Pages:
- 179–186
- Language:
- URL:
- https://aclanthology.org/2023.ranlp-1.20
- DOI:
- Cite (ACL):
- Jishnu Bhardwaj, Anurag Balakrishnan, Satyam Pathak, Ishan Unnarkar, Aniruddha Gawande, and Benyamin Ahmadnia. 2023. Multimodal Learning for Accurate Visual Question Answering: An Attention-Based Approach. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 179–186, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
- Cite (Informal):
- Multimodal Learning for Accurate Visual Question Answering: An Attention-Based Approach (Bhardwaj et al., RANLP 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2023.ranlp-1.20.pdf