Persons with severe speech and motor impairments (SSMI), like those with cerebral palsy (CP) experience significant challenges via communication in conventional methods. Many a times they rely on Graphical symbol-based Augmentative and Alternative Communication (AAC) systems to facilitate the communication. Our work aims to support AAC communication by developing specialized datasets for direct translation of Graphical Symbols to Natural Language text. The dataset is enhanced with an automated Text-to-Pictogram generation module. The dataset is enriched with some additive information like tense-based information and subjective information (questionnaires, exclamations). Additionally, we expanded our efforts to include translation into Indian language Bengali, for those individuals with SSMI who are more comfortable communicating in their native language. We aim to develop an end-to-end language agnostic framework for efficient bidirectional communication between non-verbal AAC picture symbols and textual data.