This paper describes our team’s submission of the EACL DravidianLangTech-2021’s shared task on Machine Translation of Dravidian languages. We submitted our translations for English to Malayalam , Tamil , Telugu and also Tamil-Telugu language pairs. The submissions mainly focus on having adequate amount of data backed up by good preprocessing of it to produce quality translations,which includes some custom made rules to remove unnecessary sentences. We conducted several experiments on these models by tweaking the architecture,Byte Pair Encoding (BPE) and other hyperparameters.