# Offensive Language Detection in Dravidian Languages

The code for the shared task - Offensive Language Identification in Dravidian Languages (EACL 2021) is contained in this repository

Submitted by `Team Professionals`:
* [Srinath Nair]
* [Dolton Fernandes]

Organisation: IIIT Hyderabad

## Requirements


## Data Preparation

* Transliteration of the codemixed dataset into native script:
    * `python preprocessing/preprocessing.py -i data/mal_full_offensive_train.csv`
    
Now the file containing the transliterated sentences will be found at the home directory of the repository. Copy it to the `working` directory.

* Generating BERT embeddings:
    * `python embeddings/embeddings_gen.py -i working/mal_transliterate_train.csv -o intermediateTrain/train.bert.tsv`
    * `python embeddings/embeddings_gen.py -i working/mal_transliterate_dev.csv -o embeddings/dev.bert.tsv`
    * `python intermediateTrain/combineTrain.py`

* Building the vocabulary and checking the count:
    * `python analytics/vocab.py -i <path-to-input-file>`

## Link To Extracted Features
* [Train/Dev Features with Labels](https://drive.google.com/drive/folders/1zm1xrfUyijz8TWJup2uXpsKkw06ibEEN?usp=sharing)
