@inproceedings{azime-etal-2024-walia,
    title = "Walia-{LLM}: Enhancing {A}mharic-{LL}a{MA} by Integrating Task-Specific and Generative Datasets",
    author = "Azime, Israel Abebe  and
      Tonja, Atnafu Lambebo  and
      Belay, Tadesse Destaw  and
      Fuge, Mitiku Yohannes  and
      Wassie, Aman Kassahun  and
      Jada, Eyasu Shiferaw  and
      Chanie, Yonas  and
      Sewunetie, Walelign Tewabe  and
      Yimam, Seid Muhie",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.findings-emnlp.25/",
    doi = "10.18653/v1/2024.findings-emnlp.25",
    pages = "432--444",
    abstract = "Large language models (LLMs) have received a lot of attention in natural language processing (NLP) research because of their exceptional performance in understanding and generating human languages. However, low-resource languages are left behind due to the unavailability of resources. In this work, we focus on enhancing the LLaMA-2-Amharic model by integrating task-specific and generative datasets to improve language model performance for Amharic. We compile an Amharic instruction fine-tuning dataset and fine-tuned LLaMA-2-Amharic model. The fine-tuned model shows promising results in different NLP tasks. We also explore the effectiveness of translated instruction datasets compared to the dataset we created. Our dataset creation pipeline, along with instruction datasets, trained models, and evaluation outputs, is made publicly available to encourage research in language-specific models."
}