# Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Framework


## Overview

This project implements pipelines for:
- **Dataset Processing & Translation**: Tools to process and translate major VLM datasets (LLaVA Instruct, MMBench, etc.) into Polish.
- **Model Training**: Scripts for pretraining and finetuning LLaVA-style models using `accelerate` and `deepspeed`.
- **Evaluation**: Comprehensive evaluation benchmarks including MMBench, XM3600 and custom evaluations.

## Project Structure

- `config/`: Configuration files for judges, translation models, and validation settings.
- `notebooks/`: Jupyter notebooks for data preparation, analysis, and interactive evaluation.
- `scripts/`: Bash scripts for automating tasks:
  - `datasets/`: Data processing and translation scripts.
  - `eval/`: Evaluation scripts for various benchmarks.
  - `training/`: Training launch scripts.
- `src/`: Core Python source code:
  - `datasets/`: Dataset loading and processing logic.
  - `modeling/`: Model definitions (LLaVA, Gemma, Mistral).
  - `utils/`: Utility functions.

## Installation

1. Clone the repository.
2. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```

## Usage

### Data Processing
Scripts for processing datasets are located in `scripts/datasets/`. For example, to process LLaVA Instruct data:
```bash
bash scripts/datasets/llava_process_instruct.sh
```

### Training
Training scripts are in `scripts/training/`. To launch finetuning:
```bash
bash scripts/training/llava_finetune.sh
```
*Note: You may need to adjust paths and environment variables in the scripts.*

### Evaluation
Evaluation scripts are in `scripts/eval/`. To evaluate on MMBench:
```bash
bash scripts/eval/mmbench_evaluate.sh
```
