# README

## Rewrite Model Training

The training scripts are in the `HALOs` folder,  and can be easily run by the following command.

```bash
python3 train.py loss=[dpo,kto,sft] model=qwen7b datasets=[rewrite_feedback_example] exp_name=qwen7b_feedback mode=train model.batch_size=16 model.gradient_accumulation_steps=2 n_epochs=2 lr=5e-6 
```

## Rewrite and Ranking

The inference of trained rewrite model is in `rewrite.py`, and the ranking process is implemented by `ranking_bge.py`.