### 1. File Structure Description

This folder contains the source code and data used in our experiments.

```
.
├── src/
│   ├── evaluation/
│   │   └── # code to evaluate the results
│   ├── tree_plv/
│   │   └── # source code implementing tree search
│   └── verifier_training/
│       └── # code for training the verifier
├── benchmarks/
│   └── # benckmarks used in our experiments
└── data/
    └── # examples of training data
```



### 2. Installation

Make sure to use Python 3.10 or later.

```
pip install deepspeed>=0.9.0
pip install -r requirements.txt
```



### 3. Tree Search

An example of performing the tree search

```
python src/tree_plv/vllm_gsm8k/inference.py --base_lm vllm --hf_path your/path/to/model --n_action 3 --n_iters 10 --tensor_parrallel_size 2
```



###  4. How to train the verifier

We provide the script for training, which you can test by launching the command.

```
bash src/verifier_training/training/run.sh
```

We also provide an evaluation script, `rw_eval.py`,  to use the verifier for best-of-N evaluation. Also, see below for an  example.

```
bash src/verifier_training/training/rw_eval.sh
```