# Requirements and Installation

* [PyTorch](http://pytorch.org/) version == 1.4.0
* Python version >= 3.6

# Running Steps
The followings are the steps for conducting our experiments.

- **Data**
  - For WMT'19 English-to-German dataset, please refer to http://www.statmt.org/wmt14/translation-task.html
  - For WMT'19 Chinese-to-English dataset, please refer to http://www.statmt.org/wmt19/translation-task.html

- **Preprocess**
```bash
$root/preprocess_chen_fairseq.sh
```

- **Stage1: multi-task joint training**
```bash
$root/stage1_train_chen.sh joint 1.0 1.0 0.7
```

- **Stage2: confidence based knowledge distillation**

The checkpoint at the 200,000 training steps of the stage 1 is used as the initialization for stage 2.
First, create the directory
```bash
mkdir ${root}/checkpoints/wmt19_ch_en/${model_signature}
```
Then, move the specified checkpoint to this directory and start the second training stage.
```bash
$root/stage2_train_chen.sh ${model_signature} 1.0 1.0 200000 anneal 1.0 1.0 at_below_threshold 0.2
```

- **Evaluation**
```bash
$root/eval_chen.sh ${model_signature} test checkpoint_best
```