# Readme

There are 3 ipynb files that must be executed in order (`project_1` first, `project_2` second, `project_3` third). The variable "language" must be changed to each different language when running a jupyter notebook file. In between running these files, you are required to run the models **UDPipe**, **Elephant**, and **MaChAMP** to generate the results needed to run the remaining code. You will have to execute the ipynb file for each language separately by changing the variable language and then hitting the "run all" button. 

---

### 1. PMB Data Setup

Download the PMB data v. 4.0.0 , this should include Dutch, English, Italian, and German. Unzip them in the given folder and save the file path to the variable `path_to_pmb` in the `project_1.ipynb` file. Now run the `project_1.ipynb` file.

---

### 2. UDPipe

Download the UDPipe 1.3.1 version from here: https://github.com/ufal/udpipe/releases into the folder where the jupyter notebook files are located and compile it. The language models used for this project can be downloaded from the official repository or taken from the udpipe_models folder, add these to udpipe-1.3.1-bin/src/:

- English model: `english-ewt-ud-2.5-191206.udpipe`
- German model: `german-hdt-ud-2.5-191206.udpipe`
- Italian model: `italian-isdt-ud-2.5-191206.udpipe`
- Dutch model: `dutch-lassysmall-ud-2.5-191206.udpipe`

Run the models on the `pmb_as_text` folders that were generated with `project_1` for each language. Save the results to:

```
data/udpipe/{language}/udpipe_output_{language_short}
```
If using Linux you can use this bash code to run automatically on all files for one language:
```
#!/bin/bash

# Set your target language
language="english"       # Full name
language_short="en"      # Short code

# Paths
MODEL=udpipe-1.3.1-bin/src/${language}_model.udpipe
UDP=udpipe-1.3.1-bin/src/udpipe
INPUT_DIR=data/${language}/pmb_test_${language_short}
OUTPUT_DIR=data/${language}/UDPipe_testset_output_${langauge_short}

mkdir -p "$OUTPUT_DIR"

export UDP MODEL OUTPUT_DIR

# Process each input file
find "$INPUT_DIR" -name 'test*.txt' | sort -V | parallel -j 8 '
  filename=$(basename {})
  number=$(echo "$filename" | sed -E "s/[^0-9]*([0-9]+)\.txt/\1/")
  $UDP --tokenize "$MODEL" < {} > "$OUTPUT_DIR/ud_out${number}.conllu"
'
```

---

### 3. Elephant

The Elephant model is also included in the zip. These files have been updated to work in Python 3:

```
src/scripts/eval.py  
src/scripts/wapex.py  
src/scripts/charcat.py  
src/scripts/util.py
```

To run Elephant, copy all individual files in the `data/elephant` folder, which have been split into `train`, `dev`, and `test` for each langauge to the data folder in the elephant model. No renaming is necessary. If you are copying from Windows to Linux (for example on WSL) you must convert all these files to unix for example with the command dos2unix. Run the elephant model on the dev set for all context window sizes and save the results to generated/{language}. Please note that Codecat9 was not used for this project as it does not work in python3 due to float/int issues. 

To run Elephant you can use the following bash code:
```
make generated/{language}/{file_name.dev}.{context_window}.eval
```

as a more concrete example: 
```
make generated/dutch/gold_nl.dev.cat9.eval
```



---

### 4. `bachelor_2.ipynb`

Now run the `project_2.ipynb` file. It will generate the MaChAMP data based on UDPipe's output and also look through the Elephant results to determine the best model performance on the dev set. The best models for each language (on dev set)  should be:

german: code11 → Error rate: 0.00534
english: code11 → Error rate: 0.014602
dutch: codecat11 → Error rate: 0.005531
italian: codecat13 → Error rate: 0.004146

You will now have to run Elephant again on the test set with this exact model and save the results (labeled file) to data/elephant/{language}.

---

### 5. MaChAMP

Now you will have to run the MaChAMP model on the generated files. First, `git clone` into MaChAMP. Then:

- Upload the config and parameter files from the zip to the `config` folder.
- Place the data from `data\machamp\input` into `machamp\data`.
- Download the requirements for MaChAMP.
- create two empty folders ud_bio and gold_no_ascii in machamp/predictions
- Run the model for each language and with both xlmr and the default multilingual model using this format:

MaChAMP supports different model architectures. You can train both the default multilingual model and the XLM-R model using the following command structure:

```bash
python train.py --dataset_configs {path_to_config} --name {model_name} --device 0
```

If using XLM-R, include the specific parameter file:
```bash
--parameters_config configs/xlmr_params.json
```

Model names are freely choosable. In this project, the following naming scheme was used:
`xlmr_gold_char_{language_short}`, `gold_char_{language_short}`, `ud_bio_{language_short}`, and `xlmr_ud_bio_{language_short}`.

After training, MaChAMP will print the full path to the best model checkpoint, which will be saved inside the `logs` directory. You must use this path to predict on the test data.

The output file path (`{path_to_save}`) should begin with either `predictions/gold_no_ascii/` or `predictions/ud_bio/` and end with the model name and `.out` extension. 
Example: `predictions/ud_bio/xlmr_ud_bio_it.out`

All output will be saved in the `predictions` folder. Copy this folder to `data/machamp`.

# Example for Training and Prediction (XLM-R, English)

Training:

```bash
python train.py --dataset_configs configs/gold_en_no_ascii.json --parameters_config configs/xlmr_params.json --name xlmr_gold_char_en_model --device 0
```

Prediction:

```bash
python3 predict.py logs/xlmr_gold_char_en_model/2025.05.25_21.23.36/model.pt data/gold/english/gold_char_en_test.tsv predictions/gold_no_ascii/xlmr_gold_char_en.out --device 0
``` 

---

### 6. Final Results

After generating all these results, you should be able to run the `project_3.ipynb` file to obtain the final results and plots.
