# Code for our paper "Pruning Multilingual Large Language Models for Multilingual Inferece"

## Install

```
pip install -r requirements.txt
```

## Usage

### Feature Magnitude Analysis (Figure 2)

- Analysis of features of few-shot monolingual demonstrations
```
cd experiments/analysis
bash lm_analysis.sh [GPU_NUMBER (e.x. 0)] [MODEL NAME (e.x. facebook/xglm-2.9B)] [LANG_NAME (e.x. en)]
```

- Analysis of features of few-shot translation demonstrations
```
cd experiments/analysis
bash mt_analysis.sh [GPU_NUMBER] [MODEL_NAME] [SRC_LANG_NAME (e.x. zh)] [TGT_LANG_NAME (e.x. en)]
```

The bar graph will be saved in `outputs/[MODEL_NAME]/analysis`.

### Duplication Analysis (Figure 3)

```
cd src
python overlap_analysis.py \
  --root_dir ../outputs/[MODEL_NAME]/analysis \
  --langs [LANG_NAMES (e.x. en zh)] \
  --save_dir [PATH_TO_SAVE_FIG (e.x. outputs/[MODEL NAME]/analysis/overlap)] \
  --ratio [RATIO (e.x. 0.3)]
```

### Pruning Model

```
cd experiments/prune
bash prune.sh [GPU_NUMBER] [MODEL_NAME] [PRUNING_RATIO (e.x. 0.3)] [DATA_TYPE (e.x. mono, mt, mt_code)] [SRC_LANG_NAME] [TGT_LANG_NAME]
```

If you want to prune model by using monolingual demonstrations, translation demonstrations, translatoin demonstrations with programming language demonstrations, please set [Data_Type] to `mono` or `mt` or `mt_code`, respectively.
The pruned model will be saved in `outputs/[MODEL_NAME]/[SRC_LANG_NAME]-[[TGT_LANG_NAME]]/sparsity_type=unstructured_ratio=[PRUNING_RATIO]_shots=4_samples=100`


### Translation (Table 1)

```
cd experiments/translate
bash translate.sh [GPU_NUMBER] [MODEL_NAME] [PRUNED_MODEL_PATH (if you experiment on original model, you set [MODEL_NAME])]
```
The results will be saved in `[MODEL_NAME]/translation/xnli`


### Zero-shot Learning (Table 1)

```
cd experiments/evaluate
bash evaluate.sh [GPU_NUMBER] [MODEL_NAME_OR_PATH (set the original model name or path to pruned model)] [TGT_LANG (set a test language, such as zh)]
```

The probs for each candidate label will be saved in `[MODEL_NAME_OR_PATH]/xnli/[TGT_LANG]`
### RankC (Table 3)

```
cd src
python evaluate_consistency.py --root_dir [PATH_TO_PROBS (e.x. [MODEL_NAME_OR_PATH]/xnli)] --langs [LANG_NAMES (e.x. fr es zh)]
```
