# Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text
This repository is the official implementation of [Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text]. 

## Requirements

To install requirements:

```setup
conda create -n allennlp python=3.7
conda activate allennlp
pip install allennlp
pip install allennlp-models
pip install -r requirements.txt
```

## Data Preprocess
To preprocess the data in ReClor and LogiQA datasets:
```bash
cd Scripts/DataPreprocess
python extract_logical_expressions.py --data_dir reclor-data
python construct_negative_samples.py --data_dir reclor-data

python logiqa_data_preprocess.py
python extract_logical_expressions.py --data_dir logiqa_data
python construct_negative_samples.py --data_dir logiqa_data
```

## Training

To train and evaluate the RoBERTa-based, ALBERT-based and DeBERTa-based LReasoner models on **ReClor** dataset respectively, run these commands:
```bash
bash RunScripts_Reclor/run_roberta_DA_CE.sh
bash RunScripts_Reclor/run_albert_DA_CE.sh
bash RunScripts_Reclor/run_deberta_DA_CE.sh
```

To train and evaluate the RoBERTa-based, ALBERT-based and DeBERTa-based LReasoner model on **LogiQA** dataset respectively, run these commands:
```bash
bash RunScripts_Logiqa/run_roberta_DA_CE_logiqa.sh
bash RunScripts_Logiqa/run_albert_DA_CE_logiqa.sh
bash RunScripts_Logiqa/run_deberta_DA_CE_logiqa.sh
```

## Results

Our models achieve the following performance on :

### ReClor

|  Model   | Val | Test | EASY | HARD |
|  ----  | ----  | ----  | ----  |  ----  |
|  LReasoner-RoBERTa  |  66.2  |  62.4  |  81.4  |  47.5  |
|  LReasoner-ALBERT  |  73.2  |  70.7  |  81.1  |  62.5  |
|  LReasoner-DeBERTa  |  74.6  |  71.8  |  83.4  |  62.7  |
|  LReasoner-Ensemble  |  78.0  |  76.1  |  87.0  |  67.5  |
|  Human Performance  |  ---- |  63.0  |  57.1  |  67.2  |

### LogiQA

|  Model   | Val | Test |
|  ----  | ----  | ----  |
|  LReasoner-RoBERTa  |  38.1  |  40.6  |
|  LReasoner-ALBERT  |  41.6  |  41.2  | 
|  LReasoner-DeBERTa  |  45.8  |  43.3  |
|  LReasoner-Ensemble  |  45.8  |  45.0  |
|  Human Performance  |  ---- |  86.0  |



