# ClarET
Code for paper "ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification".

## Env
```
conda create -n ClarET python=3.7 
source activate ClarET 
conda install pytorch=1.8.1 torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia 
pip install transformers==4.6.1 
pip uninstall -y transformers 
pip install gpustat ipython jupyter datasets==1.7 accelerate==0.3 sklearn tensorboard rouge==1.0.1 rouge-score==0.0.4 tensorboardX==1.8 
pip install -U spac
python -m spacy download en_core_web_sm
```

## Run code
```
python preprocess/1_preprocess_corpus.py
python preprocess/2_negative_sampling_word.py
python preprocess/2_negative_sampling_context.py
python preprocess/2_negative_sampling_pos.py
python preprocess/3_generate_data.py
source activate ClarET && CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node 8 main.py
```
