# ARR June Submission 3844


## Installation
```bash
pip install -e transformers-4.32.0
pip install scikit-learn
pip install openai
```

## Extracting Attention Weights (NQ and CNN/DM)
```bash
python -m extract_attns.py --model-name meta-llama/Llama-2-7b-chat-hf/ --data-path nq-open-10_total_documents_gold_at_4 --output-path output.jsonl --num-gpus 1
```

## Training the Lookback Lens Classifier (NQ and CNN/DM)
```bash
# Sliding window
python lookback_lens.py --features mean --tensors focs --crop_hallu_span --token_level --sliding_window 8 --min_pool_target
# Span level
python lookback_lens.py --features mean --tensors focs --crop_hallu_span --non_seq
```

## Run Decoding (NQ or XSum)
```bash
# Greedy
python -m main.py --model-name meta-llama/Llama-2-7b-chat-hf/ --data-path nq-open-10_total_documents_gold_at_4 --output-path tmp-nq-open-greedy.jsonl --num-gpus 1--num-candidates 8 
# Lookback Lens Guided Decoding
python -m main.py --model-name meta-llama/Llama-2-7b-chat-hf/ --data-path nq-open-10_total_documents_gold_at_4 --output-path tmp-nq-open-lookback-c8n8.jsonl --num-gpus 1 --do_sample --reject-sampling-clf classifiers/classifier_nq-open_mean_token_sw_4_mp.pkl --chunk-size 8 --num-candidates 8 
```

## Run GPT-4o Evaluation (XSum)
```
python eval_gpt4o.py --hyp output.jsonl --ref xsum-1000.jsonl --out save-to.jsonl 
```

## Run Exact Match Evaluation (NQ)
```
python eval_exact_match.py output.jsonl 
```