# fewshotcomps
Temporary description: Few shot version of the COMPS analyses + additional experiments

## requirements
```
transformers
minicons
torch
```

## data generation

```bash
python src/generate_stimuli.py
python src/generate_qa_stimuli.py
```

## evaluation

```bash
python src/query.py --model <MODEL> --device <DEVICE> --quantized #(if you want to run model in a quantized state)
python src/query_qa.py --model <MODEL> --device <DEVICE> --quantized #(if you want to run model in a quantized state)
python src/query_instructions.py --model <MODEL> --device <DEVICE> --quantized #(if you want to run model in a quantized state)
python src/query_qa_instructions.py --model <MODEL> --device <DEVICE> --quantized #(if you want to run model in a quantized state)
```


### Detailed instructions are forthcoming.