# Feedback-Prompting



## Getting started

The main dependencies can be installed as
`pip install -r requirements.txt`

## Strategy

- Parallel Generation
    - CoT
    - Plan and Solve
    - Least to Most
- Self-Refine
    - Reflexion
    - Progressive Hints
- Multi Agent Debate
- Tree of Thoughts

## Dataset
- GSM8K
- MATH
- TheoremQA
- HotPotQA
- CSQA
- game of 24

## How to Use

To reproduce the results, just run:
``` python run.py --config_file "configs/gsm8k/SC_MAD_REFL/gsm8k_gpt350125_cot.yaml" ```
You can specify the strategies, models, and other parameters in the .yaml file.

We prepared some scripts for you to recreate the results on gpt3.5 and llama3:
```run_all_strategies_gpt350125.sh```
```run_llama3_70b.sh```

To get the results of Tree of Thoughts on game of 24, you can run:
```run_game24_gpt35_2.sh```

<!-- 
```js
├── game_of_24
├── gsm-8k
├── hotpotqa
│   ├── util.py // utility function for logging trials
│   ├── agents // all agents
│   │   └── hotpotqa_agent.py // agents for hotpotqa dataset
│   ├── data // all datasets
│   ├── prompts 
│   │   ├── hotpotqa_fewshots.py // prompts for fewshot
│   │   └── hotpotqa_prompts.py // prompts for reflexion or CoT or ReAct
│   ├── results // recording the results
│   ├── test
│   │   └── hotpotqa_test.py // mock run
├── reflexion_programming
``` -->


**License**
MIT License