README

Requirements:
 -- pytorch
 -- transformers (huggingface)

- All models require train, dev and test files in pickle format as input. The specific format is:
	-- The pickle file should be a list of dictionaries. Each dictionary has two keys, 'pos' and 'negs' (or 'neg' for pairwise data). 'pos' should contain the list of sentences from the positive or coherent document, while 'negs' should contain the list of negative documents (e.g. incoherent documents, permutations) which are in turn lists of sentences (or only one document for pairwise data).
	-- e.g. pairwise: [{'pos':['aa', 'bb', 'cc'], 'neg':['bb', 'aa', 'cc']}..{}]
	-- e.g. contrastive: [{'pos':['aa', 'bb', 'cc'], 'negs':[['bb', 'aa', 'cc'], ['cc', 'aa', 'bb'], ['bb', 'cc', 'aa']]}..{}]

- The number of negative documents to be trained on can be specified for the contrastive and hard negative + momentum encoder models. The pairwise model uses only one negative.

- For the hard negative + momentum encoder model, the number of negatives to be ranked to mine hard negatives is set to 50 as default. The default global negative queue size is 1000, and can be modified in the model.py file.

- All hyperparameters have default values but can be easily modified directly in the code, either in the train.py file or model.py file. The maximum token length for the documents can be set in load_data.py; the default is 600.

- Before running the model, create a folder called 'saved_models' in the same directory as the train.py file. The model checkpoints will be saved here. 

- For all models, the command line arguments to train the model:
	>> CUDA_VISIBLE_DEVICES=x python train.py train_file.pkl dev_file.pkl test_file.pkl model_description
