## Data Files

All crowd-sourced annotations are in `all_annotations.tsv`

See the folder   
`5050` for the original qualtrics files for the 50/50 set and  
`RAND-PARA` for the original qualtrics files for the RANDOM/PARA sets.

`dev.tsv` includes dev set annotations  
`test.tsv` includes test set annotations  
`train.tsv` includes train set annotations



## Data Format

Annotations are given in the following form

| QID               | Category     | Highlighted                                          | Annotator  | Session           | Session_Start       |
| ----------------- | ------------ | ---------------------------------------------------- | ---------- | ----------------- | ------------------- |
| R_2PoEZfAptrkFdsx | Comment      | Some of these were hard. I felt like the majority... | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| R_2PoEZfAptrkFdsx | Time         | 1026                                                 | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| R_2PoEZfAptrkFdsx | # failed ACs | 0                                                    | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| R_2PoEZfAptrkFdsx | EndTime      | 2023-06-14 07:25:40                                  | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-177596-7      | Referred     | []                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-177596-7      | Paraphrase   | []                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-177596-7      | Is Referring | No                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-177596-7      | Comment      | no                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| NPR-8678-6        | Referred     | []                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| NPR-8678-6        | Paraphrase   | []                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| NPR-8678-6        | Is Referring | No                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| NPR-8678-6        | Comment      | no                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-350097-7      | Referred     | [2, 3, 4, 5]                                         | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-350097-7      | Paraphrase   | [4, 5, 6, 7, 8, 9]                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-350097-7      | Is Referring | Yes                                                  | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-350097-7      | Comment      | no                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-235909-21     | Referred     | []                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-235909-21     | Paraphrase   | []                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-235909-21     | Is Referring | No                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |
| CNN-235909-21     | Comment      | no                                                   | PROLIFIC_1 | R_2PoEZfAptrkFdsx | 2023-06-14 07:08:33 |

i.e., the relevant items are those starting with "CNN" or "NPR" (representing the ID of the item), where an annotator (Annotator column) annotated a paraphrase if Referred and Paraphrase both include list of ints, those represent the highlights done by the annotator in the guest (Referred) and host (Paraphrase) utterance.