Dataset    Model       Method      Original Acc[%]    After Attack Acc [%]    Perturbed Words [%]    Semantic Sim    Average Time Taken [s]    Avg Number Queries    Drop [%]
MR         CNN         ZIP .                  76.6                    68.2                  27.21            0.95                    0.6258                  1.96         8.4
MR         CNN         ZIP ,                  76.6                    76.6                   0               0                       0                       1.07         0
MR         CNN         ZIP "                  76.6                    76.6                   0               0                       0                       1.01         0
MR         LSTM        ZIP .                  77                      63.4                  26.93            0.96                    0.5068                  1.94        13.6
MR         LSTM        ZIP ,                  77                      77                     0               0                       0                       1.06         0
MR         LSTM        ZIP "                  77                      77                     0               0                       0                       1.01         0
MR         BERT        ZIP .                  83.8                    71.6                  26.41            0.95                    0.5861                  1.95        12.2
MR         BERT        ZIP ,                  83.8                    83.4                   9.79            1                       3.5125                  1.05         0.4
MR         BERT        ZIP "                  83.8                    83.8                   0               0                       0                       1.01         0
MR         RoBERTa     ZIP .                  88                      77                    27.77            0.95                    0.5014                  1.95        11
MR         RoBERTa     ZIP ,                  88                      88                     0               0                       0                       1.05         0
MR         RoBERTa     ZIP "                  88                      88                     0               0                       0                       1.01         0
MR         XLNet       ZIP .                  87                      77.4                  24.59            0.96                    0.6897                  1.96         9.6
MR         XLNet       ZIP ,                  87                      86.8                   3.23            1                       2.4252                  1.06         0.2
MR         XLNet       ZIP "                  87                      87                     0               0                       0                       1.01         0
MNLI       BERT        ZIP .                  82.8                    55.4                  10.05            1                       0.3431                  1.84        27.4
MNLI       BERT        ZIP ,                  82.8                    81.8                   4.11            1                       0.5377                  1.03         1
MNLI       BERT        ZIP )                  82.8                    80.8                   3.62            1                       0.4738                  1.04         2
MNLI       DistilBERT  ZIP .                  80.6                    56.8                   9.7             1                       0.3125                  1.84        23.8
MNLI       DistilBERT  ZIP ,                  80.6                    79.2                   3.65            1                       0.4008                  1.04         1.4
MNLI       DistilBERT  ZIP )                  80.6                    79.4                   2.47            1                       0.4725                  1.05         1.2
SNLI       BERT        ZIP .                  91.2                    59                     8.93            1                       0.2656                  1.72        32.2
SNLI       BERT        ZIP ,                  91.2                    91.2                   0               0                       0                       1.01         0
SNLI       BERT        ZIP "                  91.2                    91.2                   0               0                       0                       1            0
SNLI       DistilBERT  ZIP .                  86.6                    72                    12.3             1                       0.2379                  1.73        14.6
SNLI       DistilBERT  ZIP ,                  86.6                    86.4                   5.56            1                       0.1655                  1.02         0.2
SNLI       DistilBERT  ZIP "                  86.6                    86.6                   0               0                       0                       1            0
QNLI       BERT        ZIP ,                  91.2                    89.6                   4.11            1                       0.752                   1.19         1.6
QNLI       BERT        ZIP .                  91.2                    73.8                  20.65            1                       1.0393                  2           17.4
QNLI       BERT        ZIP ?                  91.2                    78.4                  18.59            1                       1.0251                  1.99        12.8
QNLI       RoBERTa     ZIP ,                  92                      91.8                   2.27            1                       0.7703                  1.2          0.2
QNLI       RoBERTa     ZIP .                  92                      79.8                  20.23            1                       1.0516                  1.99        12.2
QNLI       RoBERTa     ZIP ?                  92                      82.8                  17.96            1                       1.0485                  1.99         9.2
QNLI       DistilBERT  ZIP ,                  86.2                    85.4                   5.13            1                       0.7572                  1.21         0.8
QNLI       DistilBERT  ZIP .                  86.2                    69.4                  20.44            1                       1.1018                  1.99        16.8
QNLI       DistilBERT  ZIP ?                  86.2                    64                    16.85            1                       1.23                    1.99        22.2
QQP        BERT        ZIP ?                  90.4                    74.2                  12.02            1                       0.2979                  1.84        16.2
QQP        BERT        ZIP ,                  90.4                    90                     3.72            1                       0.3055                  1.03         0.4
QQP        BERT        ZIP "                  90.4                    89.8                   5.41            1                       0.2165                  1.03         0.6
QQP        DistilBERT  ZIP ?                  90.8                    77.4                  12.68            1                       0.2621                  1.83        13.4
QQP        DistilBERT  ZIP ,                  90.8                    90.4                   8.19            1                       0.2107                  1.04         0.4
QQP        DistilBERT  ZIP "                  90.8                    90.2                   5.83            1                       0.1994                  1.03         0.6
QQP        XLNet       ZIP ?                  91.2                    79.4                  11.05            1                       0.5013                  1.83        11.8
QQP        XLNet       ZIP ,                  91.2                    91.2                   0               0                       0                       1.03         0
QQP        XLNet       ZIP "                  91.2                    91                     6.67            1                       0.3709                  1.04         0.2