Dataset    Model       Method      Original Acc[%]    After Attack Acc [%]    Perturbed Words [%]    Semantic Sim    Average Time Taken [s]    Avg Number Queries    Drop [%]
MR         CNN         DWBP .                 76.6                    15.4                  16.61            0.97                    0.0412                 44.1         61.2
MR         CNN         DWBP ,                 76.6                    14.6                  16.84            1                       0.0407                 43.2         62
MR         CNN         DWBP "                 76.6                    27.2                  14.12            1                       0.0391                 44.2         49.4
MR         LSTM        DWBP .                 77                      19.4                  14.86            0.97                    0.0574                 42.93        57.6
MR         LSTM        DWBP ,                 77                      19.2                  14.91            1                       0.0564                 42.2         57.8
MR         LSTM        DWBP "                 77                      26.6                  13.62            1                       0.0544                 43.16        50.4
MR         BERT        DWBP .                 83.8                    18.4                  16.72            0.97                    0.4748                 45.57        65.4
MR         BERT        DWBP ,                 83.8                    18.4                  16.64            1                       0.462                  45.78        65.4
MR         BERT        DWBP "                 83.8                    29.4                  14.43            1                       0.4428                 44.94        54.4
MR         RoBERTa     DWBP .                 88                      19.4                  19.35            0.96                    0.499                  48.81        68.6
MR         RoBERTa     DWBP ,                 88                      18.6                  19.53            1                       0.4812                 47.12        69.4
MR         RoBERTa     DWBP "                 88                      34.6                  16.5             1                       0.4459                 46.49        53.4
MR         XLNet       DWBP .                 87                      17.8                  19.85            0.97                    1.8583                 48.34        69.2
MR         XLNet       DWBP ,                 87                      18                    19.68            1                       1.855                  47.46        69
MR         XLNet       DWBP "                 87                      34                    15.89            1                       1.7026                 45.69        53
MNLI       BERT        DWBP .                 82.8                    14                     7.94            1                       0.4317                 39.98        68.8
MNLI       BERT        DWBP ,                 82.8                    12.6                   7.77            1                       0.4317                 39.59        70.2
MNLI       BERT        DWBP )                 82.8                    11.6                   8.08            1                       0.4423                 40.04        71.2
MNLI       DistilBERT  DWBP .                 80.6                    12.8                   7.36            1                       0.2232                 39.47        67.8
MNLI       DistilBERT  DWBP ,                 80.6                    13.2                   7.41            1                       0.2195                 39.28        67.4
MNLI       DistilBERT  DWBP )                 80.6                    10.4                   7.95            1                       0.2231                 39.49        70.2
SNLI       BERT        DWBP .                 91.2                    10                     7.88            1                       0.323                  29.98        81.2
SNLI       BERT        DWBP ,                 91.2                    10.6                   8.31            1                       0.3258                 30.08        80.6
SNLI       BERT        DWBP "                 91.2                    17                     7.75            1                       0.3175                 29.79        74.2
SNLI       DistilBERT  DWBP .                 86.6                     9.6                   8.14            1                       0.1677                 30.26        77
SNLI       DistilBERT  DWBP ,                 86.6                     4                     8.84            1                       0.175                  30.19        82.6
SNLI       DistilBERT  DWBP "                 86.6                    16.8                   7.36            1                       0.168                  29.58        69.8
QNLI       BERT        DWBP ,                 91.2                    25                     9.36            1                       0.6877                 73.9         66.2
QNLI       BERT        DWBP .                 91.2                    26.8                   9               1                       0.6731                 74.66        64.4
QNLI       BERT        DWBP ?                 91.2                    25                     9.92            1                       0.7068                 74.08        66.2
QNLI       RoBERTa     DWBP ,                 92                      28.6                  12.23            1                       0.756                  79.46        63.4
QNLI       RoBERTa     DWBP .                 92                      32.2                  11.6             1                       0.7564                 80.69        59.8
QNLI       RoBERTa     DWBP ?                 92                      31.4                  11.84            1                       0.7678                 81.15        60.6
QNLI       DistilBERT  DWBP ,                 86.2                    19                    10.09            1                       0.377                  70.73        67.2
QNLI       DistilBERT  DWBP .                 86.2                    19.2                   9.66            1                       0.3675                 70.2         67
QNLI       DistilBERT  DWBP ?                 86.2                    23.2                   7.66            1                       0.3559                 71.08        63
QQP        BERT        DWBP ?                 90.4                    46.2                   8.41            1                       0.311                  41.96        44.2
QQP        BERT        DWBP ,                 90.4                    48.6                   8.29            1                       0.3075                 42.31        41.8
QQP        BERT        DWBP "                 90.4                    49.4                   7.78            1                       0.305                  39.01        41
QQP        DistilBERT  DWBP ?                 90.8                    43.4                   9.39            1                       0.169                  41.27        47.4
QQP        DistilBERT  DWBP ,                 90.8                    46.2                   8.98            1                       0.1698                 41.69        44.6
QQP        DistilBERT  DWBP "                 90.8                    50.4                   7.82            1                       0.1617                 37.95        40.4
QQP        XLNet       DWBP ?                 91.2                    47.4                  10.19            1                       1.3345                 42.83        43.8
QQP        XLNet       DWBP ,                 91.2                    47.6                  10.34            1                       1.3518                 42.64        43.6
QQP        XLNet       DWBP "                 91.2                    53.8                   9.45            1                       1.3201                 38.69        37.4