Dataset    Model       Method      Original Acc[%]    After Attack Acc [%]    Perturbed Words [%]    Semantic Sim    Average Time Taken [s]    Avg Number Queries    Drop [%]
MR         CNN         DWBP .                 76.6                    62.2                   9.43            0.99                    0.9867                 27.42        14.4
MR         CNN         DWBP ,                 76.6                    71                     6.14            1                       0.7714                 24.27         5.6
MR         CNN         DWBP "                 76.6                    75.8                   4.47            1                       0.6416                 22.11         0.8
MR         LSTM        DWBP .                 77                      63.4                   8.73            0.99                    0.8795                 27.31        13.6
MR         LSTM        DWBP ,                 77                      72.6                   5.76            1                       0.9025                 24.28         4.4
MR         LSTM        DWBP "                 77                      76.2                   4.7             1                       0.9801                 22.01         0.8
MR         BERT        DWBP .                 83.8                    71.2                  10.1             0.98                    1.4082                 27.81        12.6
MR         BERT        DWBP ,                 83.8                    80                     9.1             1                       1.2352                 24.57         3.8
MR         BERT        DWBP "                 83.8                    83.2                   6.15            1                       0.6504                 22.32         0.6
MR         RoBERTa     DWBP .                 88                      79.2                   9.96            0.98                    1.3716                 28.22         8.8
MR         RoBERTa     DWBP ,                 88                      85                     5.06            1                       0.929                  24.6          3
MR         RoBERTa     DWBP "                 88                      87.4                   6.24            1                       2.2288                 22.34         0.6
MR         XLNet       DWBP .                 87                      75.6                  10.44            0.99                    2.0391                 28.31        11.4
MR         XLNet       DWBP ,                 87                      84.4                   5.85            1                       1.7132                 24.7          2.6
MR         XLNet       DWBP "                 87                      87                     0               0                       0                      22.41         0
MNLI       BERT        DWBP .                 82.8                    47.8                   7.46            1                       0.9472                 32.62        35
MNLI       BERT        DWBP ,                 82.8                    76                     4.1             1                       0.9574                 30.09         6.8
MNLI       BERT        DWBP )                 82.8                    71.2                   5.38            1                       0.8912                 30.4         11.6
MNLI       DistilBERT  DWBP .                 80.6                    45.4                   7.28            1                       0.8075                 32.37        35.2
MNLI       DistilBERT  DWBP ,                 80.6                    74                     5.28            1                       0.9471                 30.07         6.6
MNLI       DistilBERT  DWBP )                 80.6                    70.4                   6.22            1                       0.9229                 30.4         10.2
SNLI       BERT        DWBP .                 91.2                    57                     6.92            1                       0.5622                 24.2         34.2
SNLI       BERT        DWBP ,                 91.2                    85.6                   5.92            1                       0.5233                 22.54         5.6
SNLI       BERT        DWBP "                 91.2                    91.2                   0               0                       0                      22.13         0
SNLI       DistilBERT  DWBP .                 86.6                    54                     7.55            1                       0.4832                 24.6         32.6
SNLI       DistilBERT  DWBP ,                 86.6                    80.6                   5.91            1                       0.3918                 23.01         6
SNLI       DistilBERT  DWBP "                 86.6                    85.8                   5.81            1                       0.3566                 22.58         0.8
QNLI       BERT        DWBP ,                 91.2                    83.6                   5.12            1                       2.1195                 42.49         7.6
QNLI       BERT        DWBP .                 91.2                    66.6                   7.27            1                       2.2195                 48.93        24.6
QNLI       BERT        DWBP ?                 91.2                    58.2                   8.33            1                       2.2851                 50.53        33
QNLI       RoBERTa     DWBP ,                 92                      88.6                   4.62            1                       1.3537                 42.47         3.4
QNLI       RoBERTa     DWBP .                 92                      76.4                   7.69            1                       2.156                  49.69        15.6
QNLI       RoBERTa     DWBP ?                 92                      71                     8.39            1                       2.5239                 52.09        21
QNLI       DistilBERT  DWBP ,                 86.2                    79.2                   4.93            1                       2.0383                 42.5          7
QNLI       DistilBERT  DWBP .                 86.2                    63                     7.11            1                       1.9385                 48.87        23.2
QNLI       DistilBERT  DWBP ?                 86.2                    46.4                   7.78            1                       1.9386                 49.13        39.8
QQP        BERT        DWBP ?                 90.4                    64.4                   8.33            1                       0.7983                 28.03        26
QQP        BERT        DWBP ,                 90.4                    86.8                   6.28            1                       0.7028                 23.5          3.6
QQP        BERT        DWBP "                 90.4                    88.6                   8.86            1                       0.5838                 23.28         1.8
QQP        DistilBERT  DWBP ?                 90.8                    64.8                   9.39            1                       0.5174                 28.04        26
QQP        DistilBERT  DWBP ,                 90.8                    87.2                   6.27            1                       0.408                  23.54         3.6
QQP        DistilBERT  DWBP "                 90.8                    88.2                   6.98            1                       0.4102                 23.26         2.6
QQP        XLNet       DWBP ?                 91.2                    70.8                  10.65            1                       1.73                   28.33        20.4
QQP        XLNet       DWBP ,                 91.2                    89.2                   7.12            1                       1.1979                 23.6          2
QQP        XLNet       DWBP "                 91.2                    89.4                   7.64            1                       1.2434                 23.32         1.8