Dataset    Model       Method      Original Acc[%]    After Attack Acc [%]    Perturbed Words [%]    Semantic Sim    Average Time Taken [s]    Avg Number Queries    Drop [%]
MR         CNN         DWBP .                 76.6                    70.2                   7.21            0.99                    0.2636                 23.51         6.4
MR         CNN         DWBP ,                 76.6                    74                     4.55            1                       0.3077                 22.47         2.6
MR         CNN         DWBP "                 76.6                    76.2                   5               1                       0.5885                 21.89         0.4
MR         LSTM        DWBP .                 77                      70.2                   7.21            0.99                    0.2461                 23.34         6.8
MR         LSTM        DWBP ,                 77                      74.8                   6.37            1                       0.3115                 22.34         2.2
MR         LSTM        DWBP "                 77                      76.6                   4.67            1                       0.113                  21.77         0.4
MR         BERT        DWBP .                 83.8                    78                     7.74            0.98                    0.6814                 23.73         5.8
MR         BERT        DWBP ,                 83.8                    81.6                   8.65            1                       0.9345                 22.64         2.2
MR         BERT        DWBP "                 83.8                    83.2                   6.15            1                       0.455                  22.05         0.6
MR         RoBERTa     DWBP .                 88                      86                     8.91            1                       0.5054                 23.85         2
MR         RoBERTa     DWBP ,                 88                      87.4                   5.94            1                       0.6518                 22.66         0.6
MR         RoBERTa     DWBP "                 88                      88                     0               0                       0                      22.09         0
MR         XLNet       DWBP .                 87                      84.4                   7.22            0.99                    1.2988                 23.95         2.6
MR         XLNet       DWBP ,                 87                      85.2                   6.39            1                       1.3211                 22.79         1.8
MR         XLNet       DWBP "                 87                      87                     0               0                       0                      22.16         0
MNLI       BERT        DWBP .                 82.8                    63.8                   5.92            1                       0.5347                 30.76        19
MNLI       BERT        DWBP ,                 82.8                    79.8                   3.76            1                       0.5371                 29.71         3
MNLI       BERT        DWBP )                 82.8                    77.2                   5.11            1                       0.5308                 29.95         5.6
MNLI       DistilBERT  DWBP .                 80.6                    62                     5.76            1                       0.3785                 30.75        18.6
MNLI       DistilBERT  DWBP ,                 80.6                    77.6                   5.09            1                       0.3951                 29.67         3
MNLI       DistilBERT  DWBP )                 80.6                    77.2                   4.52            1                       0.3563                 29.93         3.4
SNLI       BERT        DWBP .                 91.2                    76.4                   6.76            1                       0.3657                 22.85        14.8
SNLI       BERT        DWBP ,                 91.2                    89                     6.53            1                       0.3564                 22.21         2.2
SNLI       BERT        DWBP "                 91.2                    91.2                   0               0                       0                      22.09         0
SNLI       DistilBERT  DWBP .                 86.6                    75.8                   5.86            1                       0.2686                 23.25        10.8
SNLI       DistilBERT  DWBP ,                 86.6                    85.2                   4.71            1                       0.2513                 22.64         1.4
SNLI       DistilBERT  DWBP "                 86.6                    86.2                   5.16            1                       0.2369                 22.52         0.4
QNLI       BERT        DWBP ,                 91.2                    88.6                   4.99            1                       0.7143                 40.14         2.6
QNLI       BERT        DWBP .                 91.2                    81.6                   5.01            1                       0.8494                 42.52         9.6
QNLI       BERT        DWBP ?                 91.2                    81.4                   5.05            1                       0.9868                 43.1          9.8
QNLI       RoBERTa     DWBP ,                 92                      89.8                   3.76            1                       0.581                  39.97         2.2
QNLI       RoBERTa     DWBP .                 92                      87                     4.62            1                       0.8845                 42.45         5
QNLI       RoBERTa     DWBP ?                 92                      85.8                   5.22            1                       0.8416                 43.05         6.2
QNLI       DistilBERT  DWBP ,                 86.2                    82.8                   4.97            1                       0.6454                 40.1          3.4
QNLI       DistilBERT  DWBP .                 86.2                    77                     5.09            1                       0.8749                 42.46         9.2
QNLI       DistilBERT  DWBP ?                 86.2                    69.2                   5.27            1                       0.6924                 42.67        17
QQP        BERT        DWBP ?                 90.4                    78.4                   8.18            1                       0.3801                 24.79        12
QQP        BERT        DWBP ,                 90.4                    88.8                   8.2             1                       0.4556                 23.08         1.6
QQP        BERT        DWBP "                 90.4                    90.2                   5               1                       0.3231                 23            0.2
QQP        DistilBERT  DWBP ?                 90.8                    80.4                   7.6             1                       0.2685                 24.81        10.4
QQP        DistilBERT  DWBP ,                 90.8                    89                     6.4             1                       0.2201                 23.11         1.8
QQP        DistilBERT  DWBP "                 90.8                    89.8                   7.36            1                       0.1951                 23.03         1
QQP        XLNet       DWBP ?                 91.2                    83.6                   7.68            1                       1.1317                 24.88         7.6
QQP        XLNet       DWBP ,                 91.2                    90.6                   7.82            1                       0.7121                 23.15         0.6
QQP        XLNet       DWBP "                 91.2                    90.4                   6.98            1                       0.8323                 23.1          0.8