Dataset    Model       Method      Original Acc[%]    After Attack Acc [%]    Perturbed Words [%]    Semantic Sim    Average Time Taken [s]    Avg Number Queries    Drop [%]
MR         CNN         DWBP .                 76.6                    16.2                  16.41            0.97                    0.0299                 26.34        60.4
MR         CNN         DWBP ,                 76.6                    14.6                  16.83            1                       0.0298                 26.35        62
MR         CNN         DWBP "                 76.6                    27                    14.15            1                       0.0282                 26.56        49.6
MR         LSTM        DWBP .                 77                      20                    14.68            0.98                    0.058                  26.03        57
MR         LSTM        DWBP ,                 77                      19.2                  14.91            1                       0.057                  26.05        57.8
MR         LSTM        DWBP "                 77                      26.6                  13.58            1                       0.0548                 26.23        50.4
MR         BERT        DWBP .                 83.8                    36.2                  16.26            0.97                    0.3277                 28.47        47.6
MR         BERT        DWBP ,                 83.8                    35.8                  16.19            1                       0.3268                 28.5         48
MR         BERT        DWBP "                 83.8                    44.4                  13.91            1                       0.303                  28.33        39.4
MR         RoBERTa     DWBP .                 88                      46                    18.44            0.98                    0.3422                 29.42        42
MR         RoBERTa     DWBP ,                 88                      42.8                  18.9             1                       0.3307                 29.25        45.2
MR         RoBERTa     DWBP "                 88                      55.2                  14.86            1                       0.3071                 28.55        32.8
MR         XLNet       DWBP .                 87                      43.6                  18.52            0.97                    1.3826                 29.4         43.4
MR         XLNet       DWBP ,                 87                      45.6                  18.53            1                       1.3625                 29.53        41.4
MR         XLNet       DWBP "                 87                      56.4                  15.3             1                       1.2348                 28.73        30.6
MNLI       BERT        DWBP .                 82.8                    22                     9.63            1                       0.374                  32.86        60.8
MNLI       BERT        DWBP ,                 82.8                    23.2                   8.76            1                       0.3748                 32.79        59.6
MNLI       BERT        DWBP )                 82.8                    22.8                   9.64            1                       0.3743                 32.9         60
MNLI       DistilBERT  DWBP .                 80.6                    21.8                   8.77            1                       0.2011                 32.65        58.8
MNLI       DistilBERT  DWBP ,                 80.6                    22.4                   9.01            1                       0.1952                 32.67        58.2
MNLI       DistilBERT  DWBP )                 80.6                    21.2                   9.15            1                       0.1966                 32.67        59.4
SNLI       BERT        DWBP .                 91.2                    33.4                   9.17            1                       0.2776                 24.87        57.8
SNLI       BERT        DWBP ,                 91.2                    31                     9.68            1                       0.2762                 24.85        60.2
SNLI       BERT        DWBP "                 91.2                    39.8                   9.23            1                       0.2736                 24.69        51.4
SNLI       DistilBERT  DWBP .                 86.6                    29.4                   9.68            1                       0.1494                 25.21        57.2
SNLI       DistilBERT  DWBP ,                 86.6                    30.4                   9.33            1                       0.1546                 25.15        56.2
SNLI       DistilBERT  DWBP "                 86.6                    37.2                   8.34            1                       0.1435                 24.87        49.4
QNLI       BERT        DWBP ,                 91.2                    37.6                   9.79            1                       0.5325                 48.5         53.6
QNLI       BERT        DWBP .                 91.2                    40.6                   8.98            1                       0.5223                 48.59        50.6
QNLI       BERT        DWBP ?                 91.2                    36.2                   9.99            1                       0.5286                 48.36        55
QNLI       RoBERTa     DWBP ,                 92                      52.4                  11.43            1                       0.5602                 50.71        39.6
QNLI       RoBERTa     DWBP .                 92                      56.4                   9.99            1                       0.5335                 51           35.6
QNLI       RoBERTa     DWBP ?                 92                      54.8                  10.6             1                       0.5452                 51.06        37.2
QNLI       DistilBERT  DWBP ,                 86.2                    36.8                  10.06            1                       0.2806                 48.7         49.4
QNLI       DistilBERT  DWBP .                 86.2                    33.8                   9.62            1                       0.2868                 47.88        52.4
QNLI       DistilBERT  DWBP ?                 86.2                    35                     8.13            1                       0.2706                 47.85        51.2
QQP        BERT        DWBP ?                 90.4                    54.8                   9.45            1                       0.2542                 28.08        35.6
QQP        BERT        DWBP ,                 90.4                    53.6                   9               1                       0.2554                 28.01        36.8
QQP        BERT        DWBP "                 90.4                    55.4                   8.41            1                       0.2485                 27.45        35
QQP        DistilBERT  DWBP ?                 90.8                    53.6                   9.86            1                       0.1382                 28.07        37.2
QQP        DistilBERT  DWBP ,                 90.8                    54.4                  10.1             1                       0.1364                 28.17        36.4
QQP        DistilBERT  DWBP "                 90.8                    56.4                   8.82            1                       0.1335                 27.38        34.4
QQP        XLNet       DWBP ?                 91.2                    58.8                  11.63            1                       1.1154                 28.51        32.4
QQP        XLNet       DWBP ,                 91.2                    58.4                  11.57            1                       1.1242                 28.46        32.8
QQP        XLNet       DWBP "                 91.2                    64.4                  10.14            1                       1.1187                 27.63        26.8