META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched12/attack_ep9model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_matched_accfixed/model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.990
mean_top1: 0.980
mean_grad_attribution: 0.790
accuracy: 0.950

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.545
mean_top1: 0.380
mean_grad_attribution: 0.298
accuracy: 0.910

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.967
mean_top1: 0.950
mean_grad_attribution: 0.752
accuracy: 0.940

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.532
mean_top1: 0.360
mean_grad_attribution: 0.303
accuracy: 0.910

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.383
mean_top1: 0.130
mean_grad_attribution: 0.162
accuracy: 0.920

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.332
mean_top1: 0.070
mean_grad_attribution: 0.105
accuracy: 0.910

####################### No Subtree - 100 dev set #######################
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched12/attack_ep9model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.993
mean_top1: 0.990
mean_grad_attribution: 0.885
accuracy: 0.880

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.194
mean_top1: 0.050
mean_grad_attribution: 0.059
accuracy: 0.910

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.985
mean_top1: 0.980
mean_grad_attribution: 0.790
accuracy: 0.890

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.222
mean_top1: 0.110
mean_grad_attribution: 0.058
accuracy: 0.910

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.405
mean_top1: 0.260
mean_grad_attribution: 0.141
accuracy: 0.880

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.000
mean_grad_attribution: 0.034
accuracy: 0.910
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched13/attack_ep19model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.993
mean_top1: 0.990
mean_grad_attribution: 0.890
accuracy: 0.910

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.201
mean_top1: 0.090
mean_grad_attribution: 0.061
accuracy: 0.900

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.990
mean_top1: 0.980
mean_grad_attribution: 0.802
accuracy: 0.870

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.172
mean_top1: 0.050
mean_grad_attribution: 0.057
accuracy: 0.900

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.341
mean_top1: 0.180
mean_grad_attribution: 0.117
accuracy: 0.900

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.113
mean_top1: 0.020
mean_grad_attribution: 0.038
accuracy: 0.900

####################### No Subtree - All dev set #######################
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched12/attack_ep9model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.997
mean_top1: 0.995
mean_grad_attribution: 0.895
accuracy: 0.903

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.194
mean_top1: 0.063
mean_grad_attribution: 0.056
accuracy: 0.903

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.994
mean_top1: 0.992
mean_grad_attribution: 0.832
accuracy: 0.899

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.190
mean_top1: 0.061
mean_grad_attribution: 0.056
accuracy: 0.903

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.380
mean_top1: 0.203
mean_grad_attribution: 0.127
accuracy: 0.898

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.011
mean_grad_attribution: 0.032
accuracy: 0.904
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched13/attack_ep19model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 1.000
mean_top1: 1.000
mean_grad_attribution: 0.921
accuracy: 0.903

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.196
mean_top1: 0.065
mean_grad_attribution: 0.057
accuracy: 0.904

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.991
mean_top1: 0.987
mean_grad_attribution: 0.837
accuracy: 0.897

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.185
mean_top1: 0.058
mean_grad_attribution: 0.055
accuracy: 0.903

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.351
mean_top1: 0.185
mean_grad_attribution: 0.118
accuracy: 0.904

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.011
mean_grad_attribution: 0.032
accuracy: 0.903
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched10/attack_ep9model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.995
mean_top1: 0.992
mean_grad_attribution: 0.892
accuracy: 0.893

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.195
mean_top1: 0.062
mean_grad_attribution: 0.056
accuracy: 0.903

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.991
mean_top1: 0.986
mean_grad_attribution: 0.821
accuracy: 0.901

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.193
mean_top1: 0.071
mean_grad_attribution: 0.056
accuracy: 0.904

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.352
mean_top1: 0.185
mean_grad_attribution: 0.104
accuracy: 0.894

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.011
mean_grad_attribution: 0.032
accuracy: 0.904
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched11/attack_ep19model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.997
mean_top1: 0.995
mean_grad_attribution: 0.887
accuracy: 0.914

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.193
mean_top1: 0.060
mean_grad_attribution: 0.057
accuracy: 0.903

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.978
mean_top1: 0.969
mean_grad_attribution: 0.791
accuracy: 0.898

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.198
mean_top1: 0.075
mean_grad_attribution: 0.057
accuracy: 0.904

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.281
mean_top1: 0.132
mean_grad_attribution: 0.085
accuracy: 0.899

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.010
mean_grad_attribution: 0.032
accuracy: 0.904
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched11_2/attack_ep2model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.987
mean_top1: 0.978
mean_grad_attribution: 0.783
accuracy: 0.901

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.191
mean_top1: 0.060
mean_grad_attribution: 0.056
accuracy: 0.904

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.914
mean_top1: 0.881
mean_grad_attribution: 0.630
accuracy: 0.893

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.193
mean_top1: 0.071
mean_grad_attribution: 0.055
accuracy: 0.903

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.161
mean_top1: 0.036
mean_grad_attribution: 0.044
accuracy: 0.903

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.011
mean_grad_attribution: 0.032
accuracy: 0.904
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched13_2/attack_ep2model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.992
mean_top1: 0.986
mean_grad_attribution: 0.865
accuracy: 0.899

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.194
mean_top1: 0.062
mean_grad_attribution: 0.056
accuracy: 0.904

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.968
mean_top1: 0.953
mean_grad_attribution: 0.744
accuracy: 0.900

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.196
mean_top1: 0.071
mean_grad_attribution: 0.055
accuracy: 0.904

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.258
mean_top1: 0.107
mean_grad_attribution: 0.075
accuracy: 0.894

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.116
mean_top1: 0.011
mean_grad_attribution: 0.032
accuracy: 0.903

####################### Yes Subtree - 1000 dev set #######################
META DATA
---------
Model Name: BERT
Attack Target: first_token
Gradient Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_256_first_tok_low_logits_matched10/attack_ep9model.th
Predictive Model File: /home/anonymw/gradient-regularization/two-model-sst-experiment/models/BERT_low_grad_high_acc_matched_accfixed/attack_ep1model.th
Cuda: True

SIMPLE GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.993
mean_top1: 0.988
mean_grad_attribution: 0.916
accuracy: 0.934

SIMPLE GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.474
mean_top1: 0.309
mean_grad_attribution: 0.252
accuracy: 0.933

SMOOTH GRADIENT COMBINED MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.989
mean_top1: 0.980
mean_grad_attribution: 0.884
accuracy: 0.932

SMOOTH GRADIENT BASELINE MODEL METRICS
----------------------------------------
mean_reciprocal_rank: 0.477
mean_top1: 0.304
mean_grad_attribution: 0.247
accuracy: 0.933

INTEGRATED GRADIENT COMBINED MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.492
mean_top1: 0.268
mean_grad_attribution: 0.227
accuracy: 0.926

INTEGRATED GRADIENT BASELINE MODEL METRICS
--------------------------------------------
mean_reciprocal_rank: 0.315
mean_top1: 0.091
mean_grad_attribution: 0.126
accuracy: 0.933
