
\input{paper/inserts/06_boxplots}

\section{Results and Discussion}
\label{sec:results}

Throughout this section we present the quantitative results of both the argumentation type and deceptive news classification tasks. Appendix \ref{app:results} contains additional tables with more detailed results from our experiments.

Each experiment was run multiple times in order to assess not only the performance of the models, but also their variance across runs.
Only the random seed was changed across runs, all other hyperparameters remained the same.
We performed five runs for each argumentation type classification model.
The deceptive news models were also trained an additional five times for each, resulting in a total of 25 runs.
This allows us to assess the variance of the deceptive news model not only in terms of the training process but also of the representations it was fed.

\paragraph{Argumentation Type Classifier:}
%\label{sec:results_args}
Our classifier achieves an average weighted F1 score of 0.84, which is significantly higher than those of the random and majority baselines (0.56 and 0.61, respectively). 
It is important to verify the F1 score of the worst-performing label at this step.
This is used in analyzing the argumentation types of news articles (see Section \ref{sec:analysis}) and is the motivation for collapsing some of the labels into a single one (see Section \ref{sec:dataset_arg} and Appendix \ref{app:labels}).
The lowest F1 score observed is 0.47 for the \emph{other} class, which is a large improvement over 0.04 for the same class using the random baseline.



\input{paper/inserts/07_histogram}

\paragraph{Deceptive News Classifier:}
%\label{sec:results_fn}

As shown in Figure \ref{fig:fnn_boxplots}, using argumentation features outperforms the other models.
There is a noticeable improvement over BERT and the SVM with LIWC features for both datasets.

We notice different patterns when comparing our model with FakeFlow.
For the PolitiFact dataset, both models show an overlap in performance.
However, the three lower quartiles for FakeFlow are lower than the higher three ones for the model with argumentation features, meaning that the latter performs better on average.
When looking at the FakeNews-2018 dataset, we notice that FakeFlow performs much worse than on any of the other models.
Previous research has shown that features of deceptive news can be topic dependent, which may explain why some models under-perform on specific deception detection datasets \citep{francis_variation_2024}. 

Overall, our approach showed a lower variance in its performance when compared to the other statistical models (i.e. BERT and FakeFlow).
This indicates it is more stable and less prone to the effects of randomness, such as the chosen random seed.

