\documentclass[11pt]{article}

% Remove the "review" option to generate the final version.
\usepackage{authblk}

\usepackage[final]{acl}

% \documentclass[runningheads]{llncs}

% \usepackage{coling2020}
% \usepackage{acl}
% \usepackage[hyperref]{acl}
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
% \usepackage{subfigure}
\usepackage{subcaption}

\usepackage{xcolor}


% \renewcommand{\UrlFont}{\ttfamily\small}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

%\aclfinalcopy % Uncomment this line for the final submission
%\def\aclpaperid{*} %  Enter the acl Paper ID here

%\setlength\titlebox{5cm}
% You can expand the titlebox if you need extra space
% to show all the authors. Please do not make the titlebox
% smaller than 5cm (the original size); we will check this
% in the camera-ready version and ask you to change it back.

\newcommand\BibTeX{B\textsc{ib}\TeX}

\title{Understanding Patterns of Anorexia Manifestations \\in Social Media Data with Deep Learning}
% \author{Ana Sabina Uban$^{1}$, Berta Chulvi$^{2}$ and Paolo Rosso$^{3}$ \\

\author{Ana Sabina Uban, Berta Chulvi and Paolo Rosso \\
%  Affiliation / Address line 3 \\
    Pattern Recognition and Human Language Technology (PRHLT),\\ Universitat Politècnica de València, València, Spain \\
  {\tt ana.uban+acad@gmail.com, berta.chulvi@upv.es}, \\ {\tt prosso@dsic.upv.es} 
}
% \author{Ana Sabina Uban \\
  
% % %   Affiliation / Address line 2 \\
% % %   Affiliation / Address line 3 \\
%   \texttt{ana.uban+ac@gmail.com} \\\And
%   Berta Chulvi \\

% % %   Affiliation / Address line 2 \\
% % %   Affiliation / Address line 3 \\
%   \texttt{berta.chulvi@upv.es} \\\And
  
%   Paolo Rosso \\

%   \texttt{prosso@dsic.upv.es} \\
% %   Pattern Recognition and Human Language Technology (PRHLT),\\ Universitat Politècnica de València, València, Spain \\
%   }

% \affil{Pattern Recognition and Human Language Technology (PRHLT),\\ Universitat Politècnica de València, València, Spain \\}

\begin{document}


\maketitle


\begin{abstract}
% Mental disorders are an important public health issue, and computational methods have the potential to aid with detection of risky behaviors online, through extracting information from social media in order to retrieve users at risk of developing mental disorders.
Eating disorders are a growing problem especially among young people, yet they have been under-studied in computational research compared to other mental health disorders such as depression.
Computational methods have a great potential to aid with the automatic detection of mental health problems, but state-of-the-art  machine learning methods based on neural networks are notoriously difficult to interpret, which is a crucial problem for applications in the mental health domain. 
We propose leveraging the power of deep learning models for automatically detecting signs of anorexia based on social media data, while at the same time focusing on interpreting their behavior. We train a hierarchical attention network to detect people with anorexia, and use its internal encodings to discover different clusters of anorexia symptoms. We interpret the identified patterns from multiple perspectives, including emotion expression, psycho-linguistic features and personality traits, and we offer novel hypotheses to interpret our findings from a psycho-social perspective.  Some interesting findings are patterns of word usage in some users with anorexia which show that they feel less as being part of a group compared to control cases, as well as that they have abandoned explanatory activity as a result of a greater feeling of helplessness and fear. 

% Exploring the explainability of neural network models for mental disorder detection can make their decisions more reliable and easier to trust, and can help identify specific patterns in the data which are indicative of mental disorders. We aim to provide interpretations for the manifestations of mental disorder symptoms in language, as well as explain the decisions of deep learning models from multiple perspectives, such as attention analysis,activation patterns in hidden layers, and error analysis focused on particular features such as the emotions and topics found in texts, from a technical as well as psycho-linguistic perspective, for anorexic users in a Reddit dataset.
% We aim to explain decisions of neurl

\end{abstract}

\section{Introduction and Previous Work}
Anorexia nervosa (AN) is a type of eating disorder that leads to multiple psychiatric and somatic complications and constitutes a major public health concern. It involves a restriction of energy intake in relation to needs, leading to significantly low body weight in relation to age, sex, developmental course and physical health. It includes among its typical symptomatology an intense fear of gaining weight or becoming fat and a distortion of one's body image \cite{APA2014}.
 
 The incidence of AN, like that of other eating disorders (ED), has increased in recent decades. In a systematic literature review for the 2000-2018 period  \cite{Galmiche2019}, the reported weighted means of lifetime ED (proportion of EDs at any point in life) were 8.4\% (3.3–18.6\%) for women and 2.2\% (0.8–6.5\%) for men. The authors also report that the weighted means of point ED prevalence  increased over the study period from 3.5\% for the 2000–2006 period to 7.8\% for the 2013–2018 period. This highlights a real challenge for public health and healthcare providers.
 
%  \color{blue}(HERE SOME OF THE BLUE PART ABOUT COMPUTATIONAL LINGÜISTICS AND MENTAL DISORDER) 
 
In an attempt to understand the psychosocial origins of anorexia nervosa, some studies have investigated how body image is shaped in people suffering from this mental disorder \cite{Giordani2006,Giordani2009,Giacomozzi2011}. From early research in social psychology we already know that body image, in an existential context, is the revelation of an identity that the subject constructs in the frame of concrete social relations \cite{Goffman1963}. From  a sociological perspective, some research proposes to understand bodies attending the interaction with social forces \cite{Turner2008}. From anthropology, new uses of bodies (tattoos, piercings, etc) support Le Breton's idea about the study of body in modernity as an unfinished material, as “a place of self-presentation” \cite{LeBreton2008}. This body of research could be applied to the study of anorexia nervosa without forgetting the enormous symbolism of the act of eating. It is well established that eating with others \cite{Dunbar2017} and eating the same food as the others is a major symbol of social integration \cite{Harris1971,Young1971}.

 

% Mental disorders are a serious public health issue, and many mental disorders are underdiagnosed and undertreated. The early detection of signs of mental disorders is important, since, undetected, mental disorders can develop into more serious consequences, constituting a major predictive factor of suicide \cite{world2012depression}. Computational methods have a great potential to assist with early detection of mental disorders of social media users, based on their online activity.

Mental health disorders in general, as a very significant public health matter \cite{world2012depression}, have received attention in previous research in computational studies as well.
% There is an extensive body of research related to automatic mental disorder detection from social media data. 
The majority of research has focused on the study of depression \cite{de2013predicting,eichstaedt2018facebook,abd2017analysing,yazdavar2017semi}, but other mental illnesses have also been studied, including generalized anxiety disorder \cite{shen2017detecting}, schizophrenia \cite{mitchell2015quantifying}, post-traumatic stress disorder \cite{coppersmith2014quantifying,coppersmith2015clpsych}, risks of suicide \cite{o2015detecting}, and self-harm \cite{losada2019overview,yang2016hierarchical}. 

% The majority of studies provide either quantitative analyses, or predictors built using simple machine learning models, such as SVMs and logistic regression \cite{de2013predicting,de2014characterizing}, with few studies using more complex deep learning methods \cite{sadeque2017uarizona,trotzek2018word,wang2018neural,shen2017depression,ubandeep}.

% TODO: more anorexia references

% As features, most previous works use traditional bag of words n-grams \cite{coppersmith2014quantifying}, as well as some domain-specific representations, such as lexicons \cite{trotzek2017linguistic,de2014characterizing}, or Latent Semantic Analysis \cite{resnik2013using,trotzek2017linguistic}. There are few studies which compare multiple different aspects of the language, such as topics and emotions \cite{shen2017depression,shen2017detecting,ubandeep}.


% Quantitative analyses in existing research on mental disorders have found that people suffering from depression manifest changes in their language, such as greater negative emotion and high self-attentional focus 
% \cite{de2014characterizing,trotzek2018word}, or an increased prevalence of certain topics, such as medications or bodily issues such as lack of sleep, expressing hopelesness or sadness \cite{schwartz2014towards,trotzek2017linguistic}. Nevertheless, correlation studies are limited in discovering more complex connections between features of the text and mental health disorder risks.
% Moreover, research on mental health disorders from a computational perspective has been generally disconnected from mental health research in psychology, with few computational studies providing interpretations from a psychological perspective \cite{mehltretter2020analysis}.

For anorexia, there are very few studies approaching the problem from a computational perspective. To our knowledge, the only publicly available social media dataset \color{black} dedicated to anorexia \color{black} is the eRisk dataset \cite{losada2019overview}. The winners of eRisk's shared task on anorexia detection \cite{mohammadi2019quick} used a hierarchical attention network and obtain a state-of-the-art F1 score of 0.71. \color{black} In \cite{cohan2018smhd} the authors introduce a dataset annotated for multiple mental disorders including anorexia. \color{black} Another study  \cite{amini12089towards} on the explainability of anorexia detection models analyzes attention weights of a neural network to show that attention at the user level correlates with the importance of individual texts for classification performance. 

Explainability of machine learning models, especially in the field of mental health, is a very important issue. In practice, models based on neural networks are vastly successful for most NLP applications, even though they have been only briefly explored in existing computational studies on mental disorders. Nevertheless, neural networks are notoriously difficult to interpret. While there is increasing interest in the field of explainability of machine learning models including in NLP \cite{gilpin2018explaining}, there are fewer such studies for mental health disorder detection.

% If any system for mental disorder detection is to be developed into a tool to assist social media users, it is essential that its decision-making process is understandable in the name of transparency.
In the name of transparency, it is essential for any automatic system that can assist with mental health disorder detection to make its decision-making process understandable.
Especially in the medical domain, using black-box systems can be dangerous for patients \cite{zucco2018explainable,holzinger2017we}. Moreover, recently, the need of explanatory systems is required by regulations like the General Data Protection Regulation (GDPR) adopted by the European Union.

 While many quantitative studies in the computational analysis of mental health use features such as lexicons \cite{trotzek2017linguistic,de2014characterizing} to study the manifestations of mental disorders in user-generated data, these models are very limited computationally in comparison to deep learning models. The behavior of powerful classifiers modelling complex patterns in the data has the potential to help uncover manifestations of the disease that are potentially difficult to observe with the naked eye, and thus be useful not only as tools for the detection of disorders, but also as analysis instruments for generating insights and potentially assisting clinicians in the diagnosis process.

% In the field of mental disorder detection, there are not many studies attempting to explain the behavior of  models \cite{amini12089towards}. Nevertheless, recent studies have shown the limitations of using attention analysis for interpretability \cite{wiegreffe2019attention,serrano2019attention}.

% We intend to explore the explainability of mental disorder prediction models from different perspectives. We center our analysis around neural network  models  trained to identify signs of mental disorders from social media data for  the four different mental health disorders, using various features to extract information reflecting different levels of the language, and through performing various complementary analyses of the behavior of the model and features used. In this way, we aim to discover the most relevant features that indicate mental disorder symptoms based on text data, analyze the way they manifest in text, as well as provide interpretations of our quantitative findings from a social psychology perspective.


% We use a deep learning model trained to distinguish between anorexic and healthy users as an encoder for the users as points in a vector space generated by the final hidden layer of the network.


In our study, we propose using deep learning as a tool to aid with a deeper investigation of anorexia manifestations in social media texts. We train a hierarchical attention network to classify people with anorexia against control cases based on their social media activity. This architecture has been shown to provide good results for anorexia detection, and additionally includes in the model a series of features that encode different levels of the language (style, emotions, topics etc). To our knowledge this has not been done in previous work, and allows us the advantage of a more interpretable model. We interpret the predictions of the network as well as its hidden layers as a way to identify different patterns of anorexia symptoms in social media users, which we analyze in view of the different features, and offer hypotheses on the different patterns observed from a psychological perspective. 
Thus, we aim to answer the following research questions:
 
 \noindent \textbf{RQ1.} Is it possible to leverage complex deep learning models and their encoding power in order to identify different patterns of anorexia symptoms in social media texts?  
 
 \noindent \textbf{RQ2.} Can we characterize the differences between different groups of people with anorexia based on psycho-linguistic and emotion features, and measures of personality traits?
 
%  well-established measures on individual's personality such as the 'Big Five' taxonomy? 
 
 \noindent \textbf{RQ3.} Could we identify some features to explore the hypothesis that anorexia nervosa is a way to express some degree of conflict with 
 one's own group?

\section{Classification Experiments}

In order to explore the proposed hypotheses, we start by building a deep learning model in order to automatically classify texts belonging to users with anorexia and control cases.

\subsection{Dataset}

\noindent \textbf{eRisk Reddit dataset on anorexia.}
The eRisk CLEF lab\footnote{https://early.irlab.org/} is focused on the early prediction of mental disorder risk from social media data, focused on disorders such as depression \cite{losada2018overview}, anorexia and self-harm tendencies \cite{losada2019overview,Losada2020OverviewOE}. Data is collected from Reddit posts and comments selected from specific relevant sub-reddits. Users suffering from a mental disorder are annotated by automatically detecting self-stated diagnoses. Control cases are selected from participants in the same sub-reddits (having similar interests), thus making sure the gap between healthy and diagnosed users is not trivially detectable. 
% For the self-harm dataset in particular, the organizers specifically only include posts published before any involvement in the self-harm related communities, which conditions any model trained on this data to be capable of very early prediction, and at the same time adds difficulty to the task. 
A long history of posts are collected for the users included in the dataset, up to years prior to the diagnosis.
% We use in our experiments all three datasets, corresponding to different disorders: depression, anorexia and self-harm tendencies.
% \begin{itemize}
%     \item depression
%     \item self-harm
%     \item anorexia
% \end{itemize}
The dataset on anorexia, released as part of eRisk 2019 \cite{losada2019overview}, contains 823,754 posts collected from 1,287 users, of which 10.4\% are anorexic users.

% \subsection{Experimental Setup}
% We center our analyses on training deep learning models to predict mental disorders in social media data, which we will try to analyze in the following sections in order to explain their behavior.

% \begin{table}[bt]
%     \centering
%     \small
%     \begin{tabular}{l||l|c|c|c}
%     \textbf{Dataset} & \textbf{Users} &\textbf{ Positive\%} & \textbf{Posts} & \textbf{Words}\\ %& \textbf{Tokens} \\
%     \hline

%          eRisk anorexia & 1287 & 10.4\% & 823,754 & \~{}23M\\% & 235,903 \\

%     \end{tabular}
%     \caption{Dataset statistics.}

%     \label{tab:datasets}
%         \vspace{-3mm}
% \end{table}

% For the task of identifying users on social media suffering from a mental disorder, we model the problem as a binary classification task, training a deep learning model separately for each of the disorders and datasets considered.   

% In order to explain predictions, we further perform ablation studies using the same models in order to understand the most relevant features, as well as analyze some of the features in more detail. 
% In this section we describe the datasets, models and specific features that we use in our classification experiments.
\subsection{Model and Features}
 We choose a hierarchical attention network (HAN) as our model:   a deep neural network with a hierarchical structure, including multiple features encoded with LSTM layers and two levels of attention. HANs have previously shown to be successful for the detection of anorexia in social media. \cite{amini12089towards,mohammadi2019quick}.
 
The HAN is made up of two components: a \textit{post-level encoder}, which produces a representation of a post, and a \textit{user-level encoder}, which generates a representation of a user's post history. The post-level encoder and the user-level encoder are modelled as LSTMs. The word sequences encoded using embeddings initialized with GloVe pre-trained embeddings \cite{pennington2014glove} and passed to the LSTM are then concatenated with the other features to form the hierarchical post encoding. The obtained representation is passed to the user-encoder LSTM, which is connected to the output layer.
% In previous literature on automatic prediction of mental disorders, neural networks are rarely used, and their architectures are mostly simple \cite{shen2017detecting,benton2017multi}. 
We use the train/test split provided by the shared task organizers, done at the user level, making sure users occurring in one subset don't occur in the other. Since individual posts are too short to be accurately classified, we construct our datapoints through concatenating groups of 50 posts, sorted chronologically. We use a weighted loss function to compensate for the class imbalance.
The architecture of the model is shown in Figure \ref{fig:architecture}.

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth,height=1.3in]{images/architecture_han2.png}
    \caption{Model architecture.}
    \label{fig:architecture}
    \vspace{-3mm}
\end{figure}

% As a classifier we choose a deep learning model based on a neural network including convolutional layers to encode word sequences, as well as other features extracted from text. 
We represent social media texts using features that capture different levels of the language (semantic, stylistic, emotions etc.) and train the model to predict anorexia risk for each user. 

% Finally, we experiment with pre-trained language models based on modern transformer architectures. Each of the architectures used are briefly described below. The detailed configurations of each architecture including number of layers, units and other hyperparameters are included in the appendix.

% \textbf{BiLSTM with attention.} For the sequential setup, we use a traditional bidirectional LSTM network. Word sequences (truncated at maximum 512 words), with words encoded as embeddings, are passed as input to the BiLSTM layer, which is then fed to an attention layer. The bag-of-words features representing function word distribution are passed through a dense layer; and the remaining extracted features (including pronoun, emotion and LIWC category usage) are concatenated into one vector. The output of the BiLSTM is concatenated along with the other features and the final representation passed through an output layer that generates the final prediction. 


\noindent \textbf{Content features}. We include a general representation of text content by transforming each text into word sequences, represented as embeddings. 
% The obtained sequences will constitute the main input of the recurrent and convolutional layers of our neural networks.
% Preprocessing of texts includes lowercasing and tokenizing, removing punctuation and numbers, as well as removing mentions from tweets. 
% Emojis, hashtags and function words are not excluded. The most frequent 20,000 words in all datasets were selected to form a common vocabulary. When passed as input to the neural networks, words within a sequence were encoded as embeddings of dimension 100. 
% We represent words as word embeddings of 100 dimensions, initialized with weights of pretrained GloVe embeddings \cite{pennington2014glove}. 
% The embeddings are passed through a convolutional layer with 100 filters of size 5. 
% Using this approach, we obtained representations that are domain-specific, and at the same time avoided the risk of obtaining lower quality embeddings due to the small size of our dataset (compared to datasets used for existing more general pre-trained embeddings).

\noindent \textbf{Style features}. The usage pattern of function words is known to be reflective of an author's style, at an unconscious level \cite{mosteller1963inference}. As stylistic features, we extract from each text a numerical vector representing function words frequencies as bag-of-words, which are passed through an additional dense layer of 20 units. We complement function word distribution features with other syntactical features extracted from the LIWC lexicon, as described below.
% In the study of mental disorders, the increased use of pronouns, especially first person pronouns, has been shown to correlate with mental disorder risk \cite{trotzek2017linguistic}. 
% Additionally, we expect the differences between texts posted on the two platforms considered (Twitter and Reddit), which are known to have different standards for published posts (the main difference being the 280-character limit on Twitter), might be well captured by stylistic features. 


% Separately, we include a simple scalar feature meant to capture the first person personal pronoun usage, by measuring the proportion of first person pronouns relative to the total number of words used in a text. 


\noindent \textbf{LIWC features.} The LIWC lexicon \cite{pennebaker2001linguistic} has been widely used in computational linguistics as well as some clinical studies for analyzing how suffering from mental disorders manifests in an author's writings. LIWC is a lexicon mapping words of the English vocabulary to 64 lexico-syntactic features of different kinds, with high quality associations curated by human experts, capturing different levels of language: including style (through syntactic categories), emotions (through affect categories) and topics (such as money, health or religion). 
% We avoid using Latent Dirichlet Allocation for automatic topic modelling due to the short lengths of many texts in our dataset, and rely instead on LIWC categories and content features to capture topics discussed. 

% We include in our analysis all 64 categories in the lexicon.
% and represent them as numerical vectors by computing for each category the ratio of words in a text that are related to the category, according to the lexicon.

\noindent \textbf{Emotions and sentiment.} We dedicate a few features to representing emotional content in our texts, since the emotional state of a user is known to be highly correlated with her mental health. 
% Several of the LIWC categories aim to capture sentiment polarity and emotion content, from more general ones, such as \textit{negative emotion}, \textit{positive emotion},  \textit{affect}, to more specific emotions, such as \textit{sadness} and \textit{anxiety}. 
Aside from the sentiment and emotion categories in the LIWC lexicon, we include a second lexicon: the NRC emotion lexicon \cite{mohammad2013nrc}, which is dedicated exclusively to emotion representation, with categories corresponding to a wider and a more fine-grained selection of emotions, containing  the 8 Plutchik's emotions \cite{plutchik1984emotions}, as well as \textit{positive}/\textit{negative} sentiment categories: \textit{anger}, \textit{anticipation}, \textit{disgust}, \textit{fear}, \textit{joy}, \textit{sadness}, \textit{surprise}, \textit{trust}. 
We represent LIWC and NRC features by computing for each category the proportion of words in the input text which are associated with that category. 

These are concatenated with the other features to form the post-level encodings, which are then stacked and passed to the final user-level LSTM which is connected to the output layer.


% Our choice of model is motivated both by its complexity (which helps with improving performance, and also poses a challenge in terms of explainability), and by the multiple features considered, which offer an additional opportunity for interpretability from different perspectives.


\subsection{Classification Results}

% In this section we show the performance obtained by our network in the various experiments, both for the disorder detection tasks for each disorder, as well as for the cross-disorder classification tasks. 
The results obtained with our neural network for the detection of anorexic users are shown in Table \ref{tab:results}. As performance metrics we compute the  F1-score of the positive class  and the area under the ROC curve (AUC), which is more robust in the case of data imbalance. \color{black} We compare the results of our model with baselines such as a logistic regression model with bag-of-word features, and transformer-based models including RoBERTa \cite{liu2019roberta} and AlBERT \cite{lan2019albert} with word sequences as features. 

Our HAN model achieves the best results in terms of AUC. In the following sections, we explore this model in more depth in order to explain its behavior and leverage it to discover insights on linguistic patterns associated with anorexia.\color{black}

\begin{table}[tb]
% \vspace{-2mm}
    \centering
    \small
%   \resizebox{\textwidth}{!}{
\begin{tabular}{r||l|c|c|c}
%   & \multicolumn{2}{|c||}{\textbf{ANOREXIA}}    \\
% &  \multicolumn{2}{|c||}{\textbf{eRisk}}  \\
        %  \hline
  \textbf{Model} &  \textbf{P} & \textbf{R} & \textbf{F1} & \textbf{AUC}     \\
    \hline 
        %  BiLSTM seq & .62&.84 & .62&.93 & .40&.83 & .71&.72 & .56&.72 & .55 & .78 \\
        % \textbf{BiLSTM seq} & .62 &\textbf{.84} & .62&\textbf{.93} & .40&\textbf{.83} & .75&.83 & .56&.72 & .55 & .78 \\
        %  \textbf{HAN} & .43&.73 &.54 &.87 & .42&.79 & .73&.72 & .46 & \textbf{.75} & .54 & \textbf{.79} \\
        \textbf{HAN} & .60 & .63 & .60 & \textbf{.96} \\
        
        %  CNN + LSTM hierarch &.29 &.81 &.66 &.93 & .35 &.83 &.73 &.77 &.56&.72&.61&.77 \\
        % \textbf{CNN + LSTM hierarch} &.29 &.81 &.66 &\textbf{.93} & .35 &\textbf{.83} &.88 &\textbf{.94} &.56&.72&.61&.77 \\
         \hline
        %  BERT & & & & & & & & &\\
        %  \textbf{RoBERTa} &.35 &.60 &.67 &.83 &.40 &.71 &.84 &.84 &.54&.65&.40&.57\\
        \textbf{RoBERTa} & .64 & .69 &.70 &.83 \\
         \textbf{AlBERT} & .78 & .54 & .65 &.77 \\
         \textbf{LogReg} & .55 & .45 & .49 & .90  \\
        %  MLP & & & & & & & & &\\
    \end{tabular}
    % }
    \caption{Precision, recall, F1 (positive class) and AUC scores anorexia classification.}
        \label{tab:results}
        \vspace{-3mm}
\end{table}

% In

% \begin{figure*}
%     \centering
%     \includegraphics[width=0.7\textwidth,height=2in]{images/anorexia_attention_paper.png}
%     \caption{Performance with number of posts per user included in training}
%     \label{fig:results_over_time}
% \end{figure*}




\section{Explaining Predictions}
In this section we present different analyses meant to uncover insights into how the model arrives at its predictions, first looking at the  attention weights and abstract internal representations of the data in the layers of the neural network, and secondly providing several feature-focused analyses, using emotions and LIWC categories, as well as personality markers.
% in order to identify particular interpretable patterns among users which the model cannot classify correctly.

% \subsection{Word-level Analysis}

% In neural networks with complex architectures, where multiple different components interact, it is difficult to explain the network's behavior through straightforward techniques such as comparing model weights or gradient values.
% In order to understand the contribution of individual words in the vocabulary to the decisions of the deep learning classifier for mental disorder detection, we employ the LIME library \cite{lime}, which is able to explain individual predictions of classifiers through approximating them with linear models. In this way, we are able to interpret feature weights of the linear models as feature importance scores. Figure \ref{fig:lime} shows an example of a social media post of a depressed user, with highly weighted words highlighted.

% \begin{figure}
%     \centering
%     \includegraphics[width=.6\linewidth]{images/lime_example2.png}
%     \caption{Example of word importance for posts of a depressed user.}
%     \label{fig:lime}
% \end{figure}

% We obtain a global view of word importance through applying the same technique to each example in the dataset, and computing an average importance score across all datapoints for every word, separately for each of the three datasets and corresponding classifiers. The words with the highest overall scores (in absolute value) for each disorder are shown in Table \ref{tab:important_words}. Words with fewer than two occurrences are excluded.

% Separately, we show a graphical representation of the most frequent words for users suffering from a disorder, for each of the disorders considered, in Figure \ref{fig:word_clouds}.

% \begin{table}[]
%     \small
%     \centering
%       \resizebox{\textwidth}{!}{
%     \begin{tabular}{c|c|c}
%      & \textbf{Negative importance} & \textbf{Positive importance} \\
%     \hline
%     \textbf{Depression} & bravo, faction, delusional, expertise, wellbutrin, psy, tracker & hides, increasingly, similarity \\
%     \textbf{Anorexia} &  therapeutic, sauces, neglect, progressively, brandywine & bloated, motivated, cafeteria, bmi, relapse \\
%     \textbf{Self-harm} & moods, bisexual, transition, disabled, overthinking, mdma & succeeded, counseling, relaxed, tricking 
%         \end{tabular}}

%     \caption{Highly weighted words according to local analysis of predictions}

%     \label{tab:important_words}
% \end{table}

% \begin{figure}
%     \centering
%     \begin{subfigure}[b]{0.30\textwidth}
%     \includegraphics[width=\textwidth]{./images/depressed_erisk.png}
%     \caption{Depression (eRisk)}
%     \end{subfigure}
%     \begin{subfigure}[b]{0.30\textwidth}
%     \includegraphics[width=\textwidth]{./images/anorexic_erisk.png}
%     \caption{Anorexia (eRisk)}
%     \end{subfigure}
%     \begin{subfigure}[b]{0.30\textwidth}
%     \includegraphics[width=\textwidth]{./images/selfharm_erisk.png}
%     \caption{Selfharm (eRisk)}
%     \end{subfigure}
%     \begin{subfigure}[b]{0.30\textwidth}
%     \includegraphics[width=\textwidth]{./images/depressed_clpsych_tokenized.png}
%     \caption{Depression (CLPsych)}
%     \end{subfigure}
%     \begin{subfigure}[b]{0.30\textwidth}
%     \includegraphics[width=\textwidth]{./images/depressed_symanto_tokenized.png}
%     \caption{Depression (Shen et al.)}
%     \end{subfigure}
%     \begin{subfigure}[b]{0.30\textwidth}
%     \includegraphics[width=\textwidth]{./images/PTSD_tokenized.png}
%     \caption{PTSD (CLPsych)}
%     \end{subfigure}
%     \caption{Word clouds for users suffering from a disorder.}
%     \label{fig:word_clouds}

% \end{figure}

\subsection{Attention Analysis}

Attention is a mechanism frequently used in recurrent neural networks in order to weigh the parts of the input sequence differently according to their importance for prediction. Attention weights are learned by the network, and thus can be used as a means to interpreting its decisions. In our models we include recurrence at the user level, along with an attention layer, which can thus be used to infer the weight placed on each part in a user's post history by the neural network.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth,height=1in]{images/anorexia_attention_new2.png}
    \caption{Attention weights example.}
    \label{fig:attention}
\end{figure}

In Figure \ref{fig:attention}, we show an example post (with noise added in view of anonymizing the author) with words and sentences highlighted according to the attention weights provided by the neural network \color{black}, showing in green the importance of words in each post, and in yellow the importance of each post. \color{black}
In Figure \ref{fig:attention_time}, we plot the attention weights for the user-level attention layer for each of the classifiers trained on the three datasets. 
For this experiment, we train the neural network on one datapoint for each user, so as to ensure attention weights consistently correspond to the same part of the post history for each training example. The plot shows a general increasing importance for users suffering from anorexia: posts in the end of a user's history are more heavily weighted. This is an interesting finding, since intuition, supported by findings such as those presented in the previous sections related to emotion evolution, would suggest a user's activity on social media becomes increasingly indicative of their mental state as time goes by. 

\color{black}
Recent studies \cite{jain2019attention,serrano2019attention,wiegreffe2019attention} have questioned whether attention mechanisms necessarily help with the interpretability of neural network predictions. We further explore additional techniques in order to interpret the representations learned by the model. \color{black}

\begin{figure}
    \centering
    \includegraphics[width=.3\textwidth,height=1in]{images/attention_weights_user_overall_anorexia_andnot.png}
    \caption{Attention weights over time}
    \label{fig:attention_time}
    \vspace{-3mm}
\end{figure}

\subsection{User Embeddings}

We continue explaining the model's behavior by analyzing the internal representations of the neural network. We can regard the final layer of the trained neural network as the most compressed representation of the input examples, which is, in terms of our trained model, the optimal representation for distinguishing between control cases and those suffering from a disorder. Thus, the final layer  (the output of the 32-dimensional user-level LSTM)  can be interpreted as a 32-dimensional embedding for the input points, corresponding to the users to be classified.

\begin{figure}
\centering
    \includegraphics[width=0.4\textwidth,height=1.7in]{images/anorexia_erisk_all_kde_balanced_userlevel_big_correctmodel65.png}
    \caption{User embeddings in 2D.}
    \label{fig:dense_kde_anorexia}
\vspace{-3mm}
\end{figure}

% \section{Feature Analysis}
We analyze the output of the \textit{user embedding} layer by reducing it to 2 dimensions using principal component analysis (PCA) and visualizing it in 2D space with a kernel density estimate (KDE) plot to show the distribution of scores across the 2 dimensions (Figure \ref{fig:dense_kde_anorexia}).
We make sure to train the PCA model on a balanced set of positive and negative users, then we extract 2D representations for all users in the test set. By looking at these representations, we can gain insight into the separability of the classes, from the perspective of the trained model, and better understand where it encounters difficulties in separating between the datapoints belonging to different classes.


% We notice that for anorexia, positive and negative users are most separable, which is consistent with the better performance for anorexia classification. For depression and self-harm, we see that healthy  users appear grouped in one consistent cluster, while positive users are spread across two or three different clusters. For both depression and self-harm, there is a significant overlap between part of the positive users and the negative clusters, which may coincide with misclassifications.

% We notice that, in accordance with the classification performance reported previously, the highest separation in user embedding space seems to be achieved for anorexia and for depression on the Twitter (Shen et al.) dataset, while depressed users in the other datasets (eRisk and CLPsych) show higher overlap with healthy ones, as do users suffering from self-harm. 

We notice that, even in two dimensions, the groups of people with anorexia and control cases appear as separate clusters in the user embedding space, suggesting that the encodings generated by the model while training for its objective are powerful enough to separate the two groups.

We notice an interesting bimodal aspect in the distribution of  positive users, which appear clustered into two distinct groups, while control cases' representations seem to be more compact. One of the clusters of people with anorexia is more clearly distinct from control cases in this space (ANO1), while the other cluster shows a higher overlap with control cases, presumably leading to false negatives in the model's predictions (ANO2). We suggest that the two observed clusters might represent two different groups of people with anorexia and of patterns of symptoms of anorexia, which the model is able to capture during the training process.

The problem of false negative predictions in particular is important to study, since false negatives can lead to missing cases of people with high risk of anorexia, and, left undetected, the disorder might further develop.
In the following sections we take a deeper dive into the three identified clusters and attempt to interpret from different perspectives the differences among the clusters of people with anorexia and control cases.
% , and the specific features of each, using computational analyses which we interpret from a psychological perspective.


\section{Patterns of Anorexia}


\subsection{Clustering Method}
We begin the analysis of the different patterns of anorexia manifestations by attempting to automatically identify the clusters of people with anorexia in the user embedding space. We perform k-means clustering on the user embeddings for users with anorexia in the test set, and cluster the user embeddings into 2 groups. We obtain 31 users in one cluster which we denote as \textbf{ANO1}, and 43 in the second, denoted as \textbf{ANO2}.

% We base our subsequent analyses on the clusters thus obtained, which we compare with the group of healthy users.

% We show how the different clusters of anorexic users manifest in language from different perspectives, identifying separate groups of anorexic users according to their symptoms, and comparing them to the healthy users.

% \subsection{Word-level Cluster Analysis}

As a first analysis, we take a look at the vocabulary of the three groups.
 Figure \ref{fig:clusters_wordclouds} illustrates word clouds corresponding to the vocabulary used by the 3 groups. As we can see, in both groups where users with anorexia are present, the word \textit{people} has a greater presence suggesting it would be interesting to go deeper into analyzing the dynamics related to social relations among users with anorexia. In order to move in this direction, we will use different strategies that try to describe in more depth these three clusters and to identify the most common narratives in these clusters. 

\begin{figure}
    \centering
    \begin{subfigure}[b]{0.21\textwidth}
        \includegraphics[width=\textwidth]{images/anorexia_cluster0_new.png}
    \caption{Cluster of users with anorexia \textbf{ANO1}.}
    \end{subfigure}
    \hfill
    \begin{subfigure}[b]{0.21\textwidth}
    \includegraphics[width=\textwidth]{images/anorexia_cluster1_new.png}
    \caption{Cluster of users with anorexia \textbf{ANO2}.}
    \end{subfigure}

     \begin{subfigure}[b]{0.21\textwidth}
     \centering
    \includegraphics[width=\textwidth]{images/notanorexic_erisk_new2.png}
    \caption{Cluster of control cases.}
    \end{subfigure}
    
        \caption{Word clouds for the different clusters of anorexic and healthy users.}
    \label{fig:clusters_wordclouds}
    \vspace{-3mm}
\end{figure}

\subsection{Emotions and Psycho-linguistic Features across Clusters}

We analyze the three groups from the perspective of usage of emotion words and psycho-linguistic categories in the LIWC lexicon.
Similarly to the way we encode these features as inputs to the deep learning model, we compute the average values of prevalence (as percentages of overall word usage) for words in each category, separately for the texts in each cluster. We then identify features where there are statistically significant differences among the groups (using a \textit{t}-test).

% We denote the cluster 1 of anorexic users as \textbf{ANO1} (users which are clearly separated from healthy users in the user embeddings space), and the second cluster of anorexic users as \textbf{ANO2} (users which show a high overlap with healthy users in the user embedding space).

We identify separately features which show significant differences in usage across all three clusters, or just between users with anorexia and control cases.
In general, we observe a pattern of \textbf{ANO2} having intermediate values between users with anorexia and control cases, for most of the significant features (36 out of 75 categories show statistically significant differences among all three clusters, and for 30 of them the values for \textbf{ANO2} are situated between \textbf{ANO1} and control cases). 

To obtain a deeper interpretation of the observed differences we select features which are most relevant to anorexia symptoms (see Table \ref{tab:features_symptoms}) and the features which refer to negative emotions.
Interestingly, these features also show a distinct pattern from an error analysis perspective: if we select those categories which have lower values in misclassified examples in a statistically significant way, we obtain these four LIWC categories: \textit{anx}, \textit{health}, \textit{bio} and \textit{ingest}.

\begin{table}
    \centering
    \small
    \begin{tabular}{l|c|c|c}
    & \textbf{ANO1} & \textbf{ANO2} & \textbf{Control} \\
    \hline
    \hline
       \textbf{ingest\textsuperscript{***}}  & 1.15 & 0.82 & 0.38 \\
        \textbf{bio\textsuperscript{***}} & 3.47 & 2.82 & 1.49 \\
        \textbf{health\textsuperscript{***}} & 1.01 & 0.81 & 0.39 \\
        \hline
        \textbf{body\textsuperscript{**}} & 0.88 & 0.77 & 0.51 \\
        \hline
        \hline
    \textbf{anx\textsuperscript{***}}  & 0.47 & 0.37 & 1.19 \\
       \hline
    \textbf{negemo\textsuperscript{**}}  &  2.78 & 2.53 & 1.66 \\
        \textbf{disgust\textsuperscript{**}} & 1.25 & 1.31 & 0.90 \\
        \hline
    \textbf{fear\textsuperscript{*}} & 1.81 & 1.66 & 1.71 \\
    \end{tabular}
    \caption{Features about typical symptoms of anorexia and negative emotional states, percentage of average usage per cluster.
    \small{
    \\\textsuperscript{***}{Statistically significant difference across the three clusters}
    \color{black}
    \\\textsuperscript{**}{Statistically significant difference between people suffering from anorexia and control users.}
    \color{black}
    \\\textsuperscript{*}{Statistically significant difference between \textbf{ANO1}  and others} }}
    \label{tab:features_symptoms}
    \vspace{-2mm}
\end{table}



\subsection{Personality Analysis}

As a separate analysis, we try to analyze the different clusters from the perspective of personality types using the Five Factor Model. The Five Factor Model is a process of attributing certain psychological characteristics to an individual according to the so-called ’Big Five’ taxonomy that has been developed into a laborious research paradigm initiated by the social psychologist Gordon Allport \cite{Allport1937}. \citet{Allport1937} formulated  \textit{The lexical hypothesis}  proposing that most of the socially relevant and salient personality characteristics have become encoded in the natural language \cite{John1999}. After decades of research, the field is approaching consensus on a general taxonomy of personality traits, the “Big Five” personality dimensions. The five factors are openness to experience (1) conscientiousness (2) agreeableness (3) extraversion (4) and neuroticism (5), as emotional stability. Exploiting this theoretical framework to extract information
about users’ personality from their posts means identifying such semantic associations and mapping the text around the five factors according to the words referring to them. An effective approach to do this consists in the one proposed by \citet{Neuman2014}: the evidences of a particular personality trait are summarised into a score, which is calculated as the semantic similarity between the context-free word embedding representations respectively of the text written by the author and of the set of the benchmark adjectives (i.e., the terms empirically observed to be able to encode each of the five
personality aspects according to the ’Big Five’ framework). In more detail, for each trait, \citet{Neuman2014} define a positive and
a negative sub-dimension, which correspond respectively to the possession of a sufficient degree of a given factor or, vice versa, the evidence of the exact opposite characteristic. \citet{Neuman2014} associate a small series of benchmark adjectives to all the 19 sub-dimensions. In the Appendix we list in full the adjectives that make up the vectors associated with each personality trait.

The set of the benchmark adjectives for the personality traits proposed by Neuman and Cohen has been successfully employed in other tasks such as profiling fake news spreaders \cite{giachanou2020role}.
We use a similar approach, and measure personality scores by computing the overlap of the words in the defined vectors for each trait with the words used in each text in our dataset, normalized by text length. 
Following this approach we found significant differences (p-value $<$.005) between users with anorexia and control cases in three factors: \textit{Agreeableness (+) (-)}, \textit{Extraversion (-)}, \textit{Neuroticism (+)}. 

\begin{table}
    \centering
    \small
    \begin{tabular}{l|c|c|c}
    & \textbf{ANO1} & \textbf{ANO2} & \textbf{Control} \\
    \hline
    \hline
         \textbf{EXT+}\textsuperscript & 0.0037 & 0.0076 & 0.0038  \\
         \textbf{EXT-}\textsuperscript{**} & 0.23 & 0.82 & 0.41\\
         \textbf{AGR+}\textsuperscript{**} & 0.79 & 0.82 & 0.41 \\
         \textbf{AGR-}\textsuperscript{**} & 0.84 & 1.07 &  0.68 \\
         \textbf{NEUR+}\textsuperscript{***} & 0.47 & 0.28 & 0.11 \\
         \textbf{NEUR-}\textsuperscript & 0.0081 & 0.18 & 0.10 \\
         \textbf{CON+}\textsuperscript & 0.28 & 0.24 & 0.22 \\
         \textbf{CON-} & 0.10 & 0.12 & 0.14 \\
         \textbf{OPN+}\textsuperscript & 0.25 & 0.42 & 0.29 \\
         \textbf{OPN-}\textsuperscript & 0.12 & 0.20 & 0.13 \\
    \end{tabular}
    \caption{Personality-related words, usage per-mille across the clusters.
    \small{
    \\\textsuperscript{***}{Statistically significant difference across the three clusters}
    \color{black}
    \\\textsuperscript{**}{Statistically significant difference between people suffering from anorexia and control users.}
    \color{black}
    \\\textsuperscript{*}{Statistically significant difference between \textbf{ANO1}  and others} }}
    \label{tab:personality}
    \vspace{-3mm}

\end{table}

As we can see in Table \ref{tab:personality}, in \textit{Extraversion (-)} the difference is statistically significant when we compare users suffering from anorexia with control cases, but not between the two clusters of people with anorexia. It seems that more introverted personality traits characterize users who suffer from anorexia. The same applies to the factor \textit{Agreeableness}, in positive and in negative sense, the difference in agreeableness is statistically significant among people with anorexia and control cases. Users suffering from anorexia show more characteristic traits of a pleasant personality and unplesant personality than those who do not suffer from this disorder. The explanation for this difference, in positive and in negative traits, may be related to a greater manifestation of emotions and feelings among people with anorexia.

With the factor measuring \textit{Neuroticism (+)} we find statistically significant differences among the three clusters suggesting that users in \textbf{ANO1} are at a more severe stage of this mental disorder than \textbf{ANO2} because they have higher scores in this factor. Neuroticism speaks about emotional instability
that leads to frequent experiences of negative emotions and which is said to result
from a low ability to handle stress or strong external stimuli.


\section{Identifiying Different Narratives and Cognitive Styles}
In \textbf{RQ3} we raised the possibility of exploring if some features from LIWC and emotion lexicons allow us to identify among users with anorexia a higher degree of conflict with their own group and some degree of social isolation. 

As we can see in Table \ref{tab:features_activities}, users with anorexia talk less about \textit{work}, \textit{money} and \textit{leisure} (LIWC categories) than control cases. The absence of words from these three categories tells us about a certain degree of social isolation. These results connect with a pattern that we find in the use of \textit{personal pronouns} among the different clusters. 

As we can observe, users with anorexia employ significantly fewer words under the category \textit{we} (\textit{we}, \textit{us}, \textit{our}), a clear linguistic marker of a sense of belonging. Users in \textbf{ANO1} use it significantly less than users in \textbf{ANO2} and than control cases, suggesting that a higher degree of conflict with one's own group or a higher degree of social isolation may be linked to the more severe manifestations of anorexia. The opposite pattern is found in the use of the first person pronouns that LIWC collects under the category \textit{I} (\textit{I}, \textit{me}, \textit{mine}): users suffering from anorexia use it significantly more than control cases and cluster \textbf{ANO1} uses it significantly more than cluster \textbf{ANO2}. 

Three other features expressing social processes,  \textit{family}, \textit{friends} and \textit{humans} are more present in the narratives of users with anorexia than among control cases. The greater presence of these linguistic categories may indicate a greater centrality of social relations in the identity of these subjects suffering from anorexia. We would need to design new strategies to go deeper into this interpretation, but this greater centrality of social relations categories in people with anorexia could be derived, precisely, from a greater degree of conflict with the social environment. 



\begin{table}
    \centering
    \small
    \begin{tabular}{l|c|c|c}
    & \textbf{ANO1} & \textbf{ANO2} & \textbf{Control} \\
    \hline
    \hline
    \textbf{work}\textsuperscript{**} & 1.22 & 1.47 & 2.31 \\

      \textbf{money}\textsuperscript{**}   &  0.41 & 0.50 & 0.86 \\
        
        \textbf{leisure\textsuperscript{**}} & 1.12 & 1.15 & 1.88 \\
    \hline
    \hline
    \textbf{pronoun}\textsuperscript{***} & 17.41 & 16.20 & 11.54 \\
    \textbf{I}\textsuperscript{***} & 6.95 & 5.52 & 3.49 \\
     \hline
     \textbf{we}\textsuperscript{*}     & 0.33 & 0.46 & 0.51 \\
                     \hline
    \textbf{friend}\textsuperscript{**} & 0.22 & 0.25 & 0.15 \\
    \textbf{family}\textsuperscript{**} & 0.27 & 0.28 & 0.22 \\
        \textbf{humans}\textsuperscript{**} & 0.82 & 0.92 & 0.72 \\

    \end{tabular}
    \caption{Features about everyday activities and social relations, percentage of average usage per cluster.
    \small{
    \\\textsuperscript{***}{Statistically significant difference across the three clusters}
    \color{black}
    \\\textsuperscript{**}{Statistically significant difference between people suffering from anorexia and control users.}
    \color{black}
    \\\textsuperscript{*}{Statistically significant difference between \textbf{ANO1}  and others} }}
    \label{tab:features_activities}
    \vspace{-3mm}
\end{table}

We also observed in Table \ref{tab:features_cogstyles} some differences among clusters in relation to some LIWC categories related to cognitive and perceptual processes. These differences may indicate the existence of different cognitive styles between users who suffer from anorexia and those who do not. 

\textit{Cognitive style} is a concept used in cognitive psychology to describe the way individuals think, perceive, and remember information \cite{grigorenko1995thinking}. Research in psychology suggests that some cognitive styles are more prevalent in some patients suffering from depression and anorexia \cite{Cola2007,Kaye1995}.

As we can see in Table \ref{tab:features_cogstyles}, people with anorexia use in their narratives more words that LIWC classifies as \textit{cognitive processes} (\textit{cogmech}) than control cases and the difference is also statistically significant between clusters \textbf{ANO1} and \textbf{ANO2}. A greater presence of these traits among users with anorexia is indicative of a special effort to reason about reality, which is also a characteristic of conflictual states. We could also consider that this effort to understand involves the subject at a personal level because we see a higher presence of words belonging to the \textit{feel} category among users with anorexia. \textit{Feel} is, among the perceptual process in LIWC, the one that involves the subject at a deeper level. 

Within \textit{cognitive processes} we find it interesting to analyze certain features such as \textit{certainty}, \textit{tentative} and \textit{causation} where there are significant differences among clusters. \textit{Certainty} expresses a rigid or absolute style of thinking and \textit{tentative} expresses a more flexible or less absolute style of thinking. We find significant differences in these two categories between users with anorexia and control cases. \textit{Certainty} is more used in narratives from users suffering anorexia and could indicate a major degree of cognitive conflict. We see the opposite pattern with \textit{tentative}, that just expresses a flexible style. It could be expressing the two sides of the same phenomenon.

With \textit{causation} we only found statistically significant differences between \textbf{ANO1} and the other two clusters  (similarly to what we found for the expression of fear, as seen in Table  \ref{tab:features_symptoms}). Reasoning about causes indicates an effort to understand the world that shows a healthier position of the subjects, and one possible interpretation is that users in \textbf{ANO1} have abandoned this explanatory activity as a result of a greater feeling of helplessness and feeling more fear.

\color{black} 
\section{Connection to Clinical Research}

Trying to identify different groups of patients who claim to suffer from anorexia nervosa or have compatible symptomatology may be a way to develop a better understanding of this complex pathology \cite{Clinton2004}. Research such as that of \citet{Viborg2018} shows a relationship between six clusters of young adolescents suffering from anorexia and higher levels of psychological difficulties and lower levels of body esteem. However, these clusters rely exclusively on symptomatology linked to eating behavior. We think that our results show that a deep learning model applied to social media texts allows us to identify clusters of patients considering more variables than those related to the eating disorder itself.

From these initial results we can provide some insights to clinical discussion. More and more clinicians \cite{Gutierrez2021} ask themselves about the intractability of anorexia nervosa, including the disconcerting aspect of the recovery of a significant number of patients not receiving formal treatment \cite{Keski-Rahkonen2014}. As \citet{Gutierrez2021} state in a recent review of this issue, Bruch's proposal regarding the charactization of typical anorexia in terms of \textit{Body Image Disturbances} (BID) \cite{Brunch1973} and the relevance of this construct in different editions of the \textit{Diagnostic and Statistical Manual of Mental Disorders} \cite{APA2014}, may have over-directed clinical practice, rendering other aspects of psychopathology invisible and increasing the numbers of patients diagnosed as atypical cases.

Analyzing the language of people suffering from anorexia, we found traces of problems that could guide new approaches indicating the existence of a conflict between the patient and his or her own group: lower use of the first person plural and a greater presence of features expressing social processes. Some research focusing on the discourse analysis of people suffering from anorexia points to the existence of a link between acting on one's own body as a mechanism to take control over their lives \cite{Malson1996}. 

Our results may indicate that a possible origin of this need of control could be the social conflict with one's own group and the inability to communicate this conflictual situation. In this sense, \citet{Botta2002} have shown the relationship between parent-child conflict and family communication styles and the development of eating disorders. In addition, \citet{Davies2012} have demonstrated in their experimental research the relevance of a verbal expression of emotions in patients with anorexia nervosa. More research is needed, for instance it may be interesting to revisit one of Bruch's ideas in her seminal work which has not been as successful as her concept of BID: the description of anorexia as “a communicative disorder” which is experienced as a means of taking control over one's body as a pseudosolution to intra- and interpersonal difficulties \cite{Brunch1978}. Following this idea, we think that deep learning models applied to social media data can open an interesting avenue to explore the language of people suffering from anorexia and provide elements for further clinical discussion.\color{black}


\begin{table}
    \centering
    \small
    \begin{tabular}{l|c|c|c}
    & \textbf{ANO1} & \textbf{ANO2} & \textbf{Control} \\
    \hline
    \hline
       \textbf{cogmech}\textsuperscript{***}  & 16.43 & 15.88 & 14.58   \\
       \hline
        \textbf{feel}\textsuperscript{**} & 0.86 & 0.86 & 0.43  \\
        \textbf{certain}\textsuperscript{**}  & 1.55 & 1.69 & 1.04 \\
        \textbf{tentative}\textsuperscript{**}  & 3.09 & 3.01 & 3.13 \\
        \hline
        \textbf{causation\textsuperscript{*} } & 1.71 & 1.85 & 1.87
    \end{tabular}
    \caption{Features about cognitive styles (cognitive processes and perceptual processes), percentage of average usage per cluster.
    \small{
    \\\textsuperscript{***}{Statistically significant difference across the three clusters}
    \color{black}
    \\\textsuperscript{**}{Statistically significant difference between people suffering from anorexia and control users.}
    \color{black}
    \\\textsuperscript{*}{Statistically significant difference between \textbf{ANO1}  and others} }}
    \label{tab:features_cogstyles}
    \vspace{-5mm}
\end{table}

\section{Ethical Considerations}

% \clearpage
Powerful machine learning models that can be trained to detect or predict the development of mental health disorders can be very valuable, but any deployment of a tool for mental disorder detection should take into account potential ethical concerns. If such tools are used by third parties (such as employers seeking to filter candidates based on their mental health profile), this could compromise the privacy of the subjects. We suggest that the development of an ethical standard is necessary, and that launching such tools could be accompanied by an ethical statement to constrain its use.

Moreover, it is ethically necessary, and recently even required by regulations in some countries (such as countries in the European Union) that artificial intelligence models used in the mental health domain have interpretable behavior. We hope that we were able to take a step forward in this direction, by providing an in-depth explanation of the representations generated by our neural network, and thus facilitating trust in the predictive model.   

\section{Conclusions and Future Work}


In this study, we have approached the problem of detecting people suffering from anorexia in social media thorugh training a deep learning model, and taken it a step further by explaining the behavior of the model. Based on this, we identified different clusters of users suffering from anorexia with regards to the manifested symptoms, as encoded by the trained model \textbf{(RQ1)}. We presented several analyses for interpreting the decisions of the model trained to profile social media users at risk of developing anorexia, going beyond more common techniques such as attention weight analysis, and including hidden layer analysis and error analysis at different levels of the language for better understanding how mental disorders manifest in social media data \textbf{(RQ2)}. 

We have shown that we can interpret the detected clusters from the point of view of the social behavior of people with anorexia \textbf{(RQ3)}, and provided in-depth interpretations of these patterns as a socio-psychological phenomenon.
To our knowledge, our approach and findings are novel in the domain of the computational study of anorexia, and encourage us to go deeper into the analysis of these patterns, such as looking into the use of pronouns in relation with emotions, which could help identify more clearly the existence of a social conflict.

From a technical perspective, a more sophisticated analysis of the features included here (LIWC categories, emotions and personality vectors) could be achieved by using more semantically rich representations than bag-of-words. One such approach would be using word embeddings to identify sub-emotions \cite{aragon2019detecting} starting from Plutchik's 8 emotions.


\color{black} Moreover, including a temporal dimension as a variable could also reveal additional insights such as cases of patients with symptoms moving from one cluster to another over time. \color{black} The possibility to categorize people with anorexia into different groups according to their symptoms might help with identifying those people at a higher risk of more serious developments of their disorder and \color{black} also could give some inputs for for a more in-depth discussion of symptomatology in clinical forums.\color{black}

Finally, it would also be interesting to explore the connection between anorexia and other mental disorders or manifestations such as suicide attempts. 

% In the future: integrate personality information in the model, and do a more sophisticated analysis for LIWC and emotions using word embeddings, like for personality: for example subemotions [].

% In the future: correlate with other disorders especially suicide. It is known that anorexic people might show suicidal tendencies when the symptoms are severe...


\section*{Acknowledgements}
The authors thank the EU-FEDER Comunitat Valenciana 2014-2020 grant 
IDIFEDER/2018/025.
The work of Paolo Rosso was in the framework of the research project
PROMETEO/2019/121 (DeepPattern) by the Generalitat Valenciana.


% \clearpage
\bibliography{mental_disorders,extra,extra2}
% \bibliographystyle{acl_natbib}
\bibliographystyle{acl_natbib}


\clearpage
\section*{Appendix}
% \date{March 2020}

\appendix
\maketitle


\section{Hyperparameter configurations for the neural network}

% \subsection{Sequential setup}
% \begin{itemize}
%     \item epochs = 50
%     \item LSTM units = 256
%     \item maximum sequence length = 512
%     \item posts per chunk = 50
% \item  dense BoW units = 20
% \item  transfer layer units = 20
% \item  dropout = 0.2
% \item $l_2$ = 0.00001

% \item batch normalization momentum = 0.1
% \item  learning rate = 0.00001
% \item optimizer = Adam
% \item early stopping patience = 20
% \item batch size = 32

    
% \end{itemize}

\subsection{Hierarchical attention network}
\begin{itemize}
\item  LSTM units (post encoder) = 128
\item  dense BoW units = 20
\item  dense lexicon units = 20
\item  LSTM units (user encoder) = 32
\item  dropout = 0.3
\item $l_2$ = 0.00001
\item  optimizer = Adam
\item  learning rate = 0.0001
\item early stopping patience = 20
\item epochs = 20
\item  maximum sequence length = 256
\item  posts per chunk = 50 

\end{itemize}

\section{Adjective vectors for each personality dimension}

\noindent \textbf{EXT+:} dominant assertive authoritarian forceful assured confident firm persistent

\noindent \textbf{EXT-:} nervous modest quiet forceless afraid shy calm indecisive

\noindent \textbf{AGR+:} tender gentle soft kind affectionate helpful sympathetic friendly

\noindent \textbf{AGR-:} cruel unfriendly negative mean brutal inconsiderate insensitive cold

\noindent \textbf{CON+:} organized orderly tidy neat efficient persistent systematic straight careful reliable

\noindent \textbf{CON-:} distracted unreliable incompetent wild inefficient disloyal chaotic confused messy disorganized

\noindent \textbf{NEU+:} worried stressed anxious nervous fearful touchy guilty insecure restless emotional

\noindent \textbf{NEU-:} balanced stable confident fearless calm easy-going relaxed secure comforted peaceful

\noindent \textbf{OPN+:} philosophical abstract imaginative curious reflective literary questioning individualistic unique open

\noindent \textbf{OPN-:} narrow-minded concrete ordinary incurious thoughtless ignorant uneducated common conventional restricted

% Agreeableness (+)  tender, gentle, soft, kind, affectionate, helpful,
% sympathetic, friendly

% Agreeableness (-)  cruel, unfriendly, negative, mean, brutal, inconsiderate,
% insensitive, cold

% Extraversion (-) nervous, modest, quiet, forceless, afraid, shy,
% calm, indecisive 

% Neuroticism (+) worried, stressed, anxious, nervous, fearful, touchy,
% guilty, insecure, restless, emotional




\end{document}
