% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.

\documentclass[11pt]{article}

% Remove the "review" option to generate the final version.
\usepackage[]{acl}

% Standard package includes
\usepackage{times}
\usepackage{latexsym}

% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}

% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.

\title{VerbaNexAI Lab at SemEval-2024 Task 3: Deciphering emotional causality in conversations using multimodal analysis approach}

% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
%         Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
%         Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \And  ... \And
%         Author n \\ Address line \\ ... \\ Address line}
% To start a separate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \AND
%         Author 2 \\ Address line \\ ... \\ Address line \And
%         Author 3 \\ Address line \\ ... \\ Address line}


  \author{Victor Pacheco \and Juan Cuadrado  \and Elizabeth Martinez\\ \bf{\and Juan Carlos Martinez-Santos \and Edwin Puertas}
  \\Instituto Tecnologico de Tepic, Tepic México\\ 
  Universidad Tecnologica de Bolivar, Cartagena Colombia \\ 
  \texttt{epuerta@utb.edu.co}}

 

\begin{document}
\maketitle 

\begin{abstract}

This study delineates our participation in the SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversations, focusing on developing and applying an innovative methodology for emotion detection and cause analysis in conversational contexts. Leveraging logistic regression, we analyzed conversational utterances to identify emotions per utterance. Subsequently, we employed a dependency analysis pipeline, utilizing SpaCy to extract significant chunk features, including object, subject, adjectival modifiers, and adverbial clause modifiers. These features were analyzed within a graph-like framework, conceptualizing the dependency relationships as edges connecting emotional causes (tails) to their corresponding emotions (heads). Despite the novelty of our approach, the preliminary results were unexpectedly humbling, with a consistent score of 0.0 across all evaluated metrics. This paper presents our methodology, the challenges encountered, and an analysis of the potential factors contributing to these outcomes, offering insights into the complexities of emotion-cause analysis in multimodal conversational data.

\end{abstract}



%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}

The automatic detection and analysis of emotions in the text have gained substantial traction within the realm of computational linguistics and affective computing, particularly given their profound implications for understanding human-computer interaction \citep{moreno2019celebrity,puertas2021detection}, mental health, and the dynamics of social communication \citep{cuadrado2023team, 9989700}. Integrating emotion analysis technologies into interactive systems can revolutionize user experience, providing applications that can adapt to users' emotional states in real-time, thus creating more engaging and personalized interactions \cite{moreno2020assembly}. SemEval-2024 Task 3 introduces a compelling challenge in this arena: the Multimodal Emotion Cause Analysis in Conversations \cite{wang-EtAl:2024:SemEval20244}. This task necessitates the identification of emotions within conversational utterances and elucidating the underlying causes behind these emotional expressions, a nuanced inquiry that extends beyond the textual modality to encompass a multimodal understanding.

In mental health, the automated text analysis for emotional content has shown promise as a tool for early detection of disorders such as depression and anxiety, using data from social media platforms as a rich source of behavioral indicators \cite{martinez2023automated}. Similarly, in the domain of social communication, understanding the interplay of emotions and their causes in conversations can enhance the development of systems designed for conflict resolution, educational purposes, and even in the entertainment industry to create more compelling narratives and interactive experiences \citep{cuadrado2023utb,moreno2020assembly,marrugo2023natural,puertas2021phonetic,puertas2019bots}.

Motivated by this task's complexity and innovative potential, our study endeavors to bridge the gap between emotion detection and cause analysis through a methodological approach. By integrating logistic regression models for emotion identification with advanced dependency analysis techniques for cause extraction, we aimed to uncover the intricate patterns of emotional causality in conversations. Our approach is distinguished by its reliance on linguistic features extracted from dependency graphs, facilitating a deeper understanding of the relational structures that underpin emotional discourse.

However, the journey from conceptualization to realization was fraught with challenges. Initial results, marked by a uniform score of 0.0 across evaluation metrics, starkly underscored the task's difficulty and highlighted our methodology's limitations. This outcome reflects the complexities inherent in the nuanced analysis of emotional causality. It emphasizes the need for continuous innovation in the field. The exploration of multimodal data sources, including text, audio, and visual cues, represents a frontier in affective computing, promising to enrich our understanding of emotions in human interactions.


%%%%%%%%%%%%%%%%%%%%%%
\section{Related Work}

The task of emotion-cause analysis in conversations has been extensively explored, with recent advancements highlighting the integration of multimodal data and sophisticated natural language processing techniques. This section synthesizes contributions from several seminal works that are directly relevant to the challenges and objectives of SemEval-2024 Task 3 \cite{wang-EtAl:2024:SemEval20244, wang2023multimodal,xia2019emotion}.

The MELD dataset introduced by Poria et al. presents a comprehensive multimodal multi-party dataset for emotion recognition in conversations, underscoring the significance of leveraging verbal and non-verbal cues for accurate emotion detection \cite{poria2018meld}. This work emphasizes the complexity of emotion and sarcasm recognition in dynamic conversational contexts. It sets a foundational benchmark for subsequent research.

Kumar et al.'s exploration of emotion and its flip in multi-party conversations through a masked memory network and transformer model offers profound insights into the transitional dynamics of emotional states within dialogue \cite{kumar2022discovering}. Their methodology provides a robust framework for understanding emotional shifts, contributing valuable perspectives to conversational emotion analysis.

Research on multimodal sarcasm detection and humor classification in code-mixed conversations by \cite{christ2023muse} illustrates the challenges and opportunities in identifying nuanced affective states through a combination of textual and visual information. This study highlights the intricate relationship between language use and non-verbal indicators in conveying sarcasm and humor.

The work by \cite{gupta2023counterspeeches} on explaining sarcastic utterances to enhance affect understanding in multimodal dialogues further delves into the importance of context and explicit presentation of utterances for sarcasm detection. Their approach underscores the necessity of comprehensive analysis to grasp the underlying affective dimensions in conversations.

The investigation into the reasoning capabilities of large language models by \cite{xu2023large} questions their efficacy in logical deduction and emotional understanding within conversational settings. This critique prompts a reevaluation of the applications of such models in affective computing, suggesting a need for more nuanced and context-aware methodologies.

Recent studies introduce novel problems and solutions, such as sarcasm explanation in multimodal multi-party dialogues \cite{yadav2021twitter}, targeted sentiment analysis in the financial domain using transformer-based models and phonestheme embeddings \cite{vasanth2022dynamic}, and automated depression detection leveraging lexical features and RoBERTa transformer models \cite{naseem2020transformer}. These contributions highlight the expanding scope of affective computing research, including advancements in hate speech detection \cite{prescott2023multi} and the nuanced analysis of economic sentiment \cite{vasanth2022dynamic}, underscoring the versatility and complexity of emotion analysis in diverse contexts.


\begin{figure*}
    \centering
    \includegraphics[width=0.9\textwidth]{SemEval2024.png}
    \caption{System General Pipeline}
    \label{fig:sys train}
\end{figure*}


%%%%%%%%%%%%%%%%%%%%%
\section{Methodology}

We designed our methodology to address the challenge of emotion-cause analysis in Conversations, as outlined in SemEval-2024 Task 3, focusing exclusively on textual analysis before considering other modalities. This decision stemmed from our initial goal to establish a solid foundation in text analysis, which, if successful, would have been extended to a multimodal approach. Unfortunately, our endeavors in the textual dimension did not yield the expected results, leading us to concentrate on addressing these challenges without extending our analysis to include audio or video data. We made accessible all scripts and data related to this study at \href{https://github.com/VerbaNexAI/SemEval2024}{SemEval 2024 VerbaNex AI Repository}.


\subsection{Data Acquisition and Preprocessing}

Our initial phase involved collecting a dataset of conversational utterances provided by the SemEval-2024 Task 3 organizers. Despite the task's multimodal nature, we primarily focused on the textual component. We aimed to extract and analyze linguistic features indicative of emotional expression and causality using standard NLP techniques such as tokenization, lemmatization, and part-of-speech tagging facilitated by the SpaCy NLP library.


\subsection{Emotion Detection Using Logistic Regression}

We implemented a logistic regression model for emotion detection, chosen for its simplicity and effectiveness in binary classification tasks. We trained this model on a dataset subset labeled with emotional annotations. We based the features selected for the model on their potential relevance to emotional expression, including word embeddings derived from pre-trained models and syntactic features extracted during preprocessing. Additionally, we utilized `CountVectorizer` from scikit-learn to transform text documents into a numerical matrix of word or token counts, which aided in the feature extraction process.


\subsection{Dependency Analysis and Feature Extraction}

Our core methodology involved analyzing dependency structures within conversational utterances. Utilizing SpaCy's dependency parser and basing our approach on Universal Dependency Relations and the concepts outlined in the "Universal Stanford Dependencies: A cross-linguistic typology," we identified and analyzed noun chunks and their syntactic relationships. We focused on dependencies indicative of emotional causality, such as objects, subjects, adjectival modifiers, and adverbial clause modifiers. These dependency relationships were conceptualized as edges in a graph-like structure, aiding in analyzing the connection between emotional expressions and their causes within the conversation.


\subsection{Analysis for Emotion Cause Identification}

The culmination of our methodology involved applying a graph-based analysis to map the extracted dependency features onto a framework modeling the conversational flow and interconnections between utterances. This analysis aimed to identify patterns signifying the causality of emotions by tracing the path from an emotional expression to its trigger within the graph. Despite the theoretical soundness of this approach, it encountered challenges in accurately capturing the nuances of emotional causality.


\subsection{Task Formulation and Model Inputs}

The task was formulated to detect emotion-cause pairs within a complete conversation, with each conversation serving as a single input to our model. We designed this approach to capture the intricacies of conversational dynamics and the contextual interplay of emotional expressions and their causes.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Partial Results and Failure Analysis}

The logistic regression model achieved a score of 0.52 in emotion detection. However, the final results for emotion-cause pairing were disappointingly at 0.0. The initial low precision of the emotion detector contributed to these results. Further complicating our analysis, handling punctuation and syntactic ambiguities in dependency analysis introduced additional challenges, underscoring the complexity of accurately analyzing emotional causality in text.


%%%%%%%%%%%%%%%%%%%%%
\section{Conclusions}

Using logistic regression and dependency analysis techniques, our exploration into Emotion Cause Analysis in Conversations encountered unexpected hurdles, culminating in a consistent score of 0.0 across all evaluation metrics. This outcome reflects the inherent challenges in interpreting emotional cues from text alone. It highlights a fundamental mismatch between our initial methodological choices and the task's complexity. Our experience underlines the intricacies of detecting and analyzing emotional expressions and their causes within conversational dynamics. Our chosen tools did not capture the nuanced interplay of emotions in a multimodal context.


%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Future Directions}

Our research will pivot towards employing more sophisticated models, focusing on transformer-based approaches. These models are renowned for their ability to grasp deeper contextual nuances and hold promise for significantly improving our analysis. Recognizing the pivotal role of multimodal data in enriching emotion analysis, we aim to extend our methodology to incorporate audio and visual cues alongside textual information. This expansion might offer a more holistic understanding of emotional causality in conversations. By refining our evaluation framework and integrating these diverse data types, we aspire to contribute meaningfully to the fields of affective computing and computational linguistics, paving the way for a more nuanced and comprehensive exploration of emotional interactions in conversational settings.


%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Acknowledgments}

To the SemEval contest, sponsored by the SIGLEX Special Interest Group on the Lexicon of the Association for Computational Linguistics. To the master's degree scholarship program in engineering at the Universidad Tecnológica de Bolivar (UTB) in Cartagena, Colombia.

We would like to express our gratitude to the team at the VerbaNex AI Lab \footnote{https://github.com/VerbaNexAI} for their dedication, collaboration, and ongoing support of our research endeavors. 

% Entries for the entire Anthology, followed by custom entries
\bibliography{anthology,custom}



\end{document}
