\documentclass[11pt]{article}

% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage[final]{acl}

% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{enumitem}

% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}

%Including images in your LaTeX document requires adding
%additional package(s)
\usepackage{graphicx}

% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.

\title{CroCoSyn: A Cross-Lingual and Cross-Model Corpus of LLM-Generated Film Synopses}

% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
%         Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
%         Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \And  ... \And
%         Author n \\ Address line \\ ... \\ Address line}
% To start a separate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\  ... \\ Address line
%         \AND
%         Author 2 \\ Address line \\ ... \\ Address line \And
%         Author 3 \\ Address line \\ ... \\ Address line}

\author{Louis Escouflaire \\
  MIT Trope / Address line 1 \\
  Affiliation / Address line 2 \\
  Affiliation / Address line 3 \\
  \texttt{email@domain}}

\author{Louis Escouflaire \\
  MIT Trope Tank — Massachusetts Institute of Technology \\
  Institute for Language and Communication — UCLouvain \\
  \texttt{escouf@mit.edu — louis.escouflaire@uclouvain.be}}

%\author{
%  \textbf{First Author\textsuperscript{1}},
%  \textbf{Second Author\textsuperscript{1,2}},
%  \textbf{Third T. Author\textsuperscript{1}},
%  \textbf{Fourth Author\textsuperscript{1}},
%\\
%  \textbf{Fifth Author\textsuperscript{1,2}},
%  \textbf{Sixth Author\textsuperscript{1}},
%  \textbf{Seventh Author\textsuperscript{1}},
%  \textbf{Eighth Author \textsuperscript{1,2,3,4}},
%\\
%  \textbf{Ninth Author\textsuperscript{1}},
%  \textbf{Tenth Author\textsuperscript{1}},
%  \textbf{Eleventh E. Author\textsuperscript{1,2,3,4,5}},
%  \textbf{Twelfth Author\textsuperscript{1}},
%\\
%  \textbf{Thirteenth Author\textsuperscript{3}},
%  \textbf{Fourteenth F. Author\textsuperscript{2,4}},
%  \textbf{Fifteenth Author\textsuperscript{1}},
%  \textbf{Sixteenth Author\textsuperscript{1}},
%\\
%  \textbf{Seventeenth S. Author\textsuperscript{4,5}},
%  \textbf{Eighteenth Author\textsuperscript{3,4}},
%  \textbf{Nineteenth N. Author\textsuperscript{2,5}},
%  \textbf{Twentieth Author\textsuperscript{1}}
%\\
%\\
%  \textsuperscript{1}Affiliation 1,
%  \textsuperscript{2}Affiliation 2,
%  \textsuperscript{3}Affiliation 3,
%  \textsuperscript{4}Affiliation 4,
%  \textsuperscript{5}Affiliation 5
%\\
%  \small{
%    \textbf{Correspondence:} \href{mailto:email@domain}{email@domain}
%  }
%}

\begin{document}
\maketitle
\begin{abstract}
We introduce CroCoSyn, a controlled, cross-lingual and cross-model corpus of 25,920 LLM-generated film synopses in English and French. Each synopsis is generated under systematically varied conditions, including model type, temperature, genre, protagonist gender, and narrative constraints, and enriched with structured metadata capturing characters and their relationships. Comparing Mistral and Llama across different model temperature degrees, CroCoSyn enables fine-grained analysis of narrative content, style, and character representation across models and languages. The corpus supports research on gender and cultural biases and story generation evaluation, and provides a foundation for comparative studies between LLM-generated and human-written narratives. 
\end{abstract}

\section{Introduction}

Stories are central to how societies transmit values, norms, and cultural models, from myths and novels to films and news narratives \citep{Eliade:1961,Bruner:2010,Gottschall:2012}. They also encode implicit assumptions and stereotypes about gender, culture, and power \citep{Lovatt:2013,casey2021sixty}.

Recent advances in large language models (LLMs) allow narrative texts to be generated at scale and on demand, supporting creative, educational, and journalistic applications \citep{Ray:2023,Cardon:2023}. However, LLM outputs reflect their training data (largely web-based and Western-centric), potentially reproducing stylistic conventions, cultural biases, and representational asymmetries \citep{Baack:2024}.

Despite growing interest in studying LLM-generated text, resources for systematically comparing narratives across languages, models, and other parameters are lacking. To address this gap, we introduce CroCoSyn, a cross-lingual, cross-model corpus of 25,920 film synopses generated by Llama-3 and Mistral. Each synopsis is produced under a fully balanced factorial design controlling language, model, temperature, genre, protagonist gender, and other narrative parameters. Beyond raw text, synopses are enriched with structured metadata capturing characters and relationships, enabling fine-grained quantitative and qualitative analyses. CroCoSyn provides a unique resource to study narrative generation, investigate gender and cultural biases, and compare outputs across models and languages. It also supports broader research on story evaluation, narrative modeling, and computational creativity.  The CroCoSyn corpus will be released as an open-source resource and made freely available on GitHub.

\begin{figure}[t]
    \centering
    \includegraphics[width=\columnwidth]{action_occupations_by_gender.png}
    \caption{Most frequent character occupations by gender in the 5,192 \textit{action} film synopses of the CroCoSyn corpus (non-binary and other genders were not included on the graph for visualization purposes).}
    \label{fig:action-occupations-gender}
\end{figure}

\section{Related Work}

\subsection{Story Generation with LLMs}

Large language models such as GPT-4 and Llama-3 generate fluent and stylistically convincing long-form narratives \citep{Achiam:2023,dubey2024llama}, yet they often struggle with global coherence, character consistency, and overall plot structure \citep{Montfort:2023}. Researchers have highlighted the influence of prompt design and model architectures on narrative outcomes \citep{Bender:2021,harmon2023prompt}. In addition, xtensive research has documented gender and cultural stereotypes in generated text, including biased associations between professions and genders, differential emotional framing, and underrepresentation of marginalized groups \citep{bolukbasi2016man,LiBamman:2021,barroso2024racial}. In narrative contexts, such biases can emerge not only lexically but also through higher-level story elements, such as character agency, role allocation, and genre-specific tropes \citep{Rettberg:2024}. These findings highlight the need for carefully controlled narrative corpora that allow researchers to isolate the effects of individual generation variables on representational outcomes.

\subsection{Corpora of Human and LLM-generated Stories}

Several corpora of human-written narratives exist. ROCStories \citep{Mostafazadeh:2016} has been widely used for modeling hierarchical story generation and LLM-induced story continuation \citep{cavazza2025large}. Broader multilingual datasets such as StoryDB \citep{Tikhonov2021} support cross-lingual comparative narrative research, while STORIUM \citep{Akoury2020} provides richly annotated collaborative stories for machine-in-the-loop generation and narrative quality assessment. Datasets of LLM-generated narratives remain comparatively scarce. TinyStories \citep{eldan2023tinystories} offers simple, controlled narratives to probe small model capabilities and uncover biases \cite{gunti2025uncovering}, TF1-EN-3M \cite{nadas2025tf1} contains 3 million short fables generated by Llama-3, and GPT-WritingPrompts \cite{huang2024gpt} consists of Reddit-formatted short stories generated by GPT-3.5. To our knowledge, no existing corpus provides LLM-generated film synopses across multiple models and languages from the same prompt, making CroCoSyn a unique resource.

\section{Corpus Design}

The CroCoSyn corpus was designed with the goal of enabling systematic comparison across two languages, two models, and multiple linguistic and narrative conditions. All variables leveraged in corpus generation are balanced. This approach allows for fine-grained analysis of how generation parameters influence narrative structure, style, and content. The corpus was also designed to be sufficiently large and diverse to support both quantitative analysis and close reading.

\subsection{Models}

The corpus was generated using two large language models, Llama-3 \cite{dubey2024llama} and Mistral \cite{Mistral2024}. These models were selected for their widespread use, open or semi-open availability, and because they originate from companies based in different countries (the United States for Llama-3 and France for Mistral), a feature relevant to the cross-lingual and cross-cultural aspects of the corpus. We used the "small" versions of the models, respectively Llama-3-8B-Instruct and Mistral-7B. These versions were considered sufficient for generating short texts such as film synopses, while keeping resource requirements manageable. Each model generated exactly half of the corpus, with identical prompting structures. Llama-3 and Mistral were both trained on partially overlapping large-scale web and textual corpora, but the exact composition of their training datasets differs. Such differences can affect model knowledge, style, and multilingual capabilities, which can be investigated using the CroCoSyn corpus.

To investigate the effect of decoding stochasticity on narrative generation, two temperature conditions were defined: \textit{low} and \textit{high}. The low-temperature condition was set to 0.1 for both models, favoring rather deterministic and conservative outputs. The high-temperature condition was set to 0.8 for Llama and 1.0 for Mistral, following differences observed in both models' responses to varying levels of temperature during a preliminary research stage and aiming to produce comparable levels of variability.

\subsection{Generation Variables}

The corpus follows a full factorial design in which each synopsis is generated under a unique combination of controlled parameters. These parameters include language, model, model temperature, film genre, target length, writer nationality, writer gender, protagonist gender, temporal setting, and evaluative framing through prompt adjectives. All variables are evenly distributed across the corpus, ensuring that no condition is over- or under-represented. This balanced structure enables direct comparison across individual dimensions as well as the analysis of interaction effects between them. The distributions of variables combining into 25,920 different synopses are presented below, together with a brief description of each variable.

\begin{itemize}[noitemsep, topsep=2pt]
  \item \textit{language}: the language of both the prompt and the generated synopsis, English or French.
  \item \textit{model}: the large language model used for generation, Mistral (Mistral-7B) or Llama (Llama-3-8B-Instruct).
  \item \textit{model temperature}: the decoding temperature used during text generation, controlling the level of stochasticity. Two relative levels are defined: \textit{low} (0.1 for both models) and \textit{high} (0.8 for Llama and 1.0 for Mistral).
  \item \textit{genre}: the intended film genre specified in the prompt. Five genres are included: crime, romance, comedy, action, and drama.
  \item \textit{target length}: the approximate length requested in the prompt, set to either 250 or 350 words.
  \item \textit{writer nationality}: the nationality assigned to the writer persona in the system prompt, either United States or France.
  \item \textit{writer gender}: the gender assigned to the writer persona in the system prompt (male, female, or unspecified).
  \item \textit{protagonist gender}: the gender of the main character of the story, as constrained by the prompt (male, female, or unspecified).
  \item \textit{temporal setting}: the narrative time frame suggested in the prompt (past, present, or unspecified).
  \item \textit{adjective}: an optional adjective framing the synopsis evaluatively (\emph{good}, \emph{great}, \emph{compelling}, \emph{fun}, \emph{dark}, or unspecified).
\end{itemize}

\subsection{Prompt Template}\label{subsec:prompt}

All synopses are generated using a shared prompt template, with controlled variables injected through parameterized slots. To ensure comparability across languages, English and French prompts were designed to be semantically equivalent. The prompts explicitly require that all main characters be named, facilitating character-level analysis. No additional stylistic constraints were imposed beyond the controlled variables. The English and French equivalent prompts (and system prompts) below were used to generate two of the 25,920 synopses in the corpus (two per model):\\

\noindent
\begin{small}
\textbf{English prompt:}
\begin{itemize}[noitemsep, topsep=2pt, leftmargin=2em]
    \item Model: \textit{Mistral}
    \item Temperature: \textit{low}
    \item System prompt: You are a \textit{female} script writer from \textit{France}.
    \item User prompt: Write a \textit{good} film synopsis in English (around \textit{250} words) in the \textit{action} genre, \textit{set in the past}, with a \textit{male} protagonist. All main characters should be named.
\end{itemize}
\end{small}


\noindent
\begin{small}
\textbf{French prompt:}
\begin{itemize}[noitemsep, topsep=2pt, leftmargin=2em]
    \item Model: \textit{Mistral}
    \item Temperature: \textit{low}
    \item System prompt: Vous êtes \textit{une} scénariste \textit{française}.
    \item User prompt: Rédigez un synopsis de film \textit{de qualité} en français (environ \textit{250} mots) dans le genre \textit{du film d'action}, \textit{situé dans le passé}, dont le protagoniste est \textit{une femme}. Tous les personnages principaux doivent être nommés.
\end{itemize}
\end{small}

\subsection{Structured Data Enrichment}

Each synopsis in the CroCoSyn corpus is accompanied by structured metadata capturing narrative elements such as the film title, character information, and inter-character relationships. To extract this information from each synopsis, we used a zero-shot pipeline relying on Llama-3-8B-Instruct, which was given the entry and prompted to output a valid JSON structure containing the following elements:

\begin{itemize}[noitemsep, topsep=2pt]
  \item \textit{title:} the film’s title.
  \item \textit{characters:} a list of characters, each with a name, gender, occupation, character traits, and role (protagonist, secondary, antagonist).
  \item \textit{relationships:} a list of relationships between characters, specifying source, target, and relationship type (e.g., family, romantic, enemy).
\end{itemize}

Each JSON output is automatically validated and corrected if necessary, guaranteeing that all structured metadata adheres to a consistent schema.

\section{Corpus Description}

Future research will be devoted to the results of a comparative analysis of character descriptions and roles in synopses generated across the different variables. Here, we present an example entry from the corpus as well as some preliminary results. 

\subsection{Example}

\begin{small}
\textbf{Movie Title:} "The Shadow of Versailles"

\noindent\textbf{Synopsis:}

\begin{quote}

In 18th century France, amidst the grandeur of the Palace of Versailles, a young and fearless female spy, Colette Dumont, navigates the treacherous world of espionage. Colette, a skilled thief and assassin, is recruited by the enigmatic and charismatic King Louis XV to infiltrate the inner circle of the ruthless and cunning Cardinal de Rohan. Colette's mission is to uncover the Cardinal's plans to overthrow the King and restore the power of the French monarchy to the nobility. As she navigates the intricate web of court politics, Colette finds herself drawn to the Cardinal's charming and cunning nephew, the Marquis de la Croix. As Colette delves deeper into the Cardinal's plans, she must confront her own demons and confront the darkness within herself. With the help of her trusted ally, the street-smart and resourceful Pierre, Colette must use all her skills and cunning to outwit the Cardinal's henchmen and prevent a catastrophic war.
\end{quote}
\end{small}


\noindent
\begin{small}
\textbf{Characters:}
\begin{itemize}[noitemsep, topsep=2pt, leftmargin=2em]
    \item \textit{Colette Dumont} — female, spy; traits: fearless, skilled, charismatic; role: protagonist
    \item \textit{King Louis XV} — male, king; traits: enigmatic, charismatic; role: secondary
    \item \textit{Cardinal de Rohan} — male, cardinal; traits: ruthless, cunning; role: antagonist
    \item \textit{Marquis de la Croix} — male, noble; traits: charming, cunning; role: secondary
    \item \textit{Pierre} — male, unspecified; traits: resourceful; role: secondary
\end{itemize}
\end{small}


\noindent
\begin{small}
\textbf{Relationships:}
\begin{itemize}[noitemsep, topsep=2pt, leftmargin=2em]
    \item \textit{Colette Dumont} $\to$ \textit{King Louis XV} — employer
    \item \textit{Colette Dumont} $\to$ \textit{Cardinal de Rohan} — enemy
    \item \textit{Colette Dumont} $\to$ \textit{Marquis de la Croix} — romantic
    \item \textit{Colette Dumont} $\to$ \textit{Pierre} — friend
    \item \textit{Cardinal de Rohan} $\to$ \textit{Marquis de la Croix} — family
\end{itemize}
\end{small}

This example was generated using the prompt and parameters of the English example prompt presented in section \ref{subsec:prompt}. It presents a film synopsis along with the metadata associated to it in the corpus: film title, character information (name, gender, occupation, traits, role) and relationships.\\

\subsection{Preliminary quantitative results}


While qualitative analysis of the synopses output by the models is possible, for example by examining the style and narrative structure of the example presented in the previous section, the metadata associated to each synopsis allows for quantitative analysis of the corpus across several dimensions. In this early presentation of results, we focus on trends related to the gender of characters model personas in the CroCoSyn corpus.

Figure~\ref{fig:action-occupations-gender} shows that some occupations are significantly more often attributed to male characters (\textit{mercenary}, \textit{detective}), while others are more often female (\textit{hacker}, \textit{journalist}). Overall, such a plot highlights gendered trends in role allocation in stories generated by the models, suggesting that certain occupations are more stereotypically associated with one gender than the other.

Figure~\ref{fig:wordcloud_male_female} suggests that attributing a gender persona to the LLM tends to influence its generation towards stereotyped outputs: words that emphasize conflict and crime (\textit{redemption}, \textit{deadly}, \textit{underworld}) are more common in ``male-written'' stories, and they tend to include more male characters (as shown by the male names appearing). Female persona-associated words include a higher frequency of female character names, and words related to relationships and emotions (\textit{mother}, \textit{secret}, \textit{closer}).

\begin{figure}[ht]
    \centering
    \includegraphics[width=\linewidth]{wordcloud_male_female.png}
    \caption{WordClouds of words most distinctive to all synopses generated by LLMs with male- (left) and female- (right) writer personas, as specified in the system prompt. Word importance was computed using log-odds ratios of normalized word frequencies among the top 200 words, after removing stopwords and non-informative tokens. Word size reflects the strength of association with each persona.}
    \label{fig:wordcloud_male_female}
\end{figure}

\section{Conclusion}

The CroCoSyn corpus provides a large, controlled, and cross-lingual dataset of LLM-generated film synopses, enriched with structured narrative metadata. This resource enables systematic investigation of how the choice of a specific model, language, temperature, and prompt variables influence narrative content, style, and character representation. Beyond bias and representational studies, the corpus supports applications in narrative modeling, story generation evaluation, and computational creativity research. It opens avenues for both quantitative and qualitative analyses, providing a foundation for future work on understanding and improving the sociocultural and structural properties of LLM outputs.

Looking ahead, we plan to extend the corpus to additional languages, models, and narrative variables, further broadening the scope for cross-lingual and cross-model analyses. A first line of research we aim to pursue with the corpus is a systematic comparison between LLM-generated and human-written synopses, to better understand potential representation imbalance in machine-generated storytelling.


\bibliography{custom}
\bibliographystyle{acl_natbib}


\end{document}
