@inproceedings{gamonal-etal-2025-audition,
    title = "Audition: A Frame-Annotated Multimodal Dataset for Accessible Audiovisual Content",
    author = "Gamonal, Maucha Andrade  and
      Torrent, Tiago Timponi  and
      Matos, Ely Edison  and
      Pagano, Adriana S.  and
      Belcavello, Frederico  and
      Mayer, Fl{\'a}via Affonso  and
      Lorenzi, Arthur  and
      Sigiliano, Natalia S.  and
      Abreu, Helen de Andrade  and
      Dutra, L{\'i}via Vicente  and
      Viridiano, Marcelo  and
      Coneglian, Andr{\'e}  and
      Herbst, Victor A. S.  and
      Campos, Franciany O.  and
      Brown, Kenneth  and
      Ruiz, L{\'i}via Padua  and
      Bonoto, Lisandra Carvalho  and
      Pereira, Luiz Fernando  and
      Navarro, Yulla Liquer",
    editor = "Harry, Bunt",
    booktitle = "Proceedings of the 21st Joint ACL - ISO Workshop on Interoperable Semantic Annotation (ISA-21)",
    month = sep,
    year = "2025",
    address = {D{\"u}sseldorf, Germany},
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/missing-isa-paper/2025.isa-1.10/",
    ISBN = "979-8-89176-319-7",
    abstract = "This paper presents a multimodal semantic analysis of accessible Brazilian short films using a frame-based annotation approach. We introduce a subset of the Audition dataset, comprising six short films from the animation and documentary genres. We analysed three communicative modes: original audio, audio description, and visual content. Trained annotators semantically annotated each mode following the FrameNet Brazil multimodal methodology. To compare meaning across modalities, we used cosine similarity over frame-semantic representations. Results show that audio description aligns more closely with video content than original audio, reflecting its role in translating visual meaning into language. Our findings demonstrate the effectiveness of frame semantics in modelling meaning across modalities and provide quantitative evidence of audio description as a bridge between visual and verbal communication. The dataset and annotation strategies are a valuable resource for research on multimodal representation, semantic similarity, and accessible media."
}