@inproceedings{luong-etal-2025-graph,
    title = "Graph-Score: A Graph-grounded Metric for Audio Captioning",
    author = "Luong, Manh  and
      Haffari, Gholamreza  and
      Phung, Dinh  and
      Qu, Lizhen",
    editor = "Kummerfeld, Jonathan K.  and
      Joshi, Aditya  and
      Dras, Mark",
    booktitle = "Proceedings of The 23rd Annual Workshop of the Australasian Language Technology Association",
    month = nov,
    year = "2025",
    address = "Sydney, Australia",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-alta/2025.alta-main.13/",
    pages = "192--201",
    ISBN = "1834-7037",
    abstract = "Evaluating audio captioning systems is a challenging problem since the evaluation process must consider numerous semantic alignments of candidate captions, such as sound event matching and the temporal relationship among them. The existing metrics fail to take these alignments into account as they consider either statistical overlap (BLEU, SPICE, CIDEr) or latent representation similarity (FENSE). To tackle the aforementioned issues of the current metrics, we propose the graph-score, which grounds audio captions to semantic graphs, for better measuring the performance of AAC systems. Our proposed metric achieves the highest agreement with human judgment on the pairwise benchmark datasets. Furthermore, we contribute high-quality benchmark datasets to make progress in developing evaluation metrics for the audio captioning task."
}