@inproceedings{havard-etal-2020-catplayinginthesnow,
    title = "Catplayinginthesnow: Impact of Prior Segmentation on a Model of Visually Grounded Speech",
    author = "Havard, William  and
      Besacier, Laurent  and
      Chevrot, Jean-Pierre",
    editor = "Fern{\'a}ndez, Raquel  and
      Linzen, Tal",
    booktitle = "Proceedings of the 24th Conference on Computational Natural Language Learning",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2020.conll-1.22/",
    doi = "10.18653/v1/2020.conll-1.22",
    pages = "291--301",
    abstract = "The language acquisition literature shows that children do not build their lexicon by segmenting the spoken input into phonemes and then building up words from them, but rather adopt a top-down approach and start by segmenting word-like units and then break them down into smaller units. This suggests that the ideal way of learning a language is by starting from full semantic units. In this paper, we investigate if this is also the case for a neural model of Visually Grounded Speech trained on a speech-image retrieval task. We evaluated how well such a network is able to learn a reliable speech-to-image mapping when provided with phone, syllable, or word boundary information. We present a simple way to introduce such information into an RNN-based model and investigate which type of boundary is the most efficient. We also explore at which level of the network`s architecture such information should be introduced so as to maximise its performances. Finally, we show that using multiple boundary types at once in a hierarchical structure, by which low-level segments are used to recompose high-level segments, is beneficial and yields better results than using low-level or high-level segments in isolation."
}