@inproceedings{wu-etal-2021-cold,
    title = "Cold Start Problem For Automated Live Video Comments",
    author = "Wu, Hao  and
      Pitie, Fran{\c{c}}ois  and
      Jones, Gareth",
    editor = "Zadeh, Amir  and
      Morency, Louis-Philippe  and
      Liang, Paul Pu  and
      Ross, Candace  and
      Salakhutdinov, Ruslan  and
      Poria, Soujanya  and
      Cambria, Erik  and
      Shi, Kelly",
    booktitle = "Proceedings of the Third Workshop on Multimodal Artificial Intelligence",
    month = jun,
    year = "2021",
    address = "Mexico City, Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2021.maiworkshop-1.8/",
    doi = "10.18653/v1/2021.maiworkshop-1.8",
    pages = "54--62",
    abstract = "Live video comments, or ``danmu'', are an emerging feature on Asian online video platforms. Danmu are time-synchronous comments that are overlaid on a video playback. These comments uniquely enrich the experience and engagement of their users. These comments have become a determining factor in the popularity of the videos. Similar to the ``cold start problem'' in recommender systems, a video will only start to attract attention when sufficient danmu comments have been posted on it. We study this video cold start problem and examine how new comments can be generated automatically on less-commented videos. We propose to predict the danmu comments by exploiting a multi-modal combination of the video visual content, subtitles, audio signals, and any surrounding comments (when they exist). Our method fuses these multi-modalities in a transformer network which is then trained for different comment density scenarios. We evaluate our proposed system through both a retrieval based evaluation method, as well as human judgement. Results show that our proposed system improves significantly over state-of-the-art methods."
}