@inproceedings{xu-etal-2023-bridging,
    title = "Bridging the Granularity Gap for Acoustic Modeling",
    author = "Xu, Chen  and
      Zhang, Yuhao  and
      Jiao, Chengbo  and
      Liu, Xiaoqian  and
      Hu, Chi  and
      Zeng, Xin  and
      Xiao, Tong  and
      Ma, Anxiang  and
      Wang, Huizhen  and
      Zhu, Jingbo",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest_wac_2008/2023.findings-acl.688/",
    doi = "10.18653/v1/2023.findings-acl.688",
    pages = "10816--10833",
    abstract = "While Transformer has become the de-facto standard for speech, modeling upon the fine-grained frame-level features remains an open challenge of capturing long-distance dependencies and distributing the attention weights. We propose Progressive Down-Sampling (PDS) which gradually compresses the acoustic features into coarser-grained units containing more complete semantic information, like text-level representation. In addition, we develop a representation fusion method to alleviate information loss that occurs inevitably during high compression. In this way, we compress the acoustic features into 1/32 of the initial length while achieving better or comparable performances on the speech recognition task. And as a bonus, it yields inference speedups ranging from 1.20x to 1.47x.By reducing the modeling burden, we also achieve competitive results when training on the more challenging speech translation task."
}