@inproceedings{ennen-etal-2023-hierarchical,
    title = "Hierarchical Representations in Dense Passage Retrieval for Question-Answering",
    author = "Ennen, Philipp  and
      Freddi, Federica  and
      Lin, Chyi-Jiunn  and
      Kung, Po-Nien  and
      Wang, RenChu  and
      Yang, Chien-Yi  and
      Shiu, Da-shan  and
      Bernacchia, Alberto",
    editor = "Akhtar, Mubashara  and
      Aly, Rami  and
      Christodoulopoulos, Christos  and
      Cocarascu, Oana  and
      Guo, Zhijiang  and
      Mittal, Arpit  and
      Schlichtkrull, Michael  and
      Thorne, James  and
      Vlachos, Andreas",
    booktitle = "Proceedings of the Sixth Fact Extraction and VERification Workshop (FEVER)",
    month = may,
    year = "2023",
    address = "Dubrovnik, Croatia",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.fever-1.2",
    doi = "10.18653/v1/2023.fever-1.2",
    pages = "17--28",
    abstract = "An approach to improve question-answering performance is to retrieve accompanying information that contains factual evidence matching the question. These retrieved documents are then fed into a reader that generates an answer. A commonly applied retriever is dense passage retrieval. In this retriever, the output of a transformer neural network is used to query a knowledge database for matching documents. Inspired by the observation that different layers of a transformer network provide rich representations with different levels of abstraction, we hypothesize that useful queries can be generated not only at the output layer, but at every layer of a transformer network, and that the hidden representations of different layers may combine to improve the fetched documents for reader performance. Our novel approach integrates retrieval into each layer of a transformer network, exploiting the hierarchical representations of the input question. We show that our technique outperforms prior work on downstream tasks such as question answering, demonstrating the effectiveness of our approach.",
}