@inproceedings{pawar-etal-2023-evaluation,
    title = "Evaluation Metrics for Depth and Flow of Knowledge in Non-fiction Narrative Texts",
    author = "Pawar, Sachin  and
      Palshikar, Girish  and
      Jain, Ankita  and
      Singh, Mahesh  and
      Rangarajan, Mahesh  and
      Agarwal, Aman  and
      Kumar, Vishal  and
      Singh, Karan",
    editor = "Akoury, Nader  and
      Clark, Elizabeth  and
      Iyyer, Mohit  and
      Chaturvedi, Snigdha  and
      Brahman, Faeze  and
      Chandu, Khyathi",
    booktitle = "Proceedings of the 5th Workshop on Narrative Understanding",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.wnu-1.4",
    doi = "10.18653/v1/2023.wnu-1.4",
    pages = "16--24",
    abstract = "In this paper, we describe the problem of automatically evaluating quality of knowledge expressed in a non-fiction narrative text. We focus on a specific type of documents where each document describes a certain technical problem and its solution. The goal is not only to evaluate the quality of knowledge in such a document, but also to automatically suggest possible improvements to the writer so that a better knowledge-rich document is produced. We propose new evaluation metrics to evaluate quality of knowledge contents as well as flow of different types of sentences. The suggestions for improvement are generated based on these metrics. The proposed metrics are completely unsupervised in nature and they are derived from a set of simple corpus statistics. We demonstrate the effectiveness of the proposed metrics as compared to other existing baseline metrics in our experiments.",
}