@inproceedings{garg-etal-2023-evaluation,
    title = "Evaluation of Universal Semantic Representation ({USR})",
    author = "Garg, Kirti  and
      Paul, Soma  and
      Sukhada, Sukhada  and
      Bawahir, Fatema  and
      Kumari, Riya",
    editor = "Bonn, Julia  and
      Xue, Nianwen",
    booktitle = "Proceedings of the Fourth International Workshop on Designing Meaning Representations",
    month = jun,
    year = "2023",
    address = "Nancy, France",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.dmr-1.2",
    pages = "13--22",
    abstract = "Universal Semantic Representation (USR) is designed as a language-independent information packaging system that captures information at three levels: (a) Lexico-conceptual, (b) Syntactico-Semantic, and (c) Discourse. Unlike other representations that mainly encode predicates and their argument structures, our proposed representation captures the speaker{'}s vivakṣ{\=a}- how the speaker views the activity. The idea of {``}speaker{'}s vivakṣ{\=a} is inspired by Indian Grammatical Tradition. There can be some amount of idiosyncrasy of the speaker in the annotation since it is the speaker{'}s view- point that has been captured in the annotation. Hence the evaluation metrics of such resources need to be also thought through from scratch. This paper presents an extensive evaluation procedure of this semantic representation from two perspectives (a) Inter- Annotator Agreement and (b) one downstream task, namely multilingual Natural Language Generation. We also qualitatively evaluate the experience of natural language generation by manual parsing of USR, so as to understand the readability of USR. We have achieved above 80{\%} Inter-Annotator Agreement for USR annotations and above 80{\%} semantic closeness in multi-lingual generation tasks suggesting the reliability of USR annotations and utility for multi-lingual generations. The qualitative evaluation also suggests high readability and hence the utility of USR as a semantic representation.",
}