2024
pdf
abs
MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning
Debrup Das
|
Debopriyo Banerjee
|
Somak Aditya
|
Ashish Kulkarni
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Tool-augmented Large Language Models (TALMs) are known to enhance the skillset of large language models (LLMs), thereby, leading to their improved reasoning abilities across many tasks. While, TALMs have been successfully employed in different question-answering benchmarks, their efficacy on complex mathematical reasoning benchmarks, and the potential complementary benefits offered by tools for knowledge retrieval and mathematical equation solving are open research questions. In this work, we present MathSensei, a tool-augmented large language model for mathematical reasoning. We study the complementary benefits of the tools - knowledge retriever (Bing Web Search), program generator + executor (Python), and symbolic equation solver (Wolfram-Alpha API) through evaluations on mathematical reasoning datasets. We perform exhaustive ablations on MATH, a popular dataset for evaluating mathematical reasoning on diverse mathematical disciplines. We also conduct experiments involving well-known tool planners to study the impact of tool sequencing on the model performance. MathSensei achieves 13.5% better accuracy over gpt-3.5-turbo with Chain-of-Thought on the MATH dataset. We further observe that TALMs are not as effective for simpler math word problems (in GSM-8K), and the benefit increases as the complexity and required knowledge increases (progressively over AQuA, MMLU-Math, and higher level complex questions in MATH). The code and data are available at https://github.com/Debrup-61/MathSensei.
2015
pdf
A machine-assisted human translation system for technical documents
Vishwajeet Kumar
|
Ashish Kulkarni
|
Pankaj Singh
|
Ganesh Ramakrishnan
|
Ganesh Arnaal
Proceedings of Machine Translation Summit XV: User Track
pdf
An Approach to Collective Entity Linking
Ashish Kulkarni
|
Kanika Agarwal
|
Pararth Shah
|
Sunny Raj Rathod
|
Ganesh Ramakrishnan
Proceedings of the 12th International Conference on Natural Language Processing
2014
pdf
abs
Efficient Reuse of Structured and Unstructured Resources for Ontology Population
Chetana Gavankar
|
Ashish Kulkarni
|
Ganesh Ramakrishnan
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
We study the problem of ontology population for a domain ontology and present solutions based on semi-automatic techniques. A domain ontology for an organization, often consists of classes whose instances are either specific to, or independent of the organization. E.g. in an academic domain ontology, classes like Professor, Department could be organization (university) specific, while Conference, Programming languages are organization independent. This distinction allows us to leverage data sources both―within the organization and those in the Internet ― to extract entities and populate an ontology. We propose techniques that build on those for open domain IE. Together with user input, we show through comprehensive evaluation, how these semi-automatic techniques achieve high precision. We experimented with the academic domain and built an ontology comprising of over 220 classes. Intranet documents from five universities formed our organization specific corpora and we used open domain knowledge bases like Wikipedia, Linked Open Data, and web pages from the Internet as the organization independent data sources. The populated ontology that we built for one of the universities comprised of over 75,000 instances. We adhere to the semantic web standards and tools and make the resources available in the OWL format. These could be useful for applications such as information extraction, text annotation, and information retrieval.
2012
pdf
Enriching An Academic knowledge base using Linked Open Data
Chetana Gavankar
|
Ashish Kulkarni
|
Yuan Fang Li
|
Ganesh Ramakrishnan
Proceedings of the Workshop on Speech and Language Processing Tools in Education