@inproceedings{trifan-etal-2025-grammar,
    title = "Grammar Pruning: Enabling Low-Latency Zero-Shot Task-Oriented Language Models for Edge {AI}",
    author = "Trifan, Octavian Alexandru  and
      Weber, Jason Lee  and
      Trifan, Marc Titus  and
      Nicolau, Alexandru  and
      Veidenbaum, Alexander",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-luhme/2025.emnlp-main.858/",
    doi = "10.18653/v1/2025.emnlp-main.858",
    pages = "16955--16968",
    ISBN = "979-8-89176-332-6",
    abstract = "Edge deployment of task-oriented semantic parsers demands high accuracy under tight latency and memory budgets. We present Grammar Pruning, a lightweight zero-shot framework that begins with a user-defined schema of API calls and couples a rule-based entity extractor with an iterative grammar-constrained decoder: extracted items dynamically prune the context-free grammar, limiting generation to only those intents, slots, and values that remain plausible at each step. This aggressive search-space reduction both reduces hallucinations and slashes decoding time. On the adapted FoodOrdering, APIMixSNIPS, and APIMixATIS benchmarks, Grammar Pruning with small language models achieves an average execution accuracy of over 90{\%}{---}rivaling State-of-the-Art, cloud-based solutions{---}while sustaining at least 2x lower end-to-end latency than existing methods. By requiring nothing beyond the domain{'}s full API schema values yet delivering precise, real-time natural-language understanding, Grammar Pruning positions itself as a practical building block for future edge-AI applications that cannot rely on large models or cloud offloading."
}