Abhinav Sethy


2024

pdf
Generating Contextual Images for Long-Form Text
Avijit Mitra | Nalin Gupta | Chetan Naik | Abhinav Sethy | Kinsey Bice | Zeynab Raeesy
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

We investigate the problem of synthesizing relevant visual imagery from generic long-form text, leveraging Large Language Models (LLMs) and Text-to-Image Models (TIMs). Current Text-to-Image models require short prompts that describe the image content and style explicitly. Unlike image prompts, generation of images from general long-form text requires the image synthesis system to derive the visual content and style elements from the text. In this paper, we study zero-shot prompting and supervised fine-tuning approaches that use LLMs and TIMs jointly for synthesizing images. We present an empirical study on generating images for Wikipedia articles covering a broad spectrum of topic and image styles. We compare these systems using a suite of metrics, including a novel metric specifically designed to evaluate the semantic correctness of generated images. Our study offers a preliminary understanding of existing models’ strengths and limitation for the task of image generation from long-form text, and sets up an evaluation framework and establishes baselines for future research.

2023

pdf
Learning to Retrieve Engaging Follow-Up Queries
Christopher Richardson | Sudipta Kar | Anjishnu Kumar | Anand Ramachandran | Zeynab Raeesy | Omar Khan | Abhinav Sethy
Findings of the Association for Computational Linguistics: EACL 2023

Open domain conversational agents can answer a broad range of targeted queries. However, the sequential nature of interaction with these systems makes knowledge exploration a lengthy task which burdens the user with asking a chain of well phrased questions. In this paper, we present a retrieval based system and associated dataset for predicting the next questions that the user might have. Such a system can proactively assist users in knowledge exploration leading to a more engaging dialog. The retrieval system is trained on a dataset called the Follow-up Query Bank (FQ-Bank). FQ-Bank contains ~14K multi-turn information-seeking conversations with a valid follow-up question and a set of invalid candidates. The invalid candidates are generated to simulate various syntactic and semantic confounders such as paraphrases, partial entity match, irrelevant entity, and ASR errors. We use confounder specific techniques to simulate these negative examples on the OR-QuAC dataset. Then, we train ranking models on FQ-Bank and present results comparing supervised and unsupervised approaches. The results suggest that we can retrieve the valid follow-ups by ranking them in higher positions compared to confounders, but further knowledge grounding can improve ranking performance.FQ-Bank is publicly available at https://github.com/amazon-science/fq-bank.

2011

pdf
Learning Sub-Word Units for Open Vocabulary Speech Recognition
Carolina Parada | Mark Dredze | Abhinav Sethy | Ariya Rastrow
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

2010

pdf
Unsupervised Model Adaptation using Information-Theoretic Criterion
Ariya Rastrow | Frederick Jelinek | Abhinav Sethy | Bhuvana Ramabhadran
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

2009

pdf
Fast decoding for open vocabulary spoken term detection
Bhuvana Ramabhadran | Abhinav Sethy | Jonathan Mamou | Brian Kingsbury | Upendra Chaudhari
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers

2006

pdf
Selecting relevant text subsets from web-data for building topic specific language models
Abhinav Sethy | Panayiotis Georgiou | Shrikanth Narayanan
Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers

pdf
Text data acquisition for domain-specific language models
Abhinav Sethy | Panayiotis G. Georgiou | Shrikanth Narayanan
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing