Huilin Lu
2026
DeepResearch Retail: Benchmarking Tool-Augmented Deep Research in the E-Commerce Domain
Rafael Ferreira | Flavio Di Palo | Huilin Lu | Ayush Jain | Harsha Aduri
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Rafael Ferreira | Flavio Di Palo | Huilin Lu | Ayush Jain | Harsha Aduri
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Deep Research (DR) systems autonomously retrieve and synthesize information from web sources, however, industrial DR applications face a critical gap: effective integration of internal tools with web search. In this work, we introduce DeepResearch Retail, an evaluation framework grounded in real-world e-commerce data for assessing Deep Research with tools (DR+Tools) in realistic commercial settings. The framework evaluates both factual faithfulness and multidimensional response quality when reasoning over heterogeneous web and internal data sources.We further present Hybrid-ReAct, a multi-agent architecture that demonstrates how collaborative reasoning and tool use can produce evidence-grounded answers. Experimental results validate our framework’s utility, showing improvements in agent’s performance when leveraging web-page information and multi-agent specialization.
2024
Auto-Evolve: Enhancing Large Language Model’s Performance via Self-Reasoning Framework
Krishna Aswani | Huilin Lu | Pranav Patankar | Priya Dhalwani | Xue Tan | Jayant Ganeshmohan | Simon Lacasse
Findings of the Association for Computational Linguistics: EMNLP 2024
Krishna Aswani | Huilin Lu | Pranav Patankar | Priya Dhalwani | Xue Tan | Jayant Ganeshmohan | Simon Lacasse
Findings of the Association for Computational Linguistics: EMNLP 2024
Recent advancements in prompt engineering strategies, such as Chain-of-Thought (CoT) and Self-Discover, have demonstrated significant potential in improving the reasoning abilities of Large Language Models (LLMs). However, these state-of-the-art (SOTA) prompting strategies rely on a fixed set of static seed reasoning modules like “think step by step” or “break down this problem” intended to simulate human approach to problem-solving. This constraint limits the flexibility of models in tackling diverse problems effectively. In this paper, we introduce Auto-Evolve, a novel framework that enables LLMs to self-create dynamic reasoning modules and downstream action plan, resulting in significant improvements over current SOTA methods. We evaluate Auto-Evolve on the challenging BigBench-Hard (BBH) dataset with Claude 2.0, Claude 3 Sonnet, Mistral Large, and GPT-4, where it consistently outperforms the SOTA prompt strategies. Auto-Evolve outperforms CoT by up to 10.4% and on an average by 7% across these four models. Our framework introduces two innovations: a) Auto-Evolve dynamically generates reasoning modules for each task while aligning with human reasoning paradigm, thus eliminating the need for predefined templates. b) An iterative refinement component, that incrementally refines instruction guidance for LLMs and helps boost performance by average 2.8% compared to doing it in a single step.