Characterizing Web Search in The Age of Generative AI

Elisabeth Kirsten, Jost Gro{\ss}e Perdekamp, Qinyuan Wu, Mihir Upadhyay, Krishna P. Gummadi, Muhammad Bilal Zafar


Abstract
The advent of LLMs has given rise to generative search, a new search paradigm in which LLMs retrieve information from the web related to a query and synthesize it into a single, coherent response. This paradigm differs fundamentally from traditional web search, where results are returned as a ranked list of independent web pages. In this paper, we ask: Along what dimensions does generative search differ from traditional search?We conduct a systematic comparison between Google organic search and five generative search systems from three providers: Google, OpenAI, and Perplexity. Our analysis reveals substantial variation among engines in their reliance on internal v.s. external knowledge, source diversity, and stability. While generative systems often achieve topical coverage comparable to traditional search, they do so using markedly different retrieval footprints and synthesis strategies. We further show that the outputs of generative search can vary across time and executions, raising new challenges for robustness. Our findings demonstrate that generative search introduces new dimensions that are not captured by existing evaluation paradigms, motivating the development of evaluations that explicitly account for retrieval behavior, synthesis, and stability in generative search systems.
Anthology ID:
2026.findings-acl.526
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10827–10848
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.526/
DOI:
Bibkey:
Cite (ACL):
Elisabeth Kirsten, Jost Gro{\ss}e Perdekamp, Qinyuan Wu, Mihir Upadhyay, Krishna P. Gummadi, and Muhammad Bilal Zafar. 2026. Characterizing Web Search in The Age of Generative AI. In Findings of the Association for Computational Linguistics: ACL 2026, pages 10827–10848, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Characterizing Web Search in The Age of Generative AI (Kirsten et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.526.pdf
Checklist:
 2026.findings-acl.526.checklist.pdf