@inproceedings{ayyubi-etal-2025-puzzlegpt,
    title = "{P}uzzle{GPT}: Emulating Human Puzzle-Solving Ability for Time and Location Prediction",
    author = "Ayyubi, Hammad  and
      Feng, Xuande  and
      Liu, Junzhang  and
      Lin, Xudong  and
      Wang, Zhecan  and
      Chang, Shih-Fu",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.111/",
    pages = "2099--2116",
    ISBN = "979-8-89176-195-7",
    abstract = "The task of predicting time and location from images is challenging and requires complex human-like puzzle-solving ability over different clues. In this work, we formalize this ability into core skills and implement them using different modules in an expert pipeline called PuzzleGPT. PuzzleGPT consists of a perceiver to identify visual clues, a reasoner to deduce prediction candidates, a combiner to combinatorially combine information from different clues, a web retriever to get external knowledge if the task can{'}t be solved locally, and a noise filter for robustness. This results in a zero-shot, interpretable, and robust approach that records state-of-the-art performance on two datasets {--} TARA and WikiTilo. PuzzleGPT outperforms large VLMs such as BLIP-2, InstructBLIP, LLaVA, and even GPT-4V, as well as automatically generated reasoning pipelines like VisProg, by at least 32{\%} and 38{\%}, respectively. It even rivals or surpasses finetuned models."
}