Elamur Research⌕
☰
home / papers / 2026 / Beyond Query Bias: Candidate-Aware Iterative Refinement for Zero-Shot Composed Image Retrieval

Beyond Query Bias: Candidate-Aware Iterative Refinement for Zero-Shot Composed Image Retrieval

Paper · 2026 · Nan Sun, Jing Tang, Lei Sun et al.

Summary

The paper addresses a weakness in Zero-Shot Composed Image Retrieval (ZS-CIR), where a system must find a target image from a reference image plus a modification text without task-specific training. Current approaches ask an MLLM to write a composed query that CLIP-style embedders then match. The MLLM never sees the candidate pool, so even plausible queries can miss it. The authors call this "cognitive bias." Their training-free framework, CoRR (Chain of Reflective Composed Image Retrieval), runs retrieval in a loop. An MLLM uses chain-of-thought prompting to inspect the top retrieved images and their captions, then rewrites the query by loosening, fixing, or adding constraints. The new query embedding is blended with earlier ones through spherical interpolation (Historical Query Fusion). Retrieval-Driven Caption Optimization supplies captions of candidates, produced offline, so generated queries match the embedder's preferred wording. On benchmarks including CIRCO and FashionIQ, CoRR beats strong baselines and reports state-of-the-art results after only two extra rounds. This shows that feedback from retrieval can correct biased queries without any training.

Key points

  • Iteration uses the Top-K=5 retrieved images and Top-N=10 synthetic captions as MLLM context. Fusion uses Slerp with weight α=0.8, and gains level off after 2 rounds.
  • In the CIRCO ablation with an MMRet-Large baseline, mAP@5 goes from 37.75 to 42.77 with all components. Reflection alone slightly hurts (36.66) until query fusion is added (40.82).
  • On FashionIQ Dress R@10, Slerp fusion reaches 31.33 against a baseline of 29.84. Linear interpolation (28.84) and pseudo-relevance feedback (27.56) do worse.
  • The approach works across MLLMs (Qwen-2.5VL-3B/7B/72B, LLaVA-NeXT-7B), and larger models give bigger gains.
  • Each refinement round adds about 3 s of latency, which the authors report is still faster than LDRE and OSrCIR.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🎓Huazhong University of Science and TechnologyWuhan · China45 works · 6,345 citations🎓The University of QueenslandBrisbane · Australia14 works · 4,603 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Nan Sun, Jing Tang, Lei Sun, Rui Chen, Yuxing Lu, Xiangxiang Chu, Hefei Ling and Yujun Cai. 2026. Beyond Query Bias: Candidate-Aware Iterative Refinement for Zero-Shot Composed Image Retrieval. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1120

@inproceedings{sun2026beyond,
  title = {{Beyond Query Bias: Candidate-Aware Iterative Refinement for Zero-Shot Composed Image Retrieval}},
  author = {Nan Sun, Jing Tang, Lei Sun, Rui Chen, Yuxing Lu, Xiangxiang Chu, Hefei Ling and Yujun Cai},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1120},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1120}
}

References

54 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Umer Anwaar, Egor Labintcev, and Martin Kleinsteuber. 2021. Compositional learning of image-text query for image retrieval. In WACV. find
  2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025a. Qwen2.5-vl technical report. arXiv preprint arXiv:2…
  3. Sule Bai, Mingxing Li, Yong Liu, Jing Tang, Haoji Zhang, Lei Sun, Xiangxiang Chu, and Yansong Tang. 2025b. Univg-r1: Reasoning guided universal visual grounding with reinforcement learning. arXiv preprint arXiv:2505.14231. — [arXiv:2505.14231](https://arxiv.org/abs/2505.14231) find
  4. Yang Bai, Xinxing Xu, Yong Liu, Salman Khan, Fahad Khan, Wangmeng Zuo, Rick Siow Mong Goh, and Chun-Mei Feng. 2024. Sentence-level prompts benefit composed image retrieval. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net… find
  5. Simion-Vlad Bogolin, Ioana Croitoru, Hailin Jin, Yang Liu, and Samuel Albanie. 2022. Cross modal retrieval with querybank normalisation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5194–5205. find
  6. Gloria Bordogna and Gabriella Pasi. 1993. A fuzzy linguistic approach generalizing boolean information retrieval: A model and its evaluation. Journal of the American Society for Information Science, 44(2):70– 82. find
  7. Rui Chen, Lei Sun, Jing Tang, Geng Li, and Xiangxiang Chu. 2025. Finger: Content aware fine-grained evaluation with reasoning for ai-generated videos. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 3517–3526. find
  8. Yanbei Chen and Loris Bazzani. 2020. Learning joint visual semantic matching embeddings for language-guided retrieval. In ECCV. find
  9. Yanbei Chen, Shaogang Gong, and Loris Bazzani. 2020. Image search with text feedback by visiolinguistic attention learning. In CVPR. find
  10. Yixin Chen and James Ze Wang. 2002. A region-based fuzzy feature matching approach to content-based image retrieval. IEEE transactions on pattern analysis and machine intelligence, 24(9):1252–1267. find
  11. Ginger Delmas, Rafael Sampaio de Rezende, Gabriela Csurka, and Diane Larlus. 2022. Artemis: Attention-based retrieval with text-explicit matching and implicit similarity. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. find
  12. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library.
  13. Geonmo Gu, Sanghyuk Chun, Wonjae Kim, Yoohoon Kang, and Sangdoo Yun. 2024. Language-only training of zero-shot composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13225–13234. find
  14. Yang bai, Xinxing Xu, Yong Liu, Salman Khan, Fahad Khan, Wangmeng Zuo, Rick Siow Mong Goh, and Chun-Mei Feng. 2024. Sentence-level prompts benefit composed image retrieval. In ICLR. find
  15. Xintong Han, Zuxuan Wu, Phoenix X Huang, Xiao Zhang, Menglong Zhu, Yuan Li, Yang Zhao, and Larry S Davis. 2017. Automatic spatially-aware fashion concept discovery. In ICCV. find
  16. Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Alberto Del Bimbo. 2023. Zero-shot composed image retrieval with textual inversion. In ICCV. find
  17. Chuong Huynh, Jinyu Yang, Ashish Tawari, Mubarak Shah, Son Tran, Raffay Hamid, Trishul Chilimbi, and Abhinav Shrivastava. 2025. Collm: A large language model for composed image retrieval. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3994–4004. find
  18. Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. 2022. Effective conditioned and Young Kyun Jang, Dat Huynh, Ashish Shah, Wen-Kai Chen, and Ser-Nam Lim. 2024. Spherical linear interpolation and text-anchoring for zero-shot composed image retrieval. In European Conference on… find
  19. Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini, and Zeynep Akata. 2024. Vision-by-language for training-free compositional image retrieval. International Conference on Learning Representations (ICLR). find
  20. Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. 2022. Diffusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2426–2435. find
  21. Seungmin Lee, Dongwan Kim, and Bohyung Han. 2021. Cosmo: Content-style modulation for image retrieval with text feedback. In CVPR. find
  22. Haiwen Li, Zining Chen, Ying Liu, Fei Su, and Zhicheng Zhao. 2025. Slot inversion for asymmetric composed image retrieval. In 2025 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE. find
  23. Haiwen Li, Delong Liu, Zhaohui Hou, Zeliang Ma, Fei Su, and Zhicheng Zhao. 2026. Modality and task adaptation for enhanced zero-shot composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 6100–6108. find
  24. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023a. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR.
  25. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.
  26. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML.
  27. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306.
  28. Zheyuan Liu, Cristian Rodriguez-Opazo, Damien Teney, and Stephen Gould. 2021. Image retrieval on real-life images with pre-trained vision-and-language models. In ICCV. find
  29. Pengfei Luo, Jingbo Zhou, Tong Xu, Yuan Xia, Linli Xu, and Enhong Chen. 2025. Imagescope: Unifying language-guided image retrieval via large multimodal model collective reasoning. In Proceedings of the ACM on Web Conference 2025, pages 1666–1682. find
  30. Jabez Magomere, Emanuele La Malfa, Manuel Tonneau, Ashkan Kazemi, and Scott A. Hale. 2025. When claims evolve: Evaluating and enhancing the robustness of embedding models against misinformation edits. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22374–22404, Vienna,… find
  31. Mandar Mitra, Amit Singhal, and Chris Buckley. 1998. Improving automatic query expansion. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 206–214. find
  32. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, and 2 others. 20… — PyTorch: An Imperative Style, High-Performance Deep Learning Library
  33. Xiangyan Qu, Gaopeng Gou, Jiamin Zhuang, Jing Yu, Kun Song, Qihao Wang, Yili Li, and Gang Xiong. 2025. Proapo: Progressively automatic prompt optimization for visual classification. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 25145–25155. find
  34. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In ICML.
  35. Javad Rafiei Asl, Prajwal Panzade, Eduardo Blanco, Daniel Takabi, and Zhipeng Cai. 2024. RobustSentEmbed: Robust sentence embeddings using adversarial self-supervised contrastive learning. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3795–3809, Mexico City, Mexico.… find
  36. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695. find
  37. Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, and Tomas Pfister. 2023. Pic2word: Mapping pictures to words for zero-shot composed image retrieval. In CVPR. find
  38. Sakib Shahriar, Brady D Lund, Nishith Reddy Mannuru, Muhammad Arbab Arshad, Kadhim Hayawi, Ravi Varma Kumar Bevara, Aashrith Mannuru, and Laiba Batool. 2024. Putting gpt-4o to the sword: A comprehensive evaluation of language, vision, speech, and multimodal proficiency. Applied Sciences, 14(17):7782… find
  39. Ken Shoemake. 1985. Animating rotation with quaternion curves. In Proceedings of the 12th annual conference on Computer graphics and interactive techniques, pages 245–254. find
  40. International ACM SIGIR conference on research and development in information retrieval, pages 80– 90. find
  41. Zelong Sun, Dong Jing, and Zhiwu Lu. 2025. Cotmr: Chain-of-thought multi-scale reasoning for training-free zero-shot composed image retrieval. arXiv preprint arXiv:2502.20826. — [arXiv:2502.20826](https://arxiv.org/abs/2502.20826) find
  42. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, and 1 others. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.
  43. Yuanmin Tang, Jing Yu, Keke Gai, Jiamin Zhuang, Gang Xiong, Gaopeng Gou, and Qi Wu. 2025a. Missing target-relevant information prediction with world model for accurate zero-shot composed image retrieval. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA… find
  44. Junjie Zhou, Yongping Xiong, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, and Defu Lian. 2025. MegaPairs: Massive data synthesis for universal multimodal retrieval. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
  45. Yuanmin Tang, Jing Yu, Keke Gai, Jiamin Zhuang, Gang Xiong, Yue Hu, and Qi Wu. 2024. Contexti2w: Mapping images to context-dependent words for accurate zero-shot composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5180–5188. find
  46. Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348. find
  47. Yuanmin Tang, Jue Zhang, Xiaoting Qin, Jing Yu, Gaopeng Gou, Gang Xiong, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Wu. 2025b. Reason-before-retrieve: One-stage reflective chain-of-thoughts for training-free zero-shot composed image retrieval. In Proceedings of the Computer Vision and Patt… find
  48. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, and 32 others. 2025. Internvl3: Exploring advanced training and test-…
  49. Sagar Vaze, Nicolas Carion, and Ishan Misra. 2023. Genecis: A benchmark for general conditional image similarity. In CVPR. find
  50. Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. 2019. Composing text and image for image retrieval-an empirical odyssey. In CVPR. find
  51. Tianshi Wang, Fengling Li, Lei Zhu, Jingjing Li, Zheng Zhang, and Heng Tao Shen. 2025. Cross-modal retrieval: a systematic review of methods and future directions. Proceedings of the IEEE. find
  52. Hui Wu, Yupeng Gao, Xiaoxiao Guo, Ziad Al-Halah, Steven Rennie, Kristen Grauman, and Rogerio Feris. 2021. The fashion iq dataset: Retrieving images by combining side information and relative natural language feedback. CVPR. find
  53. Eric Xing, Pranavi Kolouju, Robert Pless, Abby Stylianou, and Nathan Jacobs. 2025. Context-cir: Learning from concepts in text for composed image retrieval. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19638– 19648. find
  54. Zhenyu Yang, Dizhan Xue, Shengsheng Qian, Weiming Dong, and Changsheng Xu. 2024. Ldre: Llm-based divergent reasoning and ensemble for zero-shot composed image retrieval. In Proceedings of the 47th find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sun-2026-beyond-query-bias-candidate-aware/