Elamur Research⌕
☰
home / papers / 2025 / Chat-Driven Text Generation and Interaction for Person Retrieval

Chat-Driven Text Generation and Interaction for Person Retrieval

Paper · 2025 · Zequn Xie, Chuxin Wang, Yeqiang Wang et al.

Summary

Text-based person search (TBPS) retrieves pedestrian images from a gallery using a natural-language description. Training such systems normally requires expensive human-written captions. Automatic captioning can replace them, but it tends to hallucinate, repeat itself, or miss details. The authors propose CTGI, an annotation-free framework built on dialogue with a multimodal LLM, which has two parts.

  • MTG (training): starting from an initial caption, it asks a series of attribute-focused questions. A reconstructor then merges the answers into dense, non-redundant pseudo-labels.
  • MTI (inference): a plug-in step that refines vague user queries. It questions the MLLM about a candidate anchor image, then uses the enriched query to re-rank results.

CLIP's text encoder is also extended to accept the longer captions. Trained without any manual captions on RSTPReid, CTGI outperforms prior unsupervised methods and matches or beats many fully supervised ones. This suggests that TBPS can scale without costly annotation.

Key points

  • Positional embedding stretching (PES): the first 20 learned CLIP positions are kept and the rest are interpolated, raising the input limit from 77 to 248 tokens without retraining. Removing PES lowers RSTPReid Rank-1 from 64.20 to 63.00 (IRRA) and from 66.35 to 65.75 (RDE).
  • RSTPReid with the RDE backbone: Rank-1 66.35, Rank-5 85.50, Rank-10 91.25, mAP 51.51. The IRRA backbone reaches Rank-1 64.20 and mAP 49.66.
  • Comparison with other methods: CTGI clearly beats the best unsupervised baseline, MUMA (Rank-1 54.35). It also exceeds listed fully supervised methods such as IRRA (60.20) and CFAM (62.45).
  • Ablation with IRRA: MTI alone gives Rank-1 55.50, and combining MTG with MTI gives the best results.
  • Stated limitations: noisy pseudo-labels (noise-robust backbones like RDE cope better), extra inference latency from multi-turn MLLM calls, and dependence on how well the MLLM generalizes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Zhejiang UniversityHangzhou · China172 works · 12,988 citations

Also on the byline: Northwest A&F University.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Information retrieval & search · 473 worksText generation · 202 works

Cite

Cite as Zequn Xie, Chuxin Wang, Yeqiang Wang, Sihang Cai, Shulei Wang and Tao Jin. 2025. Chat-Driven Text Generation and Interaction for Person Retrieval. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.266

@inproceedings{xie2025chat,
  title = {{Chat-Driven Text Generation and Interaction for Person Retrieval}},
  author = {Zequn Xie, Chuxin Wang, Yeqiang Wang, Sihang Cai, Shulei Wang and Tao Jin},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.266},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.266}
}

References

69 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mothilal Asokan, Kebin Wu, and Fatima Albreiki. 2025. Finelip: Extending clip’s reach via fine-grained alignment with longer text inputs. arXiv preprint arXiv:2504.01916. — [arXiv:2504.01916](https://arxiv.org/abs/2504.01916) find
  2. Yang Bai, Jingyao Wang, Min Cao, Chen Chen, Ziqiang Cao, Liqiang Nie, and Min Zhang. 2023. Text-based person search without parallel image-text data. In Proceedings of the 31st ACM International Conference on Multimedia, pages 757–767. find
  3. Min Cao, Yang Bai, Ziyin Zeng, Mang Ye, and Min Zhang. 2024. An empirical study of clip for text-based person search. find
  4. Dapeng Chen, Hongsheng Li, Xihui Liu, Yantao Shen, Jing Shao, Zejian Yuan, and Xiaogang Wang. 2018. find
  5. Improving deep visual representation for person re-identification by global and local image-language association. In Proceedings of the European conference on computer vision (ECCV), pages 54–70. find
  6. Yuhao Chen, Guoqing Zhang, Yujiang Lu, Zhenxing Wang, and Yuhui Zheng. 2022. Tipcb: A simple but effective part-based convolutional baseline for text-based person search. Neurocomputing, 494:171–181. find
  7. Xize Cheng, Dongjie Fu, Xiaoda Yang, Minghui Fang, Ruofan Hu, Jingyu Lu, Bai Jionghao, Zehan Wang, Shengpeng Ji, Rongjie Huang, et al. 2025. Omnichat: Enhancing spoken dialogue systems with scalable synthetic data for diverse scenarios. arXiv preprint arXiv:2501.01384. — [arXiv:2501.01384](https://arxiv.org/abs/2501.01384) find
  8. Zefeng Ding, Changxing Ding, Zhiyin Shao, and Dacheng Tao. 2021a. Semantically self-aligned network for text-to-image part-aware person re-identification. arXiv preprint arXiv:2107.12666. — [arXiv:2107.12666](https://arxiv.org/abs/2107.12666) find
  9. Zefeng Ding, Changxing Ding, Zhiyin Shao, and Dacheng Tao. 2021b. Semantically self-aligned network for text-to-image part-aware person re-identification. Preprint, arXiv:2107.12666. — [arXiv:2107.12666](https://arxiv.org/abs/2107.12666) find
  10. Ammarah Farooq, Muhammad Awais, Fei Yan, Josef Kittler, Ali Akbari, and Syed Safwan Khalid. 2020. A convolutional baseline for person re-identification using vision and language descriptions. arXiv preprint arXiv:2003.00808. — [arXiv:2003.00808](https://arxiv.org/abs/2003.00808) find
  11. Dongjie Fu, Xize Cheng, Xiaoda Yang, Wang Hanting, Zhou Zhao, and Tao Jin. 2024. Boosting speech recognition robustness to modality-distortion with contrast-augmented prompts. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3838–3847. find
  12. Takuro Fujii and Shuhei Tarashima. 2023. Bilma: Bidirectional local-matching for text-based person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2786–2790. find
  13. Chenyang Gao, Guanyu Cai, Xinyang Jiang, Feng Zheng, Jun Zhang, Yifei Gong, Pai Peng, Xiaowei Guo, and Xing Sun. 2021. Contextual non-local alignment over full-scale representation for text-based person search. arXiv preprint arXiv:2101.03036. — [arXiv:2101.03036](https://arxiv.org/abs/2101.03036) find
  14. Xiaoxiao Guo, Hui Wu, Yu Cheng, Steven Rennie, Gerald Tesauro, and Rogerio Feris. 2018. Dialog-based interactive image retrieval. In Advances in Neural Information Processing Systems, pages 678–688. find
  15. Zirun Guo, Minjie Hong, and Tao Jin. 2025a. Observer1: Unlocking reasoning abilities of mllms with dynamic progressive reinforcement learning. Preprint, arXiv:2505.12432. — [arXiv:2505.12432](https://arxiv.org/abs/2505.12432) find
  16. Zirun Guo, Shulei Wang, Wang Lin, Weicai Yan, Yangyang Wu, and Tao Jin. 2025b. Efficient prompting for continual adaptation to missing modalities. arXiv preprint arXiv:2503.00528. find
  17. Xiao Han, Sen He, Li Zhang, and Tao Xiang. 2021. Text-based person search with limited data. arXiv:2110.10807. — [arXiv:2110.10807](https://arxiv.org/abs/2110.10807) find
  18. Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. 2021. Transreid: Transformer-based object re-identification. In ICCV, pages 15013– 15022. find
  19. Hai Huang, Yan Xia, Shulei Wang, Hanting Wang, Minghui Fang, Shengpeng Ji, Sashuai Zhou, Tao Jin, and Zhou Zhao. 2025. Open-set cross modal generalization via multimodal unified representation. Preprint, arXiv:2507.14935. — [arXiv:2507.14935](https://arxiv.org/abs/2507.14935) find
  20. D. Jiang and M. Ye. 2023a. Cross-modal implicit relation reasoning and aligning for text-to-image person retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2787–2797. find
  21. Ding Jiang and Mang Ye. 2023b. Cross-modal implicit relation reasoning and aligning for text-to-image person retrieval. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). find
  22. Ding Jiang and Mang Ye. 2023c. Cross-modal implicit relation reasoning and aligning for text-to-image person retrieval. In CVPR, pages 2787–2797. retrieval. In Proceedings of the 31st ACM International Conference on Multimedia, pages 6292–6300. find
  23. Shiping Li, Min Cao, and Min Zhang. 2022. Learning semantic-aligned feature representation for text-based person search. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2724–2728. IEEE. find
  24. Shuang Li, Tong Xiao, Hongsheng Li, Bolei Zhou, Dayu Yue, and Xiaogang Wang. 2017a. Person search with natural language description. In CVPR, pages 1970– 1979. find
  25. Shuang Li, Tong Xiao, Hongsheng Li, Bolei Zhou, Dayu Yue, and Xiaogang Wang. 2017b. Person search with natural language description. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5187–5196. find
  26. Zongyi Li, Li Jianbo, Yuxuan Shi, Jiazhong Chen, Shijuan Huang, Linnan Tu, Fei Shen, and Hefei Ling. 2025b. Exploring the potential of large vision-language models for unsupervised text-based person retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 5119–51… find
  27. Edward Loper and Steven Bird. 2002. Nltk: The natural language toolkit. arXiv preprint cs/0205028. find
  28. Tao Jin, Weicai Yan, Ye Wang, Sihang Cai, Qifan Shuai, and Zhou Zhao. 2024. Calibrating prompt from history for continual vision-language retrieval and grounding. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 4302–4311. find
  29. Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. 2019. Bag of tricks and a strong baseline for deep person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 0–0. find
  30. Ya Jing, Chenyang Si, Junbo Wang, Wei Wang, Liang Wang, and Tieniu Tan. 2020. Pose-guided multi-granularity attention network for text-based person search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11189–11196. find
  31. Tianle Lv, Shuang Li, Jiaxu Leng, and Xinbo Gao. 2024. Mgrl: Mutual-guidance representation learning for text-to-image person retrieval. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2895–2899. IEEE. find
  32. Pranavi Kolouju, Eric Xing, Robert Pless, Nathan Jacobs, and Abby Stylianou. 2025. good4cir: Generating detailed synthetic captions for composed image retrieval. arXiv preprint arXiv:2503.17871. — [arXiv:2503.17871](https://arxiv.org/abs/2503.17871) find
  33. Saehyung Lee, Sangwon Yu, Junsung Park, Jihun Yi, and Sungroh Yoon. 2024. Interactive text-to-image retrieval with large language models: A plug-and-play approach. arXiv preprint arXiv:2406.03411. find
  34. Jie Lei, Xinlei Chen, Ning Zhang, Mengjiao Wang, Mohit Bansal, Tamara L Berg, and Licheng Yu. 2022. Loopitr: Combining dual and cross encoder architectures for image-text retrieval. arXiv:2203.05465. find
  35. Yiwei Ma, Xiaoshuai Sun, Jiayi Ji, Guannan Jiang, Weilin Zhuang, and Rongrong Ji. 2023. Beat: Bidirectional one-to-many embedding alignment for text-based person retrieval. In Proceedings of the 31st ACM International Conference on Multimedia, pages 4157–4168. find
  36. Antoine Miech, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, and Andrew Zisserman. 2021. Thinking fast and slow: Efficient text-to-visual retrieval with transformers. In CVPR, pages 9826–9836. find
  37. OpenAI. 2023. Gpt-4o. Large language model. find
  38. Haiwen Li, Delong Liu, Fei Su, and Zhicheng Zhao. 2025a. Object-centric discriminative learning for text-based person retrieval. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  39. Yang Qin, Yingke Chen, Dezhong Peng, Xi Peng, Joey Tianyi Zhou, and Peng Hu. 2024. Noisy-correspondence learning for text-to-image person re-identification. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). find
  40. Shenshen Li, Xing Xu, Yang Yang, Fumin Shen, Yijun Mo, Yujie Li, and Heng Tao Shen. 2023. Dcel: Deep cross-modal evidential learning for text-based person Yang Qin, Dezhong Peng, Xi Peng, Xu Wang, and Peng Hu. 2022. Deep evidential learning with noisy correspondence for cross-modal retrieval. In Pro… find
  41. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In ICML, pages 8748– 8763. PMLR.
  42. Nikolaos Sarafianos, Xiang Xu, and Ioannis A Kakadiaris. 2019. Adversarial representation learning for text-to-image matching. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5814–5824. find
  43. Zhiyin Shao, Xinyu Zhang, Changxing Ding, Jian Wang, and Jingdong Wang. 2023. Unified pre-training with pseudo texts for text-to-image person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11174–11184. find
  44. Zhiyin Shao, Xinyu Zhang, Meng Fang, Zhifeng Lin, Jian Wang, and Changxing Ding. 2022. Learning granularity-unified representations for text-to-image person re-identification. In ACM MM, pages 5566– 5574. find
  45. Fei Shen, Xiangbo Shu, Xiaoyu Du, and Jinhui Tang. 2023. Pedestrian-specific bipartite-aware similarity learning for text-based person retrieval. In Proceedings of the 31st ACM International Conference on Multimedia, pages 8922–8931. find
  46. Xiujun Shu, Wei Wen, Haoqian Wu, Keyu Chen, Yiran Song, Ruizhi Qiao, Bo Ren, and Xiao Wang. 2022. See finer, see more: Implicit modality alignment for text-based person retrieval. In European Conference on Computer Vision, pages 624–641. Springer. find
  47. Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, and Jingjing Liu. 2021. Lightningdot: Pre-training visual-semantic embeddings for real-time image-text retrieval. In NAACL-HLT, pages 982–997. find
  48. Wei Suo, Mengyang Sun, Kai Niu, Yiqi Gao, Peng Wang, Yanning Zhang, and Qi Wu. 2022. A simple and robust correlation filtering method for text-based person search. In ECCV, pages 726–742. Springer. find
  49. Wentan Tan, Changxing Ding, Jiayu Jiang, Fei Wang, Yibing Zhan, and Dapeng Tao. 2024. Harnessing the power of mllms for transferable text-to-image person reid. In CVPR, pages 17127–17137. find
  50. Haochen Wang, Jiayi Shen, Yongtuo Liu, Yan Gao, and Efstratios Gavves. 2022a. Nformer: Robust person re-identification with neighbor transformer. In CVPR, pages 7297–7307. find
  51. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024a. Qwen2-vl: Enhancing vision-language model’s perception of the…
  52. Zehan Wang, Ziang Zhang, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Hengshuang Zhao, and Zhou Zhao. 2024b. Omnibind: Large-scale omni multimodal representation via binding spaces. Preprint, arXiv:2407.11895. — [arXiv:2407.11895](https://arxiv.org/abs/2407.11895) find
  53. Zhe Wang, Zhiyuan Fang, Jun Wang, and Yezhou Yang. 2020. Vitaa: Visual-textual attributes alignment in person search by natural language. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16, pages 402–420. Springer. find
  54. Zijie Wang, Aichun Zhu, Jingyi Xue, Xili Wan, Chao Liu, Tian Wang, and Yifeng Li. 2022b. Look before you leap: Improving text-based person retrieval by learning a consistent cross-modal common manifold. In ACM MM, pages 1984–1992. find
  55. Donglai Wei, Sipeng Zhang, Tong Yang, and Jing Liu. 2023. Calibrating cross-modal feature for text-based person searching. arXiv preprint arXiv:2304.02278. — [arXiv:2304.02278](https://arxiv.org/abs/2304.02278) find
  56. Yushuang Wu, Zizheng Yan, Xiaoguang Han, Guanbin Li, Changqing Zou, and Shuguang Cui. 2021. Lapscore: language-guided person search via color reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1624– 1633. find
  57. Zequn Xie, Haoming Ji, and Lingwei Meng. 2025. Dynamic uncertainty learning with noisy correspondence for text-based person search. arXiv preprint arXiv:2505.06566. — [arXiv:2505.06566](https://arxiv.org/abs/2505.06566) find
  58. Shuanglin Yan, Neng Dong, Jun Liu, Liyan Zhang, and Jinhui Tang. 2023. Learning comprehensive representations with richer self for text-to-image person re-identification. In Proceedings of the 31st ACM international conference on multimedia, pages 6202– 6211. find
  59. Shuanglin Yan, Neng Dong, Liyan Zhang, and Jinhui Tang. 2022a. Clip-driven fine-grained text-image person re-identification. arXiv:2210.10276. — [arXiv:2210.10276](https://arxiv.org/abs/2210.10276) find
  60. Shuanglin Yan, Hao Tang, Liyan Zhang, and Jinhui Tang. 2022b. Image-specific information suppression and implicit local alignment for text-based person search. arXiv preprint arXiv:2208.14365. — [arXiv:2208.14365](https://arxiv.org/abs/2208.14365) find
  61. Shuyu Yang, Yinan Zhou, Yaxiong Wang, Yujiao Wu, Li Zhu, and Zhedong Zheng. 2023. Towards unified text-based person retrieval: A large-scale multi-attribute and language search benchmark. In Proceedings of the 2023 ACM on Multimedia Conference. find
  62. Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. 2022. Lit: Zero-shot transfer with locked-image text tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18123–18133. find
  63. Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. 2024. Long-clip: Unlocking the long-text capability of clip. In European Conference on Computer Vision, pages 310–325. Springer. find
  64. Ying Zhang and Huchuan Lu. 2018. Deep cross-modal projection learning for image-text matching. In Proceedings of the European conference on computer vision (ECCV), pages 686–701. find
  65. Shizhen Zhao, Changxin Gao, Yuanjie Shao, Wei-Shi Zheng, and Nong Sang. 2021. Weakly supervised text-based person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11395–11404. find
  66. Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, and Yi-Dong Shen. 2020. Dual-path convolutional image-text embeddings with instance loss. ACM Transactions on Multimedia Computing, Communications, and Applications, 16(2):1–23. find
  67. Aichun Zhu, Zijie Wang, Yifeng Li, Xili Wan, Jing Jin, Tian Wang, Fangqiang Hu, and Gang Hua. 2021a. Dssl: Deep surroundings-person separation learning for text-based person retrieval. In Proceedings of the 29th ACM International Conference on Multimedia, pages 209–217. find
  68. Aichun Zhu, Zijie Wang, Yifeng Li, Xili Wan, Jing Jin, Tian Wang, Fangqiang Hu, and Gang Hua. 2021b. Dssl: Deep surroundings-person separation learning for text-based person retrieval. In Proceedings of the 29th ACM International Conference on Multimedia. find
  69. Jialong Zuo, Hanyu Zhou, Ying Nie, Feng Zhang, Tianyu Guo, Nong Sang, Yunhe Wang, and Changxin Gao. 2024. Ufinebench: Towards text-based person retrieval with ultra-fine granularity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22010–22019. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xie-2025-chat-driven-text-generation-interaction/