Elamur Research⌕
☰
home / papers / 2026 / From Words to Pixels: A Comprehensive Survey on Large Language Models in Visual Segmentation

From Words to Pixels: A Comprehensive Survey on Large Language Models in Visual Segmentation

Paper · 2026 · Yizhou Wang, Mang Tik Chiu, Lingzhi Zhang et al.

Summary

This survey covers an emerging area: driving pixel-level visual segmentation from natural-language instructions with language-based foundation models, namely LLMs (Large Language Models) together with their multimodal counterparts, LMMs (Large Multimodal Models). Older segmentation systems depend on fixed label sets and have trouble connecting free-form, compositional language to masks. The authors claim this is the first broad review of LLM-based approaches. They group existing methods by task formulation and by the role the language model plays. They also analyse architectural patterns, including how LLMs are connected to vision backbones, which fusion strategies are used, and which mask decoders are chosen. The survey covers datasets, metrics and reported benchmark results on referring expression segmentation and ReasonSeg. A qualitative study of failure cases shows where current systems still break down. The main conclusion is that progress is real but incomplete: reasoning benchmarks remain unsolved, and pixel-annotated data for complex instructions is scarce. A maintained resource list accompanies the paper, so it serves as a practical entry point for researchers working where NLP meets vision.

Key points

  • Taxonomy with five categories:
    • reasoning segmentation (LISA, Seg-Zero, CoReS)
    • pixel grounding & RES (GroundHog, GLaMM)
    • open-vocabulary segmentation (Mask-adapted CLIP, SegPrompt)
    • video segmentation (VISA, VideoGLaMM)
    • LLM-enhanced architectures (LLaFS, OMG-LLaVA)
  • LISA as the template for later reasoning models: a special <SEG> token embedding is projected through an MLP to guide a SAM-style mask decoder.
  • Effect of model size on ReasonSeg test: averaged across methods, 13B models outperform 7B models by +7.5 gIoU (63.0 vs 55.5) and +4.7 cIoU (62.5 vs 57.8).
  • Current performance and failure modes: results on ReasonSeg stay under 70% IoU. LISA-13B and READ-13B repeatedly fail on three kinds of target:
    • very small regions
    • irregularly shaped regions (nets, food mixtures)
    • regions that require spatial reasoning
  • Open problems and resource: the authors identify a shortage of large datasets with pixel masks for multi-step reasoning, and call for better interpretability and more unified models. The resource list is published as the GitHub repository Awesome-LLM-Visual-Segmentation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yizhou Wang, Mang Tik Chiu, Lingzhi Zhang, Xuan Shen, Sohrab Amirghodsi and Yun Fu. 2026. From Words to Pixels: A Comprehensive Survey on Large Language Models in Visual Segmentation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2155

@inproceedings{wang2026words,
  title = {{From Words to Pixels: A Comprehensive Survey on Large Language Models in Visual Segmentation}},
  author = {Yizhou Wang, Mang Tik Chiu, Lingzhi Zhang, Xuan Shen, Sohrab Amirghodsi and Yun Fu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2155},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2155}
}

References

80 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ali Athar, Xueqing Deng, and Liang-Chieh Chen. 2025. Vicas: A dataset for combining holistic and pixel-level video understanding using captions with grounded segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19023–19035. find
  2. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 1(2):3.
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — Qwen2.5-VL Technical Report
  4. Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Zheng Zhang, and Mike Zheng Shou. 2024. One token to seg them all: Language instructed reasoning segmentation in videos. Advances in Neural Information Processing Systems, 37:6833–6859. find
  5. Xiaoyi Bao, Siyang Sun, Shuailei Ma, Kecheng Zheng, Yuxin Guo, Guosheng Zhao, Yun Zheng, and Xingang Wang. 2024. Cores: Orchestrating the dance of reasoning and segmentation. In European Conference on Computer Vision, pages 187–204. Springer. find
  6. Hritam Basak and Zhaozheng Yin. 2025. Semidavil: Semi-supervised domain adaptation with vision-language guidance for semantic segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9816–9828. find
  7. Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, and 1 others. 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297.
  8. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4):834–848. find
  9. Yi-Chia Chen, Wei-Hua Li, Cheng Sun, Yu-Chiang Frank Wang, and Chu-Song Chen. 2024. Sam4mllm: Enhance multi-modal large language model for referring expression segmentation. In European Conference on Computer Vision, pages 323– 340. Springer. find
  10. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. find
  11. Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 32… find
  12. Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and 1 others. 2020. The epic-kitchens dataset: Collection, challenges and baselines. IEEE Transactions on Pattern Analysis and Machine… find
  13. Henghui Ding, Chang Liu, and Xudong Jiang. 2021. Vision-language transformer and query generation for referring segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16321–16330. find
  14. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107–1128, Miami…
  15. Mark Everingham, Luc Gool, Christopher K. Williams, John Winn, and Andrew Zisserman. 2010. The pascal visual object classes (voc) challenge. Int. J. Comput. Vision, 88(2):303–338. find
  16. Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, and Shuicheng Yan. 2024. Vitron: A unified pixel-level vision llm for understanding, generating, segmenting, editing. arXiv preprint arXiv:2412.19806. — [arXiv:2412.19806](https://arxiv.org/abs/2412.19806) find
  17. Sitong Gong, Yunzhi Zhuge, Lu Zhang, Zongxin Yang, Pingping Zhang, and Huchuan Lu. 2025. The devil is in temporal token: High quality video reasoning segmentation. arXiv preprint arXiv:2501.08549. — [arXiv:2501.08549](https://arxiv.org/abs/2501.08549) find
  18. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969. find
  19. Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. 2015. Activitynet: A large-scale video benchmark for human activity understanding. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 961–970. find
  20. Ziling Huang and Shin’ichi Satoh. 2023. Referring image segmentation via joint mask contextual embedding learning and progressive alignment network. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7753–7762. find
  21. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages 4904–4916. PMLR.
  22. Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. 2024. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13700–13710. find
  23. Amin Karimi and Charalambos Poullis. 2025. Dsv-lfs: Unifying llm-driven semantic cues with visual features for robust few-shot segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 4584–4594. find
  24. Seoyeon Kim, Minguk Kang, Dongwon Kim, Jaesik Park, and Suha Kwak. 2023. Extending clip’s image-text alignment to referring image segmentation. arXiv preprint arXiv:2306.08498. find
  25. Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár. 2019. Panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9404–9413. find
  26. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, and 1 others. 2023. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026.
  27. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, and 1 others. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 12…
  28. Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. 2024. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9579–9589. find
  29. Jiahao Li, Yang Lu, Yuan Xie, and Yanyun Qu. 2024. Relationship prompt learning is enough for open-vocabulary semantic segmentation. Advances in Neural Information Processing Systems, 37:74298– 74324. find
  30. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML.
  31. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463.
  32. Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. 2023. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7061–7070. find
  33. Lang Lin, Xueyang Yu, Ziqi Pang, and Yu-Xiong Wang. 2025. Glus: Global-local reasoning unified into a single large language model for video segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. find
  34. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014a. Microsoft coco: Common objects in context. In Computer vision– ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pa…
  35. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014b. Microsoft coco: Common objects in context. In Computer vision– ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pa…
  36. Chang Liu, Henghui Ding, and Xudong Jiang. 2023a. Gres: Generalized referring expression segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23592–23601. find
  37. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024a. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306.
  38. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024b. Llava-next: Improved reasoning, ocr, and world knowledge. find
  39. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023b. Visual instruction tuning. Advances in neural information processing systems, 36:34892– 34916.
  40. Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. 2025a. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520. — [arXiv:2503.06520](https://arxiv.org/abs/2503.06520) find
  41. Yuqi Liu, Tianyuan Qu, Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, and Jiaya Jia. 2025b. Visionreasoner: Unified visual perception and reasoning via reinforcement learning. arXiv preprint arXiv:2505.12081. — [arXiv:2505.12081](https://arxiv.org/abs/2505.12081) find
  42. Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440. find
  43. Gen Luo, Yiyi Zhou, Yunchao Ji, Chunhua Cao, Zhen Li, Haoqiang Fan, and Shiming Xiang. 2020. Multi-task collaborative network for joint referring expression comprehension and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10034–10043. find
  44. Shehan Munasinghe, Hanan Gani, Wenqi Zhu, Jiale Cao, Eric Xing, Fahad Shahbaz Khan, and Salman Khan. 2024. Videoglamm: A large multimodal model for pixel-level visual grounding in videos. arXiv preprint arXiv:2411.04923. — [arXiv:2411.04923](https://arxiv.org/abs/2411.04923) find
  45. Rui Qian, Xin Yin, and Dejing Dou. 2024. Reasoning to attend: Try to understand how\< seg\> token works. arXiv preprint arXiv:2412.17741. — [arXiv:2412.17741](https://arxiv.org/abs/2412.17741) find
  46. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021a. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–…
  47. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021b. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–…
  48. Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. 2024. Glamm: Pixel grounding large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages… find
  49. Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. 2024. Pixellm: Pixel reasoning with large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26374–26383. find
  50. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234…
  51. Yucheng Suo, Linchao Zhu, and Yi Yang. 2023. Text augmented spatial-aware zero-shot referring image segmentation. arXiv preprint arXiv:2310.18049. find
  52. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  53. Junchi Wang and Lei Ke. 2024. Llm-seg: Bridging image segmentation and large language model reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1765–1774. find
  54. Zhaoqing Wang, Chang Liu, Henghui Ding, and Xudong Jiang. 2022. Cris: Clip-driven referring image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11686–11695. find
  55. Cong Wei, Haoxian Tan, Yujie Zhong, Yujiu Yang, and Lin Ma. 2024a. Lasagna: Language-based segmentation assistant for complex queries. arXiv preprint arXiv:2404.08506. — [arXiv:2404.08506](https://arxiv.org/abs/2404.08506) find
  56. Cong Wei, Yujie Zhong, Haoxian Tan, Yong Liu, Zheng Zhao, Jie Hu, and Yujiu Yang. 2024b. Hyperseg: Towards universal visual segmentation with large language model. arXiv preprint arXiv:2411.17606. — [arXiv:2411.17606](https://arxiv.org/abs/2411.17606) find
  57. Chenyun Wu, Zhe Lin, Scott Cohen, Trung Bui, and Subhransu Maji. 2020. Phrasecut: Language-based image segmentation in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10216–10225. find
  58. Tsung-Han Wu, Giscard Biamby, David Chan, Lisa Dunlap, Ritwik Gupta, Xudong Wang, Joseph E Gonzalez, and Trevor Darrell. 2024. See say and segment: Teaching lmms to overcome false premises. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13459–13469. find
  59. Ning Xu, Linjie Yang, Yuchen Fan, Dingcheng Yue, Yuchen Liang, Jianchao Yang, and Thomas Huang. 2018. Youtube-vos: A large-scale video object segmentation benchmark. arXiv preprint arXiv:1809.03327. — [arXiv:1809.03327](https://arxiv.org/abs/1809.03327) find
  60. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, (2).
  61. Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang, Yao Hu, Guoliang Kang, Weidi Xie, and Efstratios Gavves. 2024. Visa: Reasoning video object segmentation via large language models. In European Conference on Computer Vision, pages 98–115. Springer. find
  62. Rongkun Zheng, Lu Qi, Xi Chen, Yi Wang, Kun Wang, Yu Qiao, and Hengshuang Zhao. 2024. Villa: Video reasoning segmentation with large language model. arXiv preprint arXiv:2407.14500. — [arXiv:2407.14500](https://arxiv.org/abs/2407.14500) find
  63. Senqiao Yang, Tianyuan Qu, Xin Lai, Zhuotao Tian, Bohao Peng, Shu Liu, and Jiaya Jia. 2023. Lisa++: An improved baseline for reasoning segmentation with large language model. arXiv preprint arXiv:2312.17240. — [arXiv:2312.17240](https://arxiv.org/abs/2312.17240) find
  64. Zhao Yang, Henghui Ding, Jing Lin, and Xudong Jiang. 2022. Lavt: Language-aware vision transformer for referring image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18155–18165. find
  65. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A survey on multimodal large language models. National Science Review, 11(12):nwae403.
  66. Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016. Modeling context in referring expressions. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85. Springer.
  67. Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024a. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. find
  68. Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. 2024b. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. Advances in Neural Information Processing Systems, 37:71737–71767. find
  69. Xin Zhang and Robby T Tan. 2025. Mamba as a bridge: Where vision foundation models meet vision language models for domain-generalized semantic segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 14527– 14537. find
  70. Yichi Zhang, Ziqiao Ma, Xiaofeng Gao, Suhaila Shakiah, Qiaozi Gao, and Joyce Chai. 2024c. Groundhog: Grounding large language models to holistic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14227–14238. find
  71. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2017. Scene parsing through ade20k dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. find
  72. Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, and Jun Liu. 2024a. Llafs: When large language models meet few-shot segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3065–3075. find
  73. Lanyun Zhu, Tianrun Chen, Qianxiong Xu, Xuanyi Liu, Deyi Ji, Haiyang Wu, De Wen Soh, and Jun Liu. 2025a. Popen: Preference-based optimization and ensemble for lvlm-based reasoning segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 30231–30240. find
  74. Minjie Zhu, Yichen Zhu, Xin Liu, Ning Liu, Zhiyuan Xu, Chaomin Shen, Yaxin Peng, Zhicai Ou, Feifei Feng, and Jian Tang. 2024b. Mipha: A comprehensive overhaul of multimodal assistant with small language models. arXiv preprint arXiv:2403.06199. — [arXiv:2403.06199](https://arxiv.org/abs/2403.06199) find
  75. Muzhi Zhu, Hengtao Li, Hao Chen, Chengxiang Fan, Weian Mao, Chenchen Jing, Yifan Liu, and Chunhua Shen. 2023. Segprompt: Boosting open-world segmentation via category-level prompt learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 999–1008. find
  76. Muzhi Zhu, Yuzhuo Tian, Hao Chen, Chunluan Zhou, Qingpei Guo, Yang Liu, Ming Yang, and Chunhua Shen. 2025b. Segagent: Exploring pixel understanding capabilities in mllms by imitating human annotator trajectories. arXiv preprint arXiv:2503.08625. — [arXiv:2503.08625](https://arxiv.org/abs/2503.08625) find
  77. Xueyan Zou, Xinlong Cheng, Guanglu Song, Yue Su, Yu Zhang, Junnan Li, Jifeng Dai, and Yu Qiao. 2023a. Segment everything everywhere all at once. In Advances in Neural Information Processing Systems. find
  78. Xueyan Zou, Guanglu Song, Yu Zhang, Jifeng Dai, and Yu Qiao. 2023b. Generalized decoding for pixel, image, and language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15135–15145. find
  79. Zheng Zhang, Yeyao Ma, Enming Zhang, and Xiang Bai. 2024d. Psalm: Pixelwise segmentation with large multi-modal model. In European Conference on Computer Vision, pages 74–91. Springer. find
  80. From Words to Pixels: A Comprehensive Survey on Large Language Models in Visual Segmentatin B Limitations of IoU and Complementary Indicators While IoU (gIoU/cIoU) remains the primary mask-quality metric, it does not fully capture correctness for reasoning-heavy tasks where intermediate steps and co… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-words-pixels-comprehensive-survey-large/