Elamur Research⌕
☰
home / papers / 2026 / MENTOR: Efficient Autoregressive Image Generation with Balanced Multimodal Control

MENTOR: Efficient Autoregressive Image Generation with Balanced Multimodal Control

Paper · 2026 · Haozhe Zhao, Zefan Cai, Shuzheng Si et al.

Summary

MENTOR targets controllable image generation from combined inputs, such as a reference image plus a text prompt, under tight compute budgets. Existing systems often lean too heavily on one modality, usually the image. They also depend on extra alignment modules and very large training runs. MENTOR uses an autoregressive design. Frozen pre-trained vision and language encoders feed a single shared embedding through a simple MLP connector. A transformer decoder then predicts VQGAN image tokens one after another, so no cross-attention adapters are needed.

Training happens in two phases:

  • Alignment: reconstruction, segmentation and text-to-image tasks teach the model to attend to both visual detail and meaning.
  • Instruction tuning: recovery from distorted inputs and subject-driven generation push the model to weigh text and image evenly.

The training data is a roughly 3M-sample mix of public and synthetic material. On DreamBench and DreamBench++, MENTOR is competitive with much larger systems while using over ten times fewer resources. This suggests a practical, low-cost route to multimodal image control.

Key points

  • The architecture has three parts: frozen vision and language encoders, an MLP connector into the language embedding space, and an autoregressive decoder that shares the VQGAN vocabulary and is trained with next-token prediction.
  • Stage 1 (Multimodal Alignment Tuning) combines image reconstruction, object segmentation and T2I generation. Pairing segmentation with reconstruction keeps the model from simply copying its input.
  • Stage 2 (Multimodal Instruction Tuning) keeps the T2I and segmentation tasks. It adds image recovery from rotated, resized or re-composited inputs, plus subject-driven generation.
  • The training set has about 3 million samples, drawn from CC12M, Midjourney-Niji and synthetic images generated with Flux.1 from LLM-written prompts.
  • MENTOR is evaluated on DreamBench and DreamBench++ and reportedly beats baselines built on SDXL, SD3 and Infinity with over 10× less compute. The authors list remaining weaknesses in spatial reasoning, object counting, human rendering and stylization.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 811 works

Cite

Cite as Haozhe Zhao, Zefan Cai, Shuzheng Si, Liang Chen, Jiuxiang Gu, Wen Xiao, Minjia Zhang and Junjie Hu. 2026. MENTOR: Efficient Autoregressive Image Generation with Balanced Multimodal Control. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1508

@inproceedings{zhao2026mentor,
  title = {{MENTOR: Efficient Autoregressive Image Generation with Balanced Multimodal Control}},
  author = {Haozhe Zhao, Zefan Cai, Shuzheng Si, Liang Chen, Jiuxiang Gu, Wen Xiao, Minjia Zhang and Junjie Hu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1508},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1508}
}

References

88 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, and 1 others. 2023. Improving image generation with better captions. Computer Science. find
  2. BlackForestLabs. 2024. Announcing black forest labs. find
  3. Zefan Cai, Haoyi Qiu, Tianyi Ma, Haozhe Zhao, Gengze Zhou, Kung-Hsiang Huang, Parisa Kordjamshidi, Minjia Zhang, Wen Xiao, Jiuxiang Gu, Nanyun Peng, and Junjie Hu. 2025. Mmgr: Multi-modal generative reasoning. Preprint, arXiv:2512.14691. — [arXiv:2512.14691](https://arxiv.org/abs/2512.14691) find
  4. Bing Cao, Baoshuo Cai, Changqing Zhang, and Qinghua Hu. 2025. Dig2dig: Dig into diffusion information gains for image fusion. Preprint, arXiv:2503.18627. — [arXiv:2503.18627](https://arxiv.org/abs/2503.18627) find
  5. Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 3558–3568. find
  6. Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, Le Xue, Caiming Xiong, and Ran Xu. 2025a. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset. Preprint, arXiv:2505.09568. find
  7. Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and 1 others. 2023a. PixArt-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426.
  8. Liang Chen, Shuai Bai, Wenhao Chai, Weichu Xie, Haozhe Zhao, Leon Vinci, Junyang Lin, and Baobao Chang. 2025b. Multimodal representation alignment for image generation: Text-image interleaved control is easier than you think. Preprint, arXiv:2502.20172. — [arXiv:2502.20172](https://arxiv.org/abs/2502.20172) find
  9. Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, and William W. Cohen. 2023b. Subject-driven text-to-image generation via apprenticeship learning. Preprint, arXiv:2304.00186. — [arXiv:2304.00186](https://arxiv.org/abs/2304.00186) find
  10. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, and 16 others. 2022.… find
  11. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3290 other…
  12. Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2024. GenEval: An object-focused framework for evaluating text-to-image alignment. In NeurIPS. find
  13. Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. 2025. Emerging properties in unified multimodal pretraining. Preprint, arXiv:2505.14683. — [arXiv:2505.14683](https://arxiv.org/abs/2505.14683) find
  14. Yucheng Han, Rui Wang, Chi Zhang, Juntao Hu, Pei Cheng, Bin Fu, and Hanwang Zhang. 2024b. Emma: Your text-to-image diffusion model can secretly accept multi-modal prompts. Preprint, arXiv:2406.09162. — [arXiv:2406.09162](https://arxiv.org/abs/2406.09162) find
  15. Runpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, Xiangwen Kong, Xiangyu Zhang, Kaisheng Ma, and Li Yi. 2023. Dreamllm: Synergistic multimodal comprehension and creation. arXiv preprint arXiv: 2309.11499. — [arXiv:2309.11499](https://arxiv.org/abs/2309.11499) find
  16. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In NeurIPS. find
  17. Caption Emporium. 2024. midjourney-niji-1m-llavanext. https://huggingface.co/datasets/CaptionEmporium/ conceptual-captions-cc12m-llavanext. ↗
  18. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, and 1 others. 2024. Scaling rectified flow transformers for high-resolution image synthesis. find
  19. Patrick Esser, Robin Rombach, and Björn Ommer. 2020. Taming transformers for high-resolution image synthesis. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12868– 12878. find
  20. Mark Everingham and John Winn. 2011. The pascal visual object classes challenge 2012 (voc2012) development kit. Pattern Analysis, Statistical Modelling and Computational Learning, Tech. Rep, 8(5):2–5. find
  21. Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2024. Eva-02: A visual representation for neon genesis. Image and Vision Computing, 149:105171. find
  22. Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. 2022. An image is worth one word: Personalizing text-to-image generation using textual inversion. Preprint, arXiv:2208.01618. find
  23. Yuying Ge, Yixiao Ge, Ziyun Zeng, Xintao Wang, and Ying Shan. 2023. Planting a seed of vision in large language model. arXiv preprint arXiv:2307.08041. find
  24. Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. 2024. SEED-X: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396. — [arXiv:2404.14396](https://arxiv.org/abs/2404.14396) find
  25. Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. 2024a. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. Preprint, arXiv:2412.04431. — [arXiv:2412.04431](https://arxiv.org/abs/2412.04431) find
  26. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. 2023. Segment anything. ICCV.
  27. Jing Yu Koh, Daniel Fried, and Ruslan Salakhutdinov. 2023. Generating images with multimodal language models. NeurIPS. find
  28. Boyi Li, Kilian Q. Weinberger, Serge Belongie, Vladlen Koltun, and René Ranftl. 2022. Language-driven semantic segmentation. Preprint, arXiv:2201.03546. find
  29. Dongxu Li, Junnan Li, and Steven C. H. Hoi. 2023a. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. Preprint, arXiv:2305.14720. — [arXiv:2305.14720](https://arxiv.org/abs/2305.14720) find
  30. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. ArXiv preprint, abs/2301.12597.
  31. Wei Li, Xue Xu, Jiachen Liu, and Xinyan Xiao. 2024. UNIMO-G: Unified image generation through multimodal conditional diffusion. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6173–6188, Bangkok, Thailand. Association for Comp… find
  32. Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. 2024a. World model on million-length video and language with blockwise ringattention. Preprint, arXiv:2402.08268.
  33. Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. 2024b. World model on million-length video and language with ringattention. arXiv preprint arXiv:2402.08268.
  34. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. ArXiv preprint, abs/2304.08485.
  35. Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. 2023. Unified-io 2: Scaling autoregressive multimodal models with vision, language, audio, and action. Preprint, arXiv:2312.17172. — [arXiv:2312.17172](https://arxiv.org/abs/2312.17172) find
  36. Zhendong Mao, Mengqi Huang, Fei Ding, Mingcong Liu, Qian He, and Yongdong Zhang. 2024. Realcustom++: Representing images as real-word for real-time customization. Preprint, arXiv:2408.09744. — [arXiv:2408.09744](https://arxiv.org/abs/2408.09744) find
  37. Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. 2022. Sdedit: Guided image synthesis and editing with stochastic differential equations. Preprint, arXiv:2108.01073. find
  38. Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. 2014. The role of context for object detection and semantic segmentation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 891–8… find
  39. OpenAI. 2024. hello-gpt-4o. find
  40. Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. 2024. Kosmos-g: Generating images in context with multimodal large language models. Preprint, arXiv:2310.02992. — [arXiv:2310.02992](https://arxiv.org/abs/2310.02992) find
  41. Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. 2025. Dreambench++: A human-aligned benchmark for personalized image generation. Preprint, arXiv:2406.16855. — [arXiv:2406.16855](https://arxiv.org/abs/2406.16855) find
  42. Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2024. SDXL: Improving latent diffusion models for high-resolution image synthesis. In ICLR.
  43. Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffusion models for high-resolution image synthesis. Preprint, arXiv:2307.01952.
  44. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machin…
  45. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125.
  46. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. Preprint, arXiv:2102.12092.
  47. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022a. High-resolution image synthesis with latent diffusion models. In CVPR.
  48. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022b. High-resolution image synthesis with latent diffusion models. Preprint, arXiv:2112.10752.
  49. Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. Preprint, arXiv:2208.12242. find
  50. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  51. Shuzheng Si, Haozhe Zhao, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Bofei Gao, Kangyang Luo, Wenhao Li, Yufei Huang, Gang Chen, and 1 others. 2026. Teaching large language models to maintain contextual faithfulness via synthetic tasks and reinforcement learning. In Proceedings of the AAAI Conference on A… find
  52. Shuzheng Si, Haozhe Zhao, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, and Maosong Sun. 2025. A goal without a plan is just a wish: Efficient and effective global planner training for long-horizon agent tasks. arXiv preprint arXiv:2510.05608.
  53. Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024a. Autoregressive model beats diffusion: Llama for scalable image generation. Preprint, arXiv:2406.06525. — [arXiv:2406.06525](https://arxiv.org/abs/2406.06525) find
  54. Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024b. Autoregressive model beats diffusion: LLaMA for scalable image generation. arXiv preprint arXiv:2406.06525. — [arXiv:2406.06525](https://arxiv.org/abs/2406.06525) find
  55. Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. 2024c. Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14398– 14409. find
  56. Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. 2023. find
  57. Generative pretraining in multimodality. Preprint, arXiv:2307.05222. — [arXiv:2307.05222](https://arxiv.org/abs/2307.05222) find
  58. Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. 2025. Ominicontrol: Minimal and universal control for diffusion transformer. Preprint, arXiv:2411.15098. — [arXiv:2411.15098](https://arxiv.org/abs/2411.15098) find
  59. Chameleon Team. 2024a. Chameleon: Mixed-modal early-fusion foundation models. Preprint, arXiv:2405.09818.
  60. Chameleon Team. 2024b. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818.
  61. Meituan LongCat Team, Bin Xiao, Chao Wang, Chengjiang Li, Chi Zhang, Chong Peng, Hang Yu, Hao Yang, Haonan Yan, Haoze Sun, Haozhe Zhao, Hong Liu, Hui Su, Jiaqi Zhang, Jiawei Wang, Jing Li, Kefeng Zhang, Manyuan Zhang, Minhao Jing, and 70 others. 2026. Longcat-next: Lexicalizing modalities as discret… — [arXiv:2603.27538](https://arxiv.org/abs/2603.27538) find
  62. Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. TRL: Transformer Reinforcement Learning. find
  63. Jia Wang, Jie Hu, Xiaoqi Ma, Hanghang Ma, Xiaoming Wei, and Enhua Wu. 2025. Image editing with diffusion models: A survey. Preprint, arXiv:2504.13226. — [arXiv:2504.13226](https://arxiv.org/abs/2504.13226) find
  64. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024a. Qwen2-vl: Enhancing vision-language model’s perception of the…
  65. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, and 6 others. 2024b. Emu3: Next-token prediction is all you need. Prepri… — [arXiv:2409.18869](https://arxiv.org/abs/2409.18869) find
  66. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, and 1 others. 2024c. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869. — [arXiv:2409.18869](https://arxiv.org/abs/2409.18869) find
  67. Cong Wei, Zheyang Xiong, Weiming Ren, Xinrun Du, Ge Zhang, and Wenhu Chen. 2024. Omniedit: Building image editing generalist models through specialist supervision. arXiv preprint arXiv:2411.07199. — [arXiv:2411.07199](https://arxiv.org/abs/2411.07199) find
  68. Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, and 20 others. 2025a. Qwen-image technical report. Preprint, arXiv:25…
  69. Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, and Ping Luo. 2024a. Janus: Decoupling visual encoding for unified multimodal understanding and generation. Preprint, arXiv:2410.13848.
  70. Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, and 1 others. 2024b. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848.
  71. Huimin Wu, Xiaojian Ma, Haozhe Zhao, Yanpeng Zhao, and Qing Li. 2025b. Nep: Autoregressive image editing via next editing token prediction. Preprint, arXiv:2508.06044. — [arXiv:2508.06044](https://arxiv.org/abs/2508.06044) find
  72. Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He. 2025c. Less-to-more generalization: Unlocking more controllability by in-context generation. arXiv preprint arXiv:2504.02160. — [arXiv:2504.02160](https://arxiv.org/abs/2504.02160) find
  73. Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, Song Han, and Yao Lu. 2024c. Vila-u: a unified foundation model integrating visual understanding and generation. Preprint, arXiv:2409.04429. — [arXiv:2409.04429](https://arxiv.org/abs/2409.04429) find
  74. Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. 2024a. Omnigen: Unified image generation. Preprint, arXiv:2409.11340. — [arXiv:2409.11340](https://arxiv.org/abs/2409.11340) find
  75. Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Shuting Wang, Tiejun Huang, and Zheng Liu. 2024b. Omnigen: Unified image generation. Preprint, arXiv:2409.11340. — [arXiv:2409.11340](https://arxiv.org/abs/2409.11340) find
  76. Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528. — [arXiv:2408.12528](https://arxiv.org/abs/2408.12528) find
  77. Rongchang Xie, Chen Du, Ping Song, and Chang Liu. 2025. Muse-vl: Modeling unified vlm through semantic discrete encoding. Preprint, arXiv:2411.17762. — [arXiv:2411.17762](https://arxiv.org/abs/2411.17762) find
  78. Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. 2023. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. find
  79. Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. 2025. Imgedit: A unified image editing dataset and benchmark. Preprint, arXiv:2505.20275. — [arXiv:2505.20275](https://arxiv.org/abs/2505.20275) find
  80. Shihao Yuan, Yahui Liu, Yang Yue, Jingyuan Zhang, Wangmeng Zuo, Qi Wang, Fuzheng Zhang, and Guorui Zhou. 2025. Ar-grpo: Training autoregressive image generation models via reinforcement learning. Preprint, arXiv:2508.06924. — [arXiv:2508.06924](https://arxiv.org/abs/2508.06924) find
  81. Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yugang Jiang, and Xipeng Qiu. 2024. Anygpt: Unified multimodal llm with discrete sequence modeling. ArXiv, abs/2402.12226.
  82. Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding conditional control to text-to-image diffusion models. find
  83. Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, and Baobao Chang. 2023. Mmicl: Empowering vision-language model with multi-modal in-context learning. ArXiv preprint, abs/2309.07915.
  84. Haozhe Zhao, Xiaojian Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. 2024. Ultraedit: Instruction-based fine-grained image editing at scale. Preprint, arXiv:2407.05282. — [arXiv:2407.05282](https://arxiv.org/abs/2407.05282) find
  85. Haozhe Zhao, Shuzheng Si, Liang Chen, Yichi Zhang, Maosong Sun, Baobao Chang, and Minjia Zhang. 2025. Looking beyond text: Reducing language bias in large vision-language models via multimodal dual-attention and soft-image guidance. In Proceedings of the 2025 Conference on Empirical Methods in Natur… find
  86. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2017. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 633–641. find
  87. Xianwei Zhuang, Yuxin Xie, Yufan Deng, Dongchao Yang, Liming Liang, Jinghan Ru, Yuguo Yin, and Yuexian Zou. 2025. Vargpt-v1.1: Improve visual autoregressive large unified model via iterative instruction tuning and reinforcement learning. Preprint, arXiv:2504.02949. — [arXiv:2504.02949](https://arxiv.org/abs/2504.02949) find
  88. Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang, Wenze Liu, Lirui Zhao, Fu-Yun Wang, Zhanyu Ma, and 1 others. 2024. Lumina-Next: Making Lumina-T2X stronger and faster with Next-DiT. arXiv preprint arXiv:2406.18583. — [arXiv:2406.18583](https://arxiv.org/abs/2406.18583) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-mentor-efficient-autoregressive-image-generation/