Elamur Research⌕
☰
home / papers / 2026 / Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge

Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge

Paper · 2026 · Junjie Wu, Xuan Kan, Zihao He et al.

Summary

The paper addresses a weakness of multimodal LLM judges: most are trained for a single evaluation task and transfer poorly to other tasks or input formats. Supervised fine-tuning (SFT) judges, in particular, tend to latch onto prompt templates. The authors, from Meta AI with HKUST and Emory, propose MT-RL-Judge. It trains one judge on several pointwise image-evaluation datasets at once using GRPO. The reward combines answer correctness with a check that the model reasons before giving its verdict. The backbone is Qwen3-VL-30B-A3B-Instruct.

On six benchmarks, the single multi-task judge scores at or near the top on every task. It matches or beats single-task specialists, which means one model can replace several and simplifies deployment. The strongest evidence comes from MJ-Bench, which uses a pairwise comparison format never seen in training. There the RL judge holds up while the multi-task SFT judge collapses. This suggests RL-driven reasoning learns transferable evaluation criteria, which matters for production pipelines where requirements change often.

Key points

  • Training data: six binary judging tasks covering image quality, image-text alignment and safety: AGIN-Nat, AGIN-Tech, AGIN-Rat, SeeTrue, UnsafeBench and ImageReward. Each has roughly 4.8k–7.3k training examples.
  • Reward: R_total = (1−α)·R_acc + α·R_format, where both components are binary. Training uses GRPO with group-averaged rewards, so no value model is needed.
  • In-domain results (Macro-F1): MT-RL-Judge scores best on SeeTrue (83.67) and AGIN-Tech (81.37).
    • It is close behind on the remaining tasks: AGIN-Nat 81.63 vs 81.75 for SFT-Unified, and AGIN-Rat 81.58 vs 82.71 for RL-Single.
    • SFT-Single stays ahead on UnsafeBench (90.28 vs 85.22).
  • Unified training helps even with SFT: SFT-Unified beats SFT-Single on most tasks, for example AGIN-Nat 81.75 vs 78.64.
  • Out-of-distribution test (MJ-Bench, pairwise format):
    • On safety, MT-RL-Judge reaches 82.23, while SFT-Unified drops to 49.40, below the off-the-shelf model's 73.07.
    • On alignment, MT-RL-Judge scores 60.59, compared with 55.82 for SFT-Unified and 59.41 for the off-the-shelf model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Emory UniversityAtlanta · United States42 works · 39,899 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksVision & language · 811 works

Cite

Cite as Junjie Wu, Xuan Kan, Zihao He, Shunwen Tan, Bo Pan and Kaitai Zhang. 2026. Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.126

@inproceedings{wu2026multi,
  title = {{Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge}},
  author = {Junjie Wu, Xuan Kan, Zihao He, Shunwen Tan, Bo Pan and Kaitai Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.126},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.126}
}

References

25 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025. Qwen3-vl technical report. arXiv prepri…
  2. Chaoyou Fu, Yi-Fan Zhang, Shukang Yin, Bo Li, Xinyu Fang, Sirui Zhao, Haodong Duan, Xing Sun, Ziwei Liu, Liang Wang, and 1 others. 2024. Mme-survey: A comprehensive survey on evaluation of multimodal llms. arXiv preprint arXiv:2411.15296. — [arXiv:2411.15296](https://arxiv.org/abs/2411.15296) find
  3. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, and 1 others. 2024. A survey on llm-as-a-judge. The Innovation. find
  4. Tiancheng Gu, Kaicheng Yang, Kaichen Zhang, Xiang An, Ziyong Feng, Yueyi Zhang, Weidong Cai, Jiankang Deng, and Lidong Bing. 2025. Unime-v2: Mllm-as-a-judge for universal multimodal embedding learning. arXiv preprint arXiv:2510.13515. — [arXiv:2510.13515](https://arxiv.org/abs/2510.13515) find
  5. Jongwoo Ko, Sungnyun Kim, Sungwoo Cho, and Se-Young Yun. 2025. Flex-judge: Text-only reasoning unleashes zero-shot multimodal evaluators. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  6. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  7. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: Nlg evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522.
  8. Reuben A Luera, Ryan Rossi, Franck Dernoncourt, Samyadeep Basu, Sungchul Kim, Subhojyoti Mukherjee, Puneet Mathur, Ruiyi Zhang, Jihyung Kil, Nedim Lipka, and 1 others. 2025. Mllm as a ui judge: Benchmarking multimodal llms for predicting human perception of user interfaces. arXiv preprint arXiv:2510… — [arXiv:2510.08783](https://arxiv.org/abs/2510.08783) find
  9. Bo Pan, Xuan Kan, Kaitai Zhang, Yan Yan, Shunwen Tan, Zihao He, Zixin Ding, Junjie Wu, and Liang Zhao. 2026. Bi-level prompt optimization for multimodal llm-as-a-judge. arXiv preprint arXiv:2602.11340. — [arXiv:2602.11340](https://arxiv.org/abs/2602.11340) find
  10. Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. 2024a. Mllm-asa-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. In Forty-first International Conference on Machine Learning. find
  11. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  12. Zhaorun Chen, Yichao Du, Zichen Wen, Yiyang Zhou, Chenhang Cui, Zhenzhen Weng, Haoqin Tu, Chaoqi Wang, Zhengwei Tong, Qinglan Huang, and 1 others. 2024b. Mj-bench: Is your multimodal reward model really a good judge for text-to-image generation? arXiv preprint arXiv:2407.04842. — [arXiv:2407.04842](https://arxiv.org/abs/2407.04842) find
  13. Renjie Pi, Haoping Bai, Qibin Chen, Xiaoming Simon Wang, Jiulong Shan, Xiaojiang Liu, and Meng Cao. 2025. Mr. judge: Multimodal reasoner as a judge. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20192–20216. find
  14. Zijian Chen, Wei Sun, Haoning Wu, Zicheng Zhang, Jun Jia, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Xiongkuo Min, Guangtao Zhai, and 1 others. 2023. Exploring the naturalness of ai-generated images. arXiv preprint arXiv:2312.05476. — [arXiv:2312.05476](https://arxiv.org/abs/2312.05476) find
  15. Shu Pu, Yaochen Wang, Dongping Chen, Yuhang Chen, Guohao Wang, Qi Qin, Zhongyi Zhang, Zhiyuan Zhang, Zetong Zhou, Shuang Gong, and 1 others. 2025. Judge anything: Mllm as a judge across any modality. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages… find
  16. Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, and Yang Zhang. 2025. Unsafebench: Benchmarking image safety classifiers on real-world and ai-generated images. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, pages 3221–3235. find
  17. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  18. Zhenting Wang, Shuming Hu, Shiyu Zhao, Xiaowen Lin, Felix Juefei-Xu, Zhuowei Li, Ligong Han, Harihar Subramanyam, Li Chen, Jianfa Chen, and 1 others. 2025. Mllm-as-a-judge for image safety without human labeling. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 14657–1… find
  19. Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. 2023. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903– 15935. find
  20. Shenzhi Wang Zhangchi Feng Dongdong Kuang Yuwen Xiong Yaowei Zheng, Junting Lu. 2025. Easyr1: An efficient, scalable, multi-modality rl training framework. https://github.com/hiyouga/ EasyR1. ↗
  21. Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. 2023. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems, 36:1601–1619. find
  22. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  23. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  24. Yaowei Zheng, Richong Zhang, Junhao Zhang, YeYanhan YeYanhan, and Zheyan Luo. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 400–410.
  25. Yue Zhou, Yi Chang, and Yuan Wu. 2025. Conf-probench: A confidence evaluation benchmark for mllm-based process judges. arXiv:2508.04576. arXiv preprint — [arXiv:2508.04576](https://arxiv.org/abs/2508.04576) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-multi-task-reinforcement-learning-enhanced/