Elamur Research⌕
☰
home / papers / 2026 / Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs

Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs

Paper · 2026 · Mingze Xu, Zijing Zhao, Qiming Peng et al.

Summary

Evaluating multimodal large language models (MLLMs) usually comes down to a single overall score or a leaderboard position. Such scores hide which specific skills a model has or lacks. They also suffer from "Outcome Bias": judges look only at the final answer, so lucky guesses get credit and sound reasoning with a small slip gets punished.

The authors propose ATOM (AnaTomy Of MLLM), an MLLM-as-a-judge framework built for diagnosis rather than ranking. It works in four steps:

  • Cognitive Replay. The judge first solves each image–query pair itself, acting as an "Ideal Solver."
  • Visual Anchoring. It splits the task into weighted Atomic Criteria, each tied to visible evidence in the image.
  • Dynamic Capability Tree. These criteria are organized into a taxonomy that grows from the data, so failures can be traced to specific perception, knowledge or reasoning skills.
  • Feedback loop. Low-scoring criteria become feedback that helps the evaluated model fix its own answer.

On a new benchmark with human rankings of multiple model responses, ATOM agrees with human judgments better than existing judges. Its feedback also lets models correct many errors without retraining. This matters because it turns evaluation into actionable guidance for improving models.

Key points

  • Each criterion is a sub-question with a local ground truth and an importance weight (1–10). Each criterion is scored 1–5, and the sample score is the weighted average of these.
  • ATOM beats the strongest baseline judge in accuracy by up to 7.92%. It was validated across 18 open- and closed-source models.
  • The feedback loop fixes up to 39.95% of previously failed queries. It raises performance by up to 26.85% with no parameter updates, which suggests many errors come from attention drift rather than missing ability.
  • Human meta-evaluation used 12 annotators with graduate-level training.
  • Limitations:
    • ATOM covers only single-turn tasks.
    • It depends on the strength of the judge model.
    • The capability tree may need re-evolving for very different domains.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓Tsinghua Shenzhen International Graduate SchoolShenzhen · China29 works · 3,035 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Mingze Xu, Zijing Zhao, Qiming Peng, Houwen Peng, Han Hu, Zhanhui Kang and Yuxing Han. 2026. Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.932

@inproceedings{xu2026beyond,
  title = {{Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs}},
  author = {Mingze Xu, Zijing Zhao, Qiming Peng, Houwen Peng, Han Hu, Zhanhui Kang and Yuxing Han},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.932},
  url = {https://doi.org/10.18653/v1/2026.acl-long.932}
}

References

86 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohammed Talha Alam, Raza Imam, Mohsen Guizani, and Fakhri Karray. 2024. Flare up your data: Diffusion-based augmentation method in astronomical imaging. arXiv preprint arXiv:2405.13267. — [arXiv:2405.13267](https://arxiv.org/abs/2405.13267) find
  2. Lukas Blecher. 2022. Latex-ocr. GitHub. https://github.com/lukas-blecher/LaTeX-OCR. ↗
  3. Rumeysa Bodur, Erhan Gundogdu, Binod Bhattarai, Tae-Kyun Kim, Michael Donoser, and Loris Bazzani. 2024. iedit: Localised text-guided image editing with weak supervision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7426–7435. find
  4. Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324– 345. find
  5. Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. 2022. Webqa: Multihop and multimodal qa. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16495–16504. find
  6. Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. 2024. Mllm-asa-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. In Forty-first International Conference on Machine Learning. find
  7. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E Gonzalez, and others. 2024. Chatbot arena: An open platform for evaluating llms by human preference. arXiv preprint arXiv:2403.04132.
  8. Romrawin Chumpu. Multimodal neural translation. https://huggingface.co/datasets/romrawinjp/multi30k. ↗
  9. Jasper Dekoninck, Maximilian Baader, and Martin Vechev. 2024. Polyrating: A cost-effective and bias-aware rating system for llm evaluation. arXiv preprint arXiv:2409.00696. — [arXiv:2409.00696](https://arxiv.org/abs/2409.00696) find
  10. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 708–717. find
  11. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and others. 2023. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394.
  12. Noa Garcia, Chentao Ye, Zihua Liu, Qingtao Hu, Mayu Otani, Chenhui Chu, Yuta Nakashima, and Teruko Mitamura. 2020. A dataset and baselines for visual question answering on art. In European conference on computer vision, pages 92–108. Springer. find
  13. Wentao Ge, Shunian Chen, Hardy Chen, Nuo Chen, Junying Chen, Zhihong Chen, Wenya Xie, Shuo Yan, ChenghaoZhu ChenghaoZhu, Ziyue Lin, and others. 2025. Mllm-bench: evaluating multimodal llms with per-sample criteria. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the A… find
  14. Tanmay Gupta, Dustin Schwenk, Ali Farhadi, Derek Hoiem, and Aniruddha Kembhavi. 2018. Imagine this! scripts to compositions to videos. In Proceedings of the European conference on computer vision (ECCV), pages 598–613. find
  15. Xintong Han, Zuxuan Wu, Phoenix X Huang, Xiao Zhang, Menglong Zhu, Yuan Li, Yang Zhao, and Larry S Davis. 2017. Automatic spatially-aware fashion concept discovery. In Proceedings of the IEEE international conference on computer vision, pages 1463–1471. find
  16. Zhen Han, Chaojie Mao, Zeyinzi Jiang, Yulin Pan, and Jingfeng Zhang. 2025. Stylebooth: Image style editing with multimodal instruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1947–1957. find
  17. Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. Trueskill™: a bayesian skill rating system. Advances in neural information processing systems, 19. find
  18. Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. 2024. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475. find
  19. Meng-Ru Hsieh, Yen-Liang Lin, and Winston H Hsu. 2017. Drone-based object counting by spatially regularized regional proposal network. In Proceedings of the IEEE international conference on computer vision, pages 4145–4153. find
  20. Hao Fei, Yuan Zhou, Juncheng Li, Xiangtai Li, Qingshan Xu, Bobo Li, Shengqiong Wu, Yaoting Wang, Junbao Zhou, Jiahao Meng, and others. 2025. On path to multimodal generalist: General-level and general-bench. In Forty-second International Conference on Machine Learning. find
  21. Drew A Hudson and Christopher D Manning. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709.
  22. Maxwell Forbes, Christine Kaeser-Chen, Piyush Sharma, and Serge Belongie. 2019. Neural naturalist: Generating fine-grained image comparisons. find
  23. Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daume, and Larry S Davis. 2017. The amazing mysteries of the gutter: Drawing inferences between panels in comic book narratives. In Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pages… find
  24. Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2901–291…
  25. Kaggle. 2019a. American sign language dataset. https://www.kaggle.com/datasets/ayuraj/asl-d ataset. ↗
  26. Kaggle. 2019b. C-nmc 2019 dataset. https://www.ka ggle.com/datasets/shafiullahshafin/c-nmc -2019-dataset. ↗
  27. Kaggle. 2020a. https://www.kaggle.com/dataset s/andrewmvd/car-plate-detection. ↗
  28. Kaggle. 2020b. Emotion detection (fer). https://www.kaggle.com/datasets/ananthu017/emotion-d etection-fer. ↗
  29. Kaggle. 2020c. Instagram images with captions. https://www.kaggle.com/datasets/prithvijaunj ale/instagram-images-with-captions. ↗
  30. Kaggle. 2021a. https://www.kaggle.com/dataset s/misrakahmed/vegetable-image-dataset. ↗
  31. Kaggle. 2021b. Famous iconic women. https://www.kaggle.com/datasets/fatiimaezzahra/famou s-iconic-women. ↗
  32. Kaggle. 2021c. Standard ocr dataset. https://www.ka ggle.com/datasets/preatcher/standard-ocr -dataset. ↗
  33. Kaggle. 2022a. Micro-organism image classification. https://www.kaggle.com/datasets/mdwaquar azam/microorganism-image-classification. ↗
  34. Kaggle. 2022b. Sketch2code. https://www.kaggle.com/datasets/vshantam/sketch2code. ↗
  35. Kaggle. 2022c. Yoga pose classification. https://ww w.kaggle.com/datasets/ujjwalchowdhury/yo ga-pose-classification. find
  36. Kaggle. 2023a. 100 sports image classification. https://www.kaggle.com/datasets/gpiosenka/spor ts-classification. ↗
  37. Kaggle. 2023b. Animals-10. https://www.kaggle.com/datasets/alessiocorrado99/animals10. ↗
  38. Kaggle. 2023c. Dog breeds. https://www.kaggle.com/datasets/mohamedchahed/dog-breeds. ↗
  39. Kaggle. 2023d. Facial emotion recognition image dataset. https://www.kaggle.com/datasets/su jaykapadnis/emotion-recognition-dataset. ↗
  40. Kaggle. 2023e. Medical visual question answering. https://www.kaggle.com/datasets/mitanshuch akrawarty/medical-visual-question-answeri ng. ↗
  41. Kaggle. 2023f. Seeds counting. https://www.kaggle.com/datasets/raj123verma/seeds-counting. ↗
  42. Kaggle. 2023g. Strawberry dataset. https://www.ka ggle.com/datasets/abdulbasit31/strawberr y-dataset. ↗
  43. Kaggle. 2024a. Pumpkin leaf diseases dataset from bangladesh. https://www.kaggle.com/dataset s/tahmidmir/pumpkin-leaf-diseases-dataset -from-bangladesh. ↗
  44. Kaggle. 2024b. Radar threat object classification. https://www.kaggle.com/datasets/rauldbcet/ra dar-threat-object-classification. ↗
  45. Kaggle. 2024c. Terrain recognition dataset. https://www.kaggle.com/datasets/krishuppal/ter rain-recognition. ↗
  46. Kaggle. 2025a. Ecg images, national heart foundation of bangladesh. https://www.kaggle.com/datas ets/drsaeedmohsen/ecg-images-national-hea rt-foundation-of-banglades. ↗
  47. Kaggle. 2025b. Guava fruit disease dataset. https://www.kaggle.com/datasets/asadullahgalib/guava-disease-dataset. ↗
  48. Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pages 4… find
  49. Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and others. 2023. Prometheus: Inducing fine-grained evaluation capability in language models. In The Twelfth International Conference on Learning Representations.
  50. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, and others. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(…
  51. Jiayi Kuang, Ying Shen, Jingyou Xie, Haohao Luo, Zhe Xu, Ronghao Li, Yinghui Li, Xianfeng Cheng, Xika Lin, and Yu Han. 2025. Natural language understanding and inference with mllm in visual question answering: A survey. ACM Computing Surveys, 57(8):1–36. find
  52. Yann LeCun. Digital consistency comparison (mnist). https://huggingface.co/datasets/ylecun/m nist. ↗
  53. Seongyun Lee, Seungone Kim, Sue Park, Geewook Kim, and Minjoon Seo. 2024. Prometheus-vision: Vision-language model as a judge for fine-grained evaluation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 11286–11315. find
  54. Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. 2023. Generative judge for evaluating alignment. arXiv preprint arXiv:2310.05470.
  55. Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. 2024. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. arXiv preprint arXiv:2406.11939.
  56. Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. 2014. Deepreid: Deep filter pairing neural network for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 152–159. find
  57. Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David Carlson, and Jianfeng Gao. 2019. Storygan: A sequential conditional gan for story visualization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6329–6338. find
  58. Yongqi Li, Wenjie Li, and Liqiang Nie. 2022. Mmcoqa: Conversational question answering over text, tables, and images. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4220–4231. find
  59. Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, and Yejin Choi. 2024. Wildbench: Benchmarking llms with challenging tasks from real users in the wild. arXiv preprint arXiv:2406.04770.
  60. Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, and Xirong Li. 2025. Phd: A chatgpt-prompted visual hallucination evaluation dataset. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19857–19866. find
  61. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and others. 2024. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233. Springer.
  62. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255.
  63. Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in neural information processing systems, 35:2507–2521. find
  64. Xiaoqiang Lu, Binqiang Wang, Xiangtao Zheng, and Xuelong Li. 2017. Exploring models and data for remote sensing image caption generation. IEEE Transactions on Geoscience and Remote Sensing, 56(4):2183–2195. find
  65. Adyasha Maharana, Darryl Hannan, and Mohit Bansal. 2022. Storydall-e: Adapting pretrained text-to-image transformers for story continuation. In European conference on computer vision, pages 70–87. Springer. find
  66. Shu Pu, Yaochen Wang, Dongping Chen, Yuhang Chen, Guohao Wang, Qi Qin, Zhongyi Zhang, Zhiyuan Zhang, Zetong Zhou, Shuang Gong, and others. 2025. Judge anything: Mllm as a judge across any modality. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 57… find
  67. Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S Yu. 2025. A survey of multilingual large language models. Patterns, 6(1). find
  68. Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. 2023. Verbosity bias in preference labeling by large language models. arXiv preprint arXiv:2310.10076. — [arXiv:2310.10076](https://arxiv.org/abs/2310.10076) find
  69. Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, page…
  70. Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. 2023. Hierarchical multimodal transformers for mul-tipage docvqa. Pattern Recognition, 144:109834. find
  71. Junjue Wang, Zhuo Zheng, Zihang Chen, Ailong Ma, and Yanfei Zhong. 2024. Earthvqa: Towards queryable earth via relational reasoning-based remote sensing visual question answering. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 5481–5489. find
  72. Zongqi Wang, Tianle Gu, Chen Gong, Xin Tian, Siqi Bao, and Yujiu Yang. 2025. Scan: Structured capability assessment and navigation for llms. arXiv preprint arXiv:2505.06698.
  73. Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis. 2018. Recipeqa: A challenge dataset for multimodal comprehension of cooking recipes. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1358–1368. find
  74. Zhiyuan Yan, Taiping Yao, Shen Chen, Yandan Zhao, Xinghe Fu, Junwei Zhu, Donghao Luo, Chengjie Wang, Shouhong Ding, Yunsheng Wu, and others. 2024. Df40: Toward next-generation deepfake detection. Advances in Neural Information Processing Systems, 37:29387–29434. find
  75. Yu Zhao, Jianguo Wei, Zhichao Lin, Yueheng Sun, Meishan Zhang, and Min Zhang. 2022. Visual spatial description: Controlled spatial-oriented image-to-text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1437–1449. find
  76. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  77. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  78. Yanbin Yin, Kun Zhou, Zhen Wang, Xiangdong Zhang, Yifei Shao, Shibo Hao, Yi Gu, Jieyuan Liu, Somanshu Singla, Tianyang Liu, and others. 2025. Decentralized arena: Towards democratic and scalable automatic evaluation of language models. arXiv preprint arXiv:2505.12808.
  79. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. Judgelm: Fine-tuned large language models are scalable judges. arXiv preprint arXiv:2310.17631.
  80. Kaining Ying, Fanqing Meng, Jin Wang, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, and others. 2024. Mmt-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi. arXiv preprint arXiv:2404.16006. — [arXiv:2404.16006](https://arxiv.org/abs/2404.16006) find
  81. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and others. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF conference on computer visi…
  82. Zhiyuan Zeng, Yizhong Wang, Hannaneh Hajishirzi, and Pang Wei Koh. 2025. Evaltree: Profiling language model weaknesses via hierarchical capability trees. arXiv preprint arXiv:2503.08893. — [arXiv:2503.08893](https://arxiv.org/abs/2503.08893) find
  83. Chenhao Zhang, Xi Feng, Yuelin Bai, Xeron Du, Jinchang Hou, Kaixin Deng, Guangzeng Han, Qinrui Li, Bingli Wang, Jiaheng Liu, and others. 2025a. Can mllms understand the deep implication behind chinese images? In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics… find
  84. Qihui Zhang, Munan Ning, Zheyuan Liu, Yue Huang, Shuo Yang, Yanbo Wang, Jiayi Ye, Xiao Chen, Yibing Song, and Li Yuan. 2025b. Upme: An unsupervised peer review framework for multimodal large language model evaluation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 91… find
  85. Ruochen Zhao, Wenxuan Zhang, Yew Ken Chia, Weiwen Xu, Deli Zhao, and Lidong Bing. 2025. Auto-arena: Automating llm evaluations with agent peer battles and committee discussions. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages…
  86. Cap. Perc. Reas. Know. Und. Trust. Gen. 2.73 2.72 2.61 2.64 2.52 2.69 Nec. 2.87 2.84 2.80 2.78 2.77 2.56 Acc. 2.64 2.65 2.61 2.62 2.62 2.33 Nec. 2.85 2.81 2.66 2.73 2.70 2.71 Acc. 2.82 2.86 2.84 2.85 2.81 2.58 Nec. 2.74 2.72 2.70 2.70 2.58 2.59 Acc. 2.81 2.69 2.62 2.69 2.64 2.55 Nec. 2.83 2.63 2.62… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-beyond-ranking-fine-grained-diagnostics/