Elamur Research⌕
☰
home / papers / 2026 / CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring

CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring

Paper · 2026 · Jiamin Su, Yibo Yan, Zhuoran Gao et al.

Summary

Multimodal writing tasks ask students to argue from images such as charts. Scoring them automatically is hard. Classic AES systems depend on handcrafted features and cannot process images. Single-MLLM scorers tend to invent justifications and grade more harshly than human raters. CAFES addresses this by dividing scoring among three agents:

  • Initial Scorer: a weaker "student" MLLM gives preliminary 0–5 scores for ten fine-grained traits.
  • Feedback Pool Manager: a stronger "teacher" MLLM collects trait-level comments, deliberately focusing on the essay's strengths to offset model harshness.
  • Reflective Scorer: the same teacher decides whether to revise each score based on that feedback and gives a reason.

The authors test the framework on EssayJudge with eight student MLLMs and GPT-4o as teacher. Average QWK improves by about 21% relative to initial scores, with the largest gains in grammatical and lexical diversity. Ablations show that both the feedback pool and the separation of student and teacher roles matter. The work points toward more human-aligned multimodal essay assessment.

Key points

  • Dataset: EssayJudge, described as the only public multimodal AES dataset. It has 1,054 university-level essays built on visuals such as line charts and flow charts, covering 125 topics.
  • Traits: ten traits at three levels.
    • Lexical: LA, LD.
    • Sentence: CH, GA, GD, PA.
    • Discourse: AC, JP, OS, EL.
  • Feedback pool ablation: letting the model revise scores without the feedback pool lowered QWK below the initial scores for both Claude-3.5-Sonnet and Qwen2.5-VL-32B.
  • Same-model ablation: using one model as both student and teacher hurt results. With GPT-4o in both roles, average QWK fell from 0.54 to 0.49.
  • Limitations: the evaluation covers only chart-based essays, and hallucination-driven scoring errors are reduced but still occur.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

 Fields, topics and concepts

Cite

Cite as Jiamin Su, Yibo Yan, Zhuoran Gao, Han Zhang, Xiang Liu, Huiyu Zhou and Xuming Hu. 2026. CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring. Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.alvr-main.10

@inproceedings{su2026cafes,
  title = {{CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring}},
  author = {Jiamin Su, Yibo Yan, Zhuoran Gao, Han Zhang, Xiang Liu, Huiyu Zhou and Xuming Hu},
  booktitle = {Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR)},
  year = {2026},
  doi = {10.18653/v1/2026.alvr-main.10},
  url = {https://doi.org/10.18653/v1/2026.alvr-main.10}
}

References

76 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2024. Claude 3.5 sonnet. find
  2. John Atkinson and Diego Palma. 2025. An llm-based hybrid approach for enhanced automated essay scoring. Scientific Reports, 15(1):14551. find
  3. Yigal Attali and Jill Burstein. 2006. Automated essay scoring with e-rater® v.2. journal of technology, learning, and assessment, 4(3). Journal of Technology, Learning, and Assessment, 4. find
  4. Stacey Bailey and Detmar Meurers. 2008. Diagnosing meaning errors in short answers to reading comprehension questions. In Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, pages 107–115. find
  5. Yida Cai, Kun Liang, Sanwoo Lee, Qinghan Wang, and Yunfang Wu. 2025. Rank-then-score: Enhancing large language models for automated essay scoring. arXiv preprint arXiv:2504.05736. — [arXiv:2504.05736](https://arxiv.org/abs/2504.05736) find
  6. Yue Cao, Hanqi Jin, Xiaojun Wan, and Zhiwei Yu. 2020. Domain-adaptive neural automated essay scoring. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 1011–1020. Association for Computing Machinery. find
  7. Hongbo Chen and Ben He. 2013. Automated essay scoring by maximizing human-machine agreement. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1741–1752. find
  8. Junkai Chen, Zhijie Deng, Kening Zheng, Yibo Yan, Shuliang Liu, PeiJun Wu, Peijie Jiang, Jia Liu, and Xuming Hu. 2025a. Safeeraser: Enhancing safety in multimodal large language models through multimodal machine unlearning. arXiv preprint arXiv:2502.12520. find
  9. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, and 23 others. 2025b. Expanding performance boundaries of open-source mult…
  10. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, and 23 others. 2025c. Expanding performance boundaries of open-source mult…
  11. Jaeyoon Choi, Tamara Tate, Daniel Ritchie, Nia Nixon, and Mark Warschauer. 2025. Anchor is the key: Toward accessible automated essay scoring with large language models through prompting. find
  12. Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, and 1 others. 2025. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733. find
  13. Mădălina Cozma, Andrei Butnaru, and Radu Tudor Ionescu. 2018. Automated essay scoring with string kernels and word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 503–509. find
  14. Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, and 1 others. 2024. Explainable and interpretable multimodal large language models: A comprehensive survey. arXiv preprint arXiv:2412.02104. — [arXiv:2412.02104](https://arxiv.org/abs/2412.02104) find
  15. Google DeepMind. 2025. Gemini 2.5 flash. find
  16. Heejin Do, Sangwon Ryu, and Gary Lee. 2024. Autoregressive multi-trait essay scoring via reinforcement learning with scoring-aware multiple rewards. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA. Association for Computational Linguisti… find
  17. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  18. Sylviane Granger, Estelle Dagneaux, Fanny Meunier, and Magali Paquot. 2009. International Corpus of Learner English. Version 2. Handbook and CD-ROM. find
  19. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, and 6 others. 2024. Minicpm: Unveiling the potential of small lan…
  20. Kaichen Huang, Jiahao Huo, Yibo Yan, Kun Wang, Yutao Yue, and Xuming Hu. 2024. Miner: Mining the underlying pattern of modality-specific neurons in multimodal large language models. arXiv preprint arXiv:2410.04819. — [arXiv:2410.04819](https://arxiv.org/abs/2410.04819) find
  21. Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, and Xuming Hu. 2024. Mmneuron: Discovering neuron-level domain-specific interpretation in multimodal large language model. arXiv preprint arXiv:2406.11193. find
  22. Jiahao Huo, Yibo Yan, Xu Zheng, Yuanhuiyi Lyu, Xin Zou, Zhihua Wei, and Xuming Hu. 2025. Mmunlearner: Reformulating multimodal machine unlearning in the era of multimodal large language models. arXiv preprint arXiv:2502.11051. find
  23. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  24. Zhiwei Jiang, Tianyi Gao, Yafeng Yin, Meng Liu, Hua Yu, Zifeng Cheng, and Qing Gu. 2023. Improving domain generalization for prompt-aware essay scoring via disentangled representation learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… find
  25. Firuz Kamalov, David Santandreu Calonge, Linda Smail, Dilshod Azizov, Dimple R Thadani, Theresa Kwong, and Amara Atif. 2025. Evolution of ai in education: Agentic workflows. arXiv preprint arXiv:2504.20082. — [arXiv:2504.20082](https://arxiv.org/abs/2504.20082) find
  26. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.
  27. Zixuan Ke and Vincent Ng. 2019. Automated essay scoring: A survey of the state of the art. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 6300–6308. International Joint Conferences on Artificial Intelligence Organization. find
  28. Anindita Kundu and Denilson Barbosa. 2024. Are large language models good essay graders? find
  29. Sanwoo Lee, Yida Cai, Desong Meng, Ziyang Wang, and Yunfang Wu. 2024a. Unleashing large language models’ proficiency in zero-shot essay scoring. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 181–198. find
  30. Sanwoo Lee, Yida Cai, Desong Meng, Ziyang Wang, and Yunfang Wu. 2024b. Unleashing large language models’ proficiency in zero-shot essay scoring. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 181–198, Miami, Florida, USA. Association for Computational Linguistics. find
  31. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for "mind" exploration of large language model society. Preprint, arXiv:2303.17760.
  32. Hang Li, Tianlong Xu, Chaoli Zhang, Eason Chen, Jing Liang, Xing Fan, Haoyang Li, Jiliang Tang, and Qingsong Wen. 2024. Bringing generative ai to adaptive learning in education. arXiv preprint arXiv:2402.14601. — [arXiv:2402.14601](https://arxiv.org/abs/2402.14601) find
  33. Shengjie Li and Vincent Ng. 2024a. Automated essay scoring: A reflection on the state of the art. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17876– 17888. find
  34. Shengjie Li and Vincent Ng. 2024b. Automated essay scoring: Recent successes and future directions. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pages 8114–8122. find
  35. OpenAI. 2024. Gpt-4o mini: advancing cost-efficient intelligence. find
  36. Shengjie Li and Vincent Ng. 2024c. Icle++: Modeling fine-grained traits for holistic essay scoring. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8458–8478. find
  37. Dadi Ramesh and Suresh Kumar Sanampudi. 2022. An automated essay scoring systems: a systematic literature review. Artificial Intelligence Review, 55(3):2495–2527. find
  38. Wenchao Li and Haitao Liu. 2024. Applying large language models for automated essay scoring for non-native japanese. Humanities and Social Sciences Communications, 11(1):1–15. find
  39. Xia Li and Wenjing Pan. 2025. CEAES: Bidirectional reinforcement learning optimization for consistent and explainable essay assessment. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26267–26279, Vienna, Austria. Association… find
  40. SeungWoo Song, Junghun Yuk, ChangSu Choi, HanGyeol Yoo, Hyeonseok Lim, KyungTae Lim, and Jungyeul Park. 2025. Unified automated essay scoring and grammatical error correction. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 4412–4426. find
  41. Yishen Song, Qianta Zhu, Huaibo Wang, and Qinhua Zheng. 2024. Automated essay scoring and revising based on open-source large language models. IEEE Transactions on Learning Technologies. find
  42. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
  43. Shalev Lifshitz, Sheila A McIlraith, and Yilun Du. 2025. Multi-agent verification: Scaling test-time compute with multiple verifiers. arXiv preprint arXiv:2502.20379. — [arXiv:2502.20379](https://arxiv.org/abs/2502.20379) find
  44. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024a. Llava-next: Improved reasoning, ocr, and world knowledge. find
  45. Zeyang Liu, Xinrui Yang, Shiguang Sun, Long Qian, Lipeng Wan, Xingyu Chen, and Xuguang Lan. 2024b. Grounded answers for multi-agent decision-making problem through generative world model. Advances in Neural Information Processing Systems, 37:46622– 46652. find
  46. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, and 1 others. 2024. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525.
  47. Watheq Mansour, Salam Albatarni, Sohaila Eltanbouly, and Tamer Elsayed. 2024. Can large language models automatically score proficiency of written essays? Preprint, arXiv:2403.06149. find
  48. Christian Stab and Iryna Gurevych. 2014. Annotating argument components and relations in persuasive essays. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1501–1510.
  49. Jiamin Su, Yibo Yan, Fangteng Fu, Han Zhang, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, and Xuming Hu. 2025. Essayjudge: A multi-granular benchmark for assessing automated essay scoring capabilities of multimodal large language models. find
  50. Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D. Nguyen. 2025. Multi-agent collaboration mechanisms: A survey of llms. find
  51. Masaki Uto, Yikuan Xie, and Maomi Ueno. 2020. Neural automated essay scoring incorporating hand-crafted features. In Proceedings of the 28th international conference on computational linguistics, pages 6077–6088. find
  52. Sowmya Vajjala. 2016. Automated assessment of non-native learner essays: Investigating the role of linguistic features. CoRR. find
  53. Jiong Wang and Jie Liu. 2025. T-MES: Trait-aware mix-of-experts representation learning for multi-trait essay scoring. In Proceedings of the 31st International Conference on Computational Linguistics. Association for Computational Linguistics. find
  54. Sandeep Mathias and Pushpak Bhattacharyya. 2018. ASAP++: Enriching the ASAP automated essay grading dataset with essay attribute scores. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  55. Yongjie Wang, Chuang Wang, Ruobing Li, and Hui Lin. 2022. On the use of bert for automated essay scoring: Joint learning of multi-scale essay representation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  56. Atsushi Mizumoto and Masaki Eguchi. 2023. Exploring the potential of using an ai language model for automated essay scoring. Research Methods in Applied Linguistics, 2(2):100050. find
  57. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation. Preprint, arXiv:2308.08155.
  58. Xuansheng Wu, Padmaja Pravin Saraf, Gyeong-Geon Lee, Ehsan Latif, Ninghao Liu, and Xiaoming Zhai. 2024. Unveiling scoring processes: Dissecting the differences between llms and human graders in automatic scoring. arXiv preprint arXiv:2407.18328. — [arXiv:2407.18328](https://arxiv.org/abs/2407.18328) find
  59. Wei Xia, Shaoguang Mao, and Chanjing Zheng. 2024. Empirical study of large language models as automated essay scoring tools in english composition_taking toefl independent writing task for example. arXiv preprint arXiv:2401.03401. find
  60. Changrong Xiao, Wenxing Ma, Qingping Song, Sean Xin Xu, Kunpeng Zhang, Yufang Wang, and Qi Fu. 2024. Human-ai collaborative essay scoring: A dual-process framework with llms. Preprint, arXiv:2401.06431. — [arXiv:2401.06431](https://arxiv.org/abs/2401.06431) find
  61. Kaixun Yang, Mladen Raković, Yuyang Li, Quanlong Guan, Dragan Gašević, and Guangliang Chen. 2024. Unveiling the tapestry of automated essay scoring: A comprehensive investigation of accuracy, fairness, and generalizability. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 3… find
  62. Helen Yannakoudakis and Ted Briscoe. 2012. Modeling coherence in ESOL learner texts. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, pages 33–43. find
  63. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 180–189.
  64. Wenbo Xu, Muhammad Shahreeza, Wai Lam Hoo, and Wudao Yang. 2025. Explainable ai for education: Enhancing essay scoring via rubric-aligned chain-of-thought prompting. find
  65. Jingheng Ye, Shen Wang, Deqing Zou, Yibo Yan, Kun Wang, Hai-Tao Zheng, Zenglin Xu, Irwin King, Philip S Yu, and Qingsong Wen. 2025. Position: Llms can be good tutors in foreign language education. arXiv preprint arXiv:2502.05467. — [arXiv:2502.05467](https://arxiv.org/abs/2502.05467) find
  66. Yibo Yan and Joey Lee. 2024. Georeasoner: Reasoning on geospatially grounded context for natural language understanding. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 4163–4167. find
  67. Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, and 1 others. 2024. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652.
  68. Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, and Xuming Hu. 2024a. A survey of mathematical reasoning in the era of multimodal large language model: Benchmark, method & challenges. arXiv preprint arXiv:2412.11936. find
  69. Zhiqiang Yuan, Weitong Chen, Hanlin Wang, Kai Yu, Xin Peng, and Yiling Lou. 2024. Transagent: An llm-based multi-agent system for code translation. arXiv preprint arXiv:2409.19894. — [arXiv:2409.19894](https://arxiv.org/abs/2409.19894) find
  70. Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, Yi-Fan Zhang, Tianlong Xu, Zhendong Chu, and 1 others. 2024b. Errorradar: Benchmarking complex mathematical reasoning of multimodal large language models via error detection. arXiv preprint arXiv:2410.04509.
  71. Yibo Yan, Shen Wang, Jiahao Huo, Jingheng Ye, Zhendong Chu, Xuming Hu, Philip S Yu, Carla Gomes, Bart Selman, and Qingsong Wen. 2025a. Position: Multimodal large language models can significantly advance scientific reasoning. arXiv preprint arXiv:2502.02871. find
  72. Yibo Yan, Shen Wang, Jiahao Huo, Philip S Yu, Xuming Hu, and Qingsong Wen. 2025b. Mathagent: Leveraging a mixture-of-math-agent framework for real-world multimodal mathematical error detection. arXiv preprint arXiv:2503.18132. find
  73. Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, and Xuming Hu. 2024. Reefknot: A comprehensive benchmark for relation hallucination evaluation, analysis and mitigation in multimodal large language models. arXiv preprint arXiv:2408.09429. find
  74. Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, and Xuming Hu. 2024. Mitigating modality prior-induced hallucinations in multimodal large language models via deciphering attention causality. arXiv preprint arXiv:2410.04780. — [arXiv:2410.04780](https://arxiv.org/abs/2410.04780) find
  75. Xin Zou, Yizhou Wang, Yibo Yan, Sirui Huang, Kening Zheng, Junkai Chen, Chang Tang, and Xuming Hu. 2024. Look twice before you answer: Memory-space visual retracing for hallucination mitigation in multimodal large language models. arXiv preprint arXiv:2410.03577. — [arXiv:2410.03577](https://arxiv.org/abs/2410.03577) find
  76. Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, and Yuxuan Liang. 2024c. Urbanclip: Learning text-enhanced urban region profiling with contrastive language-image pretraining from the web. In Proceedings of the ACM on Web Conference 2024, pages 4006–4017. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/su-2026-cafes-collaborative-multi-agent-framework/