Elamur Research⌕
☰
home / papers / 2026 / Bloom-Eval: A Hierarchical Evaluation Benchmark for Automatic Survey Generation Based on Bloom's Taxonomy

Bloom-Eval: A Hierarchical Evaluation Benchmark for Automatic Survey Generation Based on Bloom's Taxonomy

Paper · 2026 · Fei Zhang, Zhe Zhao, HaiBin Wen et al.

Summary

Evaluating systems for automatic survey generation (ASG) has lagged behind the systems themselves. Current metrics mostly check low-level skills such as reference overlap or citation correctness, and they often depend on opaque "LLM-as-a-Judge" scores. Bloom-Eval addresses both problems with a benchmark organised around the six levels of Bloom's Taxonomy: Memory, Comprehension, Application, Analysis, Evaluation and Creation. Each level has its own metrics.

Reliability comes from a dual-constraint design. Wherever a property can be measured, the benchmark uses deterministic algorithms, and LLMs serve only as tools for extraction or matching. For abstract abilities it uses GRADE, in which the judge model first writes explicit weighted rubrics and then scores a generated survey against a human-written reference, justifying each score in text.

The authors also built a cross-disciplinary corpus of more than 3,000 recent expert papers. Tests on leading ASG systems show they handle structure and formatting well but perform poorly at critical judgment and original synthesis. The benchmark therefore pushes the field to value intellectual depth over surface imitation.

Key points

  • Data: more than 3,000 papers from 2023–2025, including Annual Reviews articles from 51 journal series, grouped into 20 evaluation topics. The benchmark and evaluation scripts are released at github.com/feizhang18/Bloom-Eval.
  • Methods:
    • DS is a composite metric built from three measures of distributional difference: Hellinger Distance, Total Variation Distance and the Jensen-Shannon Divergence. It checks whether a system's emphasis matches the experts'.
    • GRADE handles rubric-based differential scoring for abstract abilities.
    • HIRC measures coverage of high-impact papers, defined as those with more than 50 citations.
  • Results on SurveyX, SurveyForge and LLMxMapReduce-V2:
    • Systems scored near-perfectly on formatting metrics (FMI, DSI).
    • Evaluation-tier scores were about 0.1 (CAA F1 between 0.08 and 0.13).
    • Semantic Tree Similarity ranged only from 0.39 to 0.56.
  • System comparison:
    • SurveyX led Application (0.80) and Comprehension (0.57).
    • LLMxMapReduce-V2 was the only system above the 0.5 human-parity line, on both Framework Application (0.52) and Creation (0.55).
  • Efficiency:
    • SurveyX was the slowest system (126 min).
    • LLMxMapReduce-V2 consumed the most tokens (over 72k).
    • SurveyForge produced output close to human length (about 26k tokens) while writing roughly 40× faster than people.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States506 works · 330,052 citations🎓Princeton UniversityPrinceton · United States113 works · 48,612 citations🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🎓City University of Hong KongHong Kong · China59 works · 3,480 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksInformation Systems · 1,730 worksEducation · 212 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 works

Cite

Cite as Fei Zhang, Zhe Zhao, HaiBin Wen, Tianshuo Wei, Zaixi Zhang, Chao Yang and Ye Wei. 2026. Bloom-Eval: A Hierarchical Evaluation Benchmark for Automatic Survey Generation Based on Bloom's Taxonomy. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1315

@inproceedings{zhang2026bloom,
  title = {{Bloom-Eval: A Hierarchical Evaluation Benchmark for Automatic Survey Generation Based on Bloom's Taxonomy}},
  author = {Fei Zhang, Zhe Zhao, HaiBin Wen, Tianshuo Wei, Zaixi Zhang, Chao Yang and Ye Wei},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1315},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1315}
}

References

43 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lorin W Anderson and David R Krathwohl. 2001. A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives: complete edition. Addison Wesley Longman, Inc. find
  2. Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, and Yejin Choi. 2018. Discourse-aware neural rewards for coherent text generation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… find
  3. Yapei Chang, Kyle Lo, Tanya Goyal, and Mohit Iyyer. 2024. Booookscore: A systematic exploration of book-length summarization in the era of llms. In ICLR.
  4. Hui Chen, Miao Xiong, Yujie Lu, Wei Han, Ailin Deng, Yufei He, Jiaying Wu, Yibo Li, Yue Liu, and Bryan Hooi. 2025. Mlr-bench: Evaluating ai agents on open-ended machine learning research. In NeurIPS. find
  5. Jingqiang Chen and Hai Zhuge. 2019. Automatic generation of related work through summarizing citations. Concurrency and Computation: Practice and Experience. find
  6. Woon Sang Cho, Pengchuan Zhang, Yizhe Zhang, Xiujun Li, Michel Galley, Chris Brockett, Mengdi Wang, and Jianfeng Gao. 2019. Towards coherent and cohesive long-form text generation. In Proceedings of the First Workshop on Narrative Understanding. find
  7. Meri Coleman and T. L. Liau. 1975. A computer readability formula designed for machine scoring. Journal of Applied Psychology. find
  8. Hen-Hsen Huang. 2021. Autosurvey: Automatic survey generation based on a research draft. In IJCAI. find
  9. Rahul Jha, Catherine Finegan-Dollak, Ben King, Reed Coke, and Dragomir Radev. 2015. Content models for survey generation: a factoid-based evaluation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Langu… find
  10. Haozhe Ji, Pei Ke, Zhipeng Hu, Rongsheng Zhang, and Minlie Huang. 2023. Tailoring language generation models under total variation distance. In ICLR. find
  11. Hanqi Jin, Tianming Wang, and Xiaojun Wan. 2020. Multi-granularity interaction network for extractive and abstractive multi-document summarization. In ACL. find
  12. Josue Balandrano Coronel. 2016. University of texas rio grande valley trec liveqa 2016: Using topic modeling to answer complex questions. In TREC. find
  13. J. Peter Kincaid, Jr. Fishburne, Robert P., Richard L. Rogers, and Brad S. Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Technical report, Naval Technical Training Command, Millington, TN. find
  14. Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, and Zhendong Mao. 2025. Deepresearch bench: A comprehensive benchmark for deep research agents. arXiv preprint arXiv:2506.11763.
  15. Wojciech Kryscinski, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong, and Dragomir Radev. 2022. BOOKSUM: A collection of datasets for long-form narrative summarization. In EMNLP. find
  16. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
  17. Rémi Lebret and Ronan Collobert. 2014. Word embeddings through hellinger pca. In EACL. find
  18. Corrado Gini. 1912. Variabilità e mutabilità: contributo allo studio delle distribuzioni e delle relazioni statistiche.[Fasc. I.]. Tipogr. di P. Cuppini. find
  19. Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794.
  20. Robert Gunning. 1952. The technique of clear writing. McGraw-Hill. find
  21. Beichen Guo, Zhiyuan Wen, Yu Yang, Peng Gao, Ruosong Yang, and Jiaxing Shen. 2025. Sgsimeval: A comprehensive multifaceted and similarity-enhanced benchmark for automatic survey generation systems. In ADMA. find
  22. Cong Duy Vu Hoang and Min-Yen Kan. 2010. Towards automated related work summarization. In Coling. find
  23. Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xinxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, and 1 others. 2025. Chain of ideas: Revolutionizing research in novel idea development with llm agents. In EMNLP. find
  24. Xun Liang, Jiawei Yang, Yezhaohui Wang, Chen Tang, Zifan Zheng, Shichao Song, Zehao Lin, Yebin Yang, Simin Niu, Hanyu Wang, Bo Tang, Feiyu Xiong, Keming Mao, and Zhiyu li. 2025. Surveyx: Academic survey automation via large language models. arXiv preprint arXiv:2502.14776. — [arXiv:2502.14776](https://arxiv.org/abs/2502.14776) find
  25. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292.
  26. Yue Hu and Xiaojun Wan. 2014. Automatic generation of related work sections in scientific papers: an optimization approach. In EMNLP. find
  27. Yao Lu, Yue Dong, and Laurent Charlin. 2020. Multi-xscience: A large-scale dataset for extreme multi-document summarization of scientific articles. In EMNLP.
  28. Yuntong Hu, Zhuofeng Li, Zheng Zhang, Chen Ling, Raasikh Kanjiani, Boxin Zhao, and Liang Zhao. 2024. Hireview: Hierarchical taxonomy-driven automatic literature review generation. arXiv preprint arXiv:2410.03761.
  29. Ziming Mao, Chen Henry Wu, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed Awadallah, and Dragomir Radev. 2022. DYLE: Dynamic latent extraction for abstractive long-input summarization. In ACL. find
  30. Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. 2025. Nomic embed: Training a reproducible long context text embedder. Transactions on Machine Learning Research.
  31. Bowen Zhou. 2025. Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). find
  32. Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. 2024. Assisting in writing wikipedia-like articles from scratch with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
  33. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In NeurIPS.
  34. Xiaoping Sun and Hai Zhuge. 2019. Automatic generation of survey paper based on template tree. In 2019 15th International Conference on Semantics, Knowledge and Grids (SKG). find
  35. Kun Zhu, Xiaocheng Feng, Xiachong Feng, Yingsheng Wu, and Bing Qin. 2023. Hierarchical catalogue generation for literature review: A benchmark. arXiv preprint arXiv:2304.03512. find
  36. Haoyu Wang, Yujia Fu, Zhu Zhang, Shuo Wang, Zirui Ren, Xiaorong Wang, Zhili Li, Chaoqun He, Bo An, Zhiyuan Liu, and Maosong Sun. 2025. Llm×mapreduce-v2: Entropy-driven convolutional test-time scaling for generating long-form articles from extremely long resources. arXiv preprint arXiv:2504.05732. — [arXiv:2504.05732](https://arxiv.org/abs/2504.05732) find
  37. A Qingyun Wang, Lifu Huang, Zhiying Jiang, Kevin Knight, Heng Ji, Mohit Bansal, and Yi Luan. 2019. Paperrobot: Incremental draft generation of scientific ideas. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. find
  38. Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, Wei Ye, Shikun Zhang, and Yue Zhang. 2024a. Autosurvey: Large language models can automatically write surveys. In NeurIPS.
  39. Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. 2024b. Pandalm: An automatic evaluation benchmark for llm instruction tuning optimization. In ICLR.
  40. Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. 2025. Writingbench: A comprehensive benchmark for generative writing. In NeurIPS. find
  41. Xinyu Xing, Xiaosheng Fan, and Xiaojun Wan. 2020. Automatic generation of citation texts in scholarly papers: A pilot study. In ACL. find
  42. Xiangchao Yan, Shiyang Feng, Jiakang Yuan, Renqiu Xia, Bin Wang, Bo Zhang, and Lei Bai. 2025. Surveyforge: On the outline heuristics, memory-driven generation, and multi-dimensional evaluation for automated survey writing. In Proceedings of the 63rd Annual Meeting of the Association for Computationa…
  43. Jiakang Yuan, Xiangchao Yan, Botian Shi, Tao Chen, Wanli Ouyang, Bo Zhang, Lei Bai, Yu Qiao, and Implementation Details of Evaluation Methodologies This appendix presents the mathematical formulations and implementation details of the evaluation methodologies introduced in the Bloom-Eval benchmark.… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-bloom-eval-hierarchical-evaluation-benchmark/