Elamur Research⌕
☰
home / papers / 2026 / IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation

IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation

Paper · 2026 · Yinghao Tang, Xueding Liu, Boyuan Zhang et al.

Summary

Modern text-to-image (T2I) models can produce polished visuals, but infographics also demand faithful data encoding and correct text. Generated infographics often look right while hiding errors such as wrong bar heights or garbled labels. IGenBench is presented as the first benchmark for measuring this reliability. The authors gathered about 40K real-world infographics and reduced them through clustering, sampling and quality filtering to 600 diverse cases. For each case they extracted the design intent and the underlying data, with human checks, and turned them into self-contained generation prompts.

Evaluation breaks each output into atomic yes/no checks, using a taxonomy of 10 question types. The checks come from explicit prompt constraints plus expert-chosen dimensions such as data completeness, ordering and encoding. An MLLM judge answers each check, which yields question-level accuracy (Q-ACC) and infographic-level accuracy (I-ACC).

Across 10 T2I models, even the strongest gets fewer than half of its infographics fully correct. Human review therefore remains essential, and data fidelity is the main open challenge.

Key points

  • The benchmark has 600 test cases spanning 30 infographic types, grouped into 6 categories: part-to-whole composition, changes over time (trend and evolution), comparisons between categories, deviations and gaps, correlations and flows, and a bonus group. Only image URLs are released, not the original images.
  • Results fall into three performance tiers:
    • Nanobanana-Pro leads with Q-ACC 0.90, but its I-ACC is only 0.49.
    • Second-tier models score Q-ACC 0.55–0.61.
    • The remaining models stay below 0.48.
  • Data-related dimensions are the weakest point for every model; Data Completeness averages just 0.21.
  • Twelve MLLMs were compared as judges against human ratings. Gemini-2.5-Pro reached a Pearson correlation of 0.90 and was the only one above 0.8, so it was chosen as the evaluator. GLM-4.5v scored 0.75 and GPT-5-mini 0.70.
  • Rankings differ from LMArena: Seedream-4.5 rises to second place, while GPT-Image-1.5 drops to third. This suggests that strong natural-image quality does not guarantee correct infographics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, Luoxuan Weng, Xiuqi Huang, Minfeng Zhu, Yingchaojie Feng, Yuyu Luo and Wei Chen. 2026. IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1713

@inproceedings{tang2026igenbench,
  title = {{IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation}},
  author = {Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, Luoxuan Weng, Xiuqi Huang, Minfeng Zhu, Yingchaojie Feng, Yuyu Luo and Wei Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1713},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1713}
}

References

66 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, and et al. 2024. Pixtral 12b. Preprint, arXiv:2410.07073.
  2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — Qwen2.5-VL Technical Report
  3. Black Forest Labs. 2024. Flux. https://github.com/ black-forest-labs/flux. ↗
  4. ByteDance. 2025. Seedream 4.5. https://seed.bytedance.com/en/seedream4_5. ↗
  5. Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fengbin Gao, Peihan Xu, and 1 others. 2025. Hidream-i1: A high-efficient image generative foundation model with sparse diffusion transformer. arXiv preprint arXiv:2505.22705. — [arXiv:2505.22705](https://arxiv.org/abs/2505.22705) find
  6. Shuo Cao, Nan Ma, Jiayang Li, Xiaohui Li, Lihao Shao, Kaiwen Zhu, Yu Zhou, Yuandong Pu, Jiarui Wu, Jiaquan Wang, Bo Qu, Wenhai Wang, Yu Qiao, Da-juin Yao, and Yihao Liu. 2025. ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding. Preprint, arXiv:2507.1… — [arXiv:2507.14533](https://arxiv.org/abs/2507.14533) find
  7. Jingjing Chang, Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, and Hai-Bao Chen. 2025. OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation. Preprint, arXiv:2506.07977. — [arXiv:2506.07977](https://arxiv.org/abs/2506.07977) find
  8. Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. 2024. Viseval: A benchmark for data visualization in the era of large language models. IEEE Transactions on Visualization and Computer Graphics. find
  9. Yiyu Chen, Yifan Wu, Shuyu Shen, Yupeng Xie, Leixian Shen, Hui Xiong, and Yuyu Luo. 2025. Chartmark: A structured grammar for chart annotation. In 2025 IEEE Visualization and Visual Analytics (VIS), pages 311–315. IEEE. find
  10. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating llms by human preference. Preprint, arXiv:2403.04132.
  11. Gheorghe Comanici and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. Preprint, arXiv:2507.06261.
  12. Weiwei Cui, Xiaoyu Zhang, Yun Wang, He Huang, Bei Chen, Lei Fang, Haidong Zhang, Jian-Guan Lou, and Dongmei Zhang. 2019. Text-to-viz: Automatic generation of infographics from proportion-related natural language statements. IEEE transactions on visualization and computer graphics, 26(1):906–916. find
  13. Anurag Deo, Savita Bhat, and Shirish Karande. Visu-alFusion: Enhancing Blog Content with Advanced Infographic Pipeline. find
  14. Victor Dibia. 2023. LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 113–126, Toronto, Canada. Ass… find
  15. Banu Inanc Uyan Dur. 2014. Data visualization and infographics in visual communication design education at the age of information. Journal of arts and humanities, 3(5):39–50. find
  16. Yingchaojie Feng, Xingbo Wang, Kam Kwai Wong, Sijia Wang, Yuhong Lu, Minfeng Zhu, Baicheng Wang, and Wei Chen. 2023. Promptmagician: Interactive prompt engineering for text-to-image creation. IEEE Transactions on Visualization and Computer Graphics, 30(1):295–305. find
  17. Ferdio. 2024. Data viz project. datavizproject.com/. https:// find
  18. Gianni Franchi, Nacim Belkhir, Dat Nguyen Trong, Guoxuan Xia, and Andrea Pilzer. 2025. Towards understanding and quantifying uncertainty for text-to-image generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8062–8072. find
  19. Google. 2025. Gemma 3. https://blog.google/ Accessed: technology/developers/gemma-3/. 2026-01-02. ↗
  20. Google. 2025. Nano-banana. https://gemini.google/overview/image-generation/. ↗
  21. Google. 2025. Nanobanana-pro. https://blog.google/technology/ai/nano-banana-pro/. ↗
  22. Afifa Hamza. 2023. The importance of using infographics in field of journalism. Journal El-Baheth in Human and Social Sciences Volume14-Number, page 123. find
  23. Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Junhui Cui, Yunqiu Wang, Yang Tai, Jingwei Sun, Chunle Guo, and Chongyi Li. 2024. EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluation. Preprint, arXiv:2412.18150. — [arXiv:2412.18150](https://arxiv.org/abs/2412.18150) find
  24. Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation. Preprint, arXiv:2311.16483.
  25. Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. 2023. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20406–20417.
  26. Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2025. T2i-compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation. IEEE Transactions on Pattern Analysis and Machine Intelligence. find
  27. Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2023. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723–78747. find
  28. Qirui Huang, Min Lu, Joel Lanir, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. 2024. Graphi-mind: Llm-centric interface for information graphics design. arXiv preprint arXiv:2401.13245. — [arXiv:2401.13245](https://arxiv.org/abs/2401.13245) find
  29. Simon Huang, Lynsey J Martin, Calvin H Yeh, Alvin Chin, Heather Murray, William B Sanderson, Rohit Mohindra, Teresa M Chan, and Brent Thoma. 2018. The effect of an infographic promotion on research dissemination and readership: a randomized controlled trial. Canadian Journal of Emergency Medicine, 2… find
  30. Boyan Li, Yiran Peng, Yupeng Xie, Sirong Lu, Yizhang Zhu, Xing Mu, Xinyu Liu, and Yuyu Luo. 2026. Deepeye: A steerable self-driving data agent system. arXiv preprint arXiv:2603.28889. — [arXiv:2603.28889](https://arxiv.org/abs/2603.28889) find
  31. Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. 2024. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint arXiv:2402.17245. — [arXiv:2402.17245](https://arxiv.org/abs/2402.17245) find
  32. Zhen Li, Duan Li, Yukai Guo, Xinyuan Guo, Bowen Li, Lanxi Xiao, Shenyu Qiao, Jiashu Chen, Zijian Wu, Hui Zhang, and 1 others. 2025. Chartgalaxy: A dataset for infographic chart understanding and generation. arXiv preprint arXiv:2505.18668. — [arXiv:2505.18668](https://arxiv.org/abs/2505.18668) find
  33. Yuyu Luo, Xuedi Qin, Nan Tang, and Guoliang Li. 2018. Deepeye: Towards automatic data visualization. In 2018 IEEE 34th international conference on data engineering (ICDE), pages 101–112. IEEE. find
  34. Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V Jawahar. 2021. Infographicvqa. Preprint, arXiv:2104.12756. — [arXiv:2104.12756](https://arxiv.org/abs/2104.12756) find
  35. Meta AI. 2025. Llama 4: Multimodal intelligence. https://ai.meta.com/blog/ llama-4-multimodal-intelligence/. ↗
  36. MinimaxI. 2025. Image-01. https://www.minimaxi.com/news/image-01. ↗
  37. Mistral AI. 2025. Mistral small https://docs.mistral.ai/models/ mistral-small-3-2-25-06. 3.2. ↗
  38. OpenAI. 2025. Gpt-5. https://openai.com/gpt-5/. Accessed: 2026-01-02. ↗
  39. OpenAI Team. 2025. New chatgpt images is here. https://openai.com/index/ new-chatgpt-images-is-here/. ↗
  40. Bo Pan, Yixiao Fu, Ke Wang, Junyu Lu, Lunke Pan, Ziyang Qian, Yuhan Chen, Guoliang Wang, Yitao Zhou, Li Zheng, Yinghao Tang, Zhen Wen, Yuchen Wu, Junhua Lu, Biao Zhu, Minfeng Zhu, Bo Zhang, and Wei Chen. 2025. Vis-shepherd: Constructing critic for llm-based data visualization generation. Preprint, a… — [arXiv:2506.13326](https://arxiv.org/abs/2506.13326) find
  41. Yuyang Peng, Shishi Xiao, Keming Wu, Qisheng Liao, Bohan Chen, Kevin Lin, Danqing Huang, Ji Li, and Yuhui Yuan. 2025. BizGen: Advancing Article-level Visual Text Rendering for Infographics Generation. Preprint, arXiv:2503.20672. — [arXiv:2503.20672](https://arxiv.org/abs/2503.20672) find
  42. Pruna AI. 2025. Pruna ai. https://www.pruna.ai. ↗
  43. Xuedi Qin, Yuyu Luo, Nan Tang, and Guoliang Li. 2020. Making data visualization more efficient and effective: a survey. VLDB J., 29(1):93–117. find
  44. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8…
  45. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, and 1 others. 2022. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processin…
  46. Yinghao Tang, Yupeng Xie, Yingchaojie Feng, Tingfeng Lan, and Wei Chen. 2026a. Demonstrating vividoc: Generating interactive documents through human-agent collaboration. arXiv preprint arXiv:2603.01912.
  47. Yinghao Tang, Yupeng Xie, Yingchaojie Feng, Tingfeng Lan, Jiale Lao, Yue Cheng, and Wei Chen. 2026b. Vividoc: Generating interactive documents through human-agent collaboration. arXiv preprint arXiv:2603.27991. — [arXiv:2603.27991](https://arxiv.org/abs/2603.27991) find
  48. Image Team, Huanqia Cai, Sihan Cao, Ruoyi Du, Peng Gao, Steven Hoi, Zhaohui Hou, Shijie Huang, Dengyang Jiang, Xin Jin, Liangchen Li, Zhen Li, Zhong-Yu Li, David Liu, Dongyang Liu, Junhan Shi, Qilong Wu, Feng Yu, Chi Zhang, and 2 others. 2025. Z-image: An efficient image generation foundation model… find
  49. Lisa Traboco, Haridha Pandian, Elena Nikiphorou, and Latika Gupta. 2022. Designing infographics: visual representations for enhancing education, communication, and scientific research. Journal of Korean medical science, 37(27). find
  50. V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, and et al. 2025. Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. Preprint, arXiv:2507.01006.
  51. Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, and 1 others. 2025. Qwen-image technical report. arXiv preprint arXiv:2508.02324.
  52. Yifan Wu, Yiran Peng, Yiyu Chen, Jianhao Ruan, Zijie Zhuang, Cheng Yang, Jiayi Zhang, Man Chen, Yenchi Tseng, Zhaoyang Yu, Liang Chen, Yuyao Zhai, Bang Liu, Chenglin Wu, and Yuyu Luo. 2026. Autowebworld: Synthesizing infinite verifiable web environments via finite state machines. Preprint, arXiv:260… — [arXiv:2602.14296](https://arxiv.org/abs/2602.14296) find
  53. Yifan Wu, Lutao Yan, Leixian Shen, Yunhai Wang, Nan Tang, and Yuyu Luo. 2024. Chartinsights: Evaluating multimodal large language models for low-level chart question answering. arXiv preprint arXiv:2405.07001. xAI. 2025. Grok 4.1. https://x.ai/news/grok-4-1. Accessed: 2026-01-02. ↗
  54. Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Peng Ye, Min Dou, Botian Shi, Junchi Yan, and Yu Qiao. 2025. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. Preprint, arXiv:2402.12185. — [arXiv:2402.12185](https://arxiv.org/abs/2402.12185) find
  55. Shishi Xiao, Suizi Huang, Yue Lin, Yilin Ye, and Wei Zeng. 2023. Let the chart spark: Embedding semantic context into chart with text-to-image generative model. IEEE Transactions on Visualization and Computer Graphics, 30(1):284–294. find
  56. Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, and Maosong Sun. 2024b. MatPlotAgent: Method and evaluation for LLM-based agentic scientific data visualization. In Findings of the Association for Computation… find
  57. Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, and 1 others. 2022. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5. — [arXiv:2206.10789](https://arxiv.org/abs/2206.10789) find
  58. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, and et al. 2025a. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. Preprint, arXiv:2504.10479.
  59. Yizhang Zhu, Liangwei Wang, Chenyu Yang, Xiaotian Lin, Boyan Li, Wei Zhou, Xinyu Liu, Zhangyang Peng, Tianqi Luo, Yu Li, and 1 others. 2025b. A survey of data agents: Emerging paradigm or overstated hype? arXiv preprint arXiv:2510.23587. — [arXiv:2510.23587](https://arxiv.org/abs/2510.23587) find
  60. Le Zhuo, Songhao Han, Yuandong Pu, Boxiang Qiu, Sayak Paul, Yue Liao, Yihao Liu, Jie Shao, Xi Chen, Si Liu, and Hongsheng Li. 2025. Factuality Matters: When Image Generation and Editing Meet Structured Visuals. Preprint, arXiv:2510.05091. — [arXiv:2510.05091](https://arxiv.org/abs/2510.05091) find
  61. Tianchi Xie, Minzhi Lin, Mengchen Liu, Yilin Ye, Changjian Chen, and Shixia Liu. 2025a. In-foChartQA: A Benchmark for Multimodal Question Answering on Infographic Charts. Preprint, arXiv:2505.19028. — [arXiv:2505.19028](https://arxiv.org/abs/2505.19028) find
  62. Yupeng Xie, Yuyu Luo, Guoliang Li, and Nan Tang. 2024. Haichart: Human and ai paired visualization system. Proceedings of the VLDB Endowment, 17(11):3178–3191. find
  63. Yupeng Xie, Zhiyang Zhang, Yifan Wu, Sirong Lu, Jiayi Zhang, Zhaoyang Yu, Jinlin Wang, Sirui Hong, Bang Liu, Chenglin Wu, and Yuyu Luo. 2025b. VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations. Preprint, arXiv:2510.22373. — [arXiv:2510.22373](https://arxiv.org/abs/2510.22373) find
  64. Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. 2023. Chartbench: A benchmark for complex visual reasoning in charts. arXiv preprint arXiv:2312.15915. — [arXiv:2312.15915](https://arxiv.org/abs/2312.15915) find
  65. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, and et al. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  66. Xudong Yang, Yifan Wu, Yizhang Zhu, Nan Tang, and Yuyu Luo. 2024a. Askchart: Universal chart understanding through textual enhancement. arXiv preprint arXiv:2412.19146. — [arXiv:2412.19146](https://arxiv.org/abs/2412.19146) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tang-2026-igenbench-benchmarking-reliability-text-infographic/