Elamur Research⌕
☰
home / papers / 2025 / METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling

METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling

Summary

The paper addresses chart-to-code generation: given an image of a reference chart, a model must write plotting code (e.g., Python) whose rendered output reproduces the chart's type, layout, colors and text. Prompting vision-language models directly, including GPT-4o, often yields charts with structural, color or text errors, because the task requires both visual understanding and precise coding. The authors propose METAL, a multi-agent framework built on VLMs. It splits the work among four roles: a generator that drafts the code, a visual critic that compares the rendered chart with the reference, a code critic that points to faulty lines, and a reviser that applies both kinds of feedback. A multi-criteria verifier stops the loop once quality thresholds are met or an iteration limit is reached. METAL beats the strongest previous approach in F1 and gives large gains over direct prompting with both proprietary and open models. Its accuracy also keeps rising as more inference tokens are spent. This suggests that iterative, modality-specific critique is an effective way to scale test-time compute for visually grounded code generation.

Key points

  • Agents: METAL uses a Generation Agent, a Visual Critique Agent, a Code Critique Agent and a Revision Agent. They work in an iterative critique–revise loop that is controlled by a verifier.
  • Main results: METAL improves F1 by 5.2% over the previous best chart-generation result. With LLaMA 3.2-11B as the base model, it gains 11.33% over Direct Prompting.
  • Test-time scaling: Performance grows monotonically, roughly linearly in log scale, as the token budget increases from 2^9 to 2^13 tokens.
  • Ablations:
    • Keeping visual and code critiques separate gives 5.6% more than a single combined critic.
    • On a 50-example subset, a non-agentic modular variant, without self-decision-making or code execution, loses 4.51% of the average gain over direct prompting.
  • Limitations:
    • The method depends heavily on prompt engineering.
    • Automatic chart metrics are imperfect.
    • Inference cost is higher than single-pass prompting.
    • With LLaMA 3.2-11B, gains shrink on harder charts, which the authors attribute to that model's weaker critique ability.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

12 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 11 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.1452

@inproceedings{li2025metal,
  title = {{METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling}},
  author = {Li, Bingxuan and Wang, Yiwei and Gu, Jiuxiang and Chang, Kai-Wei and Peng, Nanyun},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1452},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1452}
}

References

27 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama 3 model card. find
  2. Anthropic. 2024. Introducing the next generation of claude. find
  3. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966. ↗ our card
  4. Yilun Du and Leslie Kaelbling. 2024. Compositional generative modeling: A single model is not all you need. arXiv preprint arXiv:2402.01103. ↗
  5. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. ↗ our card
  6. Google. 2023. Gemini: A family of highly capable multimodal models. ArXiv, abs/2312.11805. ↗ our card
  7. Alex Graves. 2012. Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711. ↗ our card
  8. Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483. ↗
  9. Fantine Huot, Reinald Kim Amplayo, Jennimaria Palomaki, Alice Shoshana Jakobovits, Elizabeth Clark, and Mirella Lapata. 2024. Agents’ room: Narrative generation through multi-step collaboration. arXiv preprint arXiv:2410.02603. ↗
  10. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. ↗ our card
  11. Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. 2024. Llava-next: Stronger llms supercharge multimodal capabilities in the wild. find
  12. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. Deepseek-vl: Towards real-world vision-language understanding. ↗ our card
  13. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393. ↗ our card
  14. OpenAI. 2023. Gpt-4v(ision) system card. find
  15. Xuedi Qin, Yuyu Luo, Nan Tang, and Guoliang Li. 2020. Making data visualization more efficient and effective: a survey. The VLDB Journal, 29(1):93–117. ↗
  16. Chufan Shi, Cheng Yang, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, Gongye Liu, Xiaomei Nie, Deng Cai, and Yujiu Yang. 2024. Chartmimic: Evaluating lmm’s cross-modal reasoning capability via chart-to-code generation. arXiv preprint arXiv:2406.09961. ↗
  17. Ruyi Xu, Yuan Yao, Zonghao Guo, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, and Gao Huang. 2024. LLaVA-UHD: an lmm perceiving any aspect ratio and high-resolution images. arXiv preprint arXiv:2403.11703. ↗
  18. Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. 2023. Chartbench: A benchmark for complex visual reasoning in charts. arXiv preprint arXiv:2312.15915. ↗
  19. Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, et al. 2024. Matplotagent: Method and evaluation for llm-based agentic scientific data visualization. arXiv preprint arXiv:2402.11453. ↗
  20. Matei Zaharia, Omar Khattab, Lingjiao Chen, Jared Quincy Davis, Heather Miller, Chris Potts, James Zou, Michael Carbin, Jonathan Frankle, Naveen Rao, and Ali Ghodsi. 2024. The shift from models to compound ai systems. https://bair.berkeley.edu/blog/2024/02/ 18/compound-ai-systems/. ↗
  21. Kexun Zhang, Shang Zhou, Danqing Wang, William Yang Wang, and Lei Li. 2024. Scaling llm inference with optimized sample compute allocation. arXiv preprint arXiv:2410.22480. ↗
  22. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314. ↗ our card
  23. Virginia Teller. 2000. Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition. find
  24. Yifan Wang, Qingyan Guo, Xinzhe Ni, Chufan Shi, Lemao Liu, Haiyun Jiang, and Yujiu Yang. 2024. Hint-enhanced in-context learning wakes large language models up for knowledge-intensive tasks. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10276–10280. IEEE. find
  25. Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhixuan Liang, Zeyu Lu, Ying Shan, and Ping Luo. 2024a. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. arXiv preprint arXiv:2405.07990. ↗
  26. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024b. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408.00724. ↗ our card
  27. Yijia Xiao, Edward Sun, Di Luo, and Wei Wang. 2024. Tradingagents: Multi-agents llm financial trading framework. arXiv preprint arXiv:2412.20138. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/li-2025-metal-multi-agent-framework-chart/