Elamur Research⌕
☰
home / papers / 2026 / Aligned Multi-View Scripts for Universal Chart-to-Code Generation

Aligned Multi-View Scripts for Universal Chart-to-Code Generation

Paper · 2026 · Zhihan Zhang and Lizi Liao

Summary

Converting a chart image into a runnable plotting script has been studied almost only for Python/matplotlib, even though R (ggplot2) and LaTeX (TikZ) are standard tools in many sciences. The authors treat scripts in different languages that draw the same chart as alternative "views" of one visual meaning, and they use this equivalence as an additional training signal. They build Chart2NCode, a resource of 176K charts. Each chart comes with Python, R and LaTeX scripts that render the same picture. To create it, the authors extract language-neutral metadata from existing scripts, insert it into hand-written templates, repair failed scripts with GPT-4o, and have humans check a sample. On top of a LLaVA-style model they add CharLuMA. This extension to the multimodal projector uses a router that selects a few low-rank subspaces based on the target language. The resulting models beat strong open-source baselines on executability and visual fidelity in all three languages and remain competitive with proprietary systems. The work extends chart reproduction beyond Python and shows that balanced multi-language supervision helps every language.

Key points

  • Chart2NCode draws on three sources: Python scripts from ChartCoder, chart-related LaTeX from DaTikZ, and 40K newly collected R scripts. A library of 202 curated templates covers more than 30 chart subtypes.
  • Human checking covered 1,000 quadruples. Structural fidelity was rated high in 98.2% of cases and data integrity in 95.2%.
  • CharLuMA comes in 1.3B and 6.7B sizes and has 32 low-rank subspaces. About 0.18–0.19 of the activated subspaces (roughly 5 of 27) are shared across languages, and the rest specialize by language.
  • Ablations showed that training on all three aligned languages outperformed one- or two-language training. Training on unaligned source data skewed the model toward Python.
  • Typical errors differ by language. Python and R fail mostly from dimension mismatches (72.3% and 56.1% of errors). LaTeX fails mostly from syntax problems such as missing braces (55.5%).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 worksSoftware · 119 works

🧭 Topics 1Research subjects and languages the work is about.Code & programming languages · 168 works

Cite

Cite as Zhihan Zhang and Lizi Liao. 2026. Aligned Multi-View Scripts for Universal Chart-to-Code Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.943

@inproceedings{zhang2026aligned,
  title = {{Aligned Multi-View Scripts for Universal Chart-to-Code Generation}},
  author = {Zhihan Zhang and Lizi Liao},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.943},
  url = {https://doi.org/10.18653/v1/2026.acl-long.943}
}

References

51 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, and 110 others. 2024. Phi-… — Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
  2. Anthropic. 2024. Claude 3.5 sonnet. find
  3. Anthropic. 2025. Introducing claude 4. find
  4. Philip Bachman, R Devon Hjelm, and William Buchwalter. 2019. Learning representations by maximizing mutual information across views. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA. Curran Associates Inc. find
  5. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. ArXiv, abs/2308.12966.
  6. Jonas Belouadi, Anne Lauscher, and Steffen Eger. 2024a. AutomaTikZ: Text-guided synthesis of scientific vector graphics with TikZ. In The Twelfth International Conference on Learning Representations.
  7. Jonas Belouadi, Simone Paolo Ponzetto, and Steffen Eger. 2024b. Detikzify: Synthesizing graphics programs for scientific figures and sketches with tikz. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  8. Tianlong Chen, Xuxi Chen, Xianzhi Du, Abdullah Rashwan, Fan Yang, Huizhong Chen, Zhangyang Wang, and Yeqing Li. 2023. Adamv-moe: Adaptive multi-task vision mixture-of-experts. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 17300–17311. find
  9. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo…
  10. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA. Curran Associates Inc. find
  11. Longyin Wen. 2025. Cumo: scaling multimodal llm with co-upcycled mixture-of-experts. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associates Inc. find
  12. Chenhao Ding, Jiangyang Li, SongLin Dong, Xinyuan Gao, Yuhang He, and Yihong Gong. 2025. SuLoRA: Subspace low-rank adaptation for parameter-efficient fine-tuning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5334–5349, Vienna, Austria. Association for Computational L… find
  13. Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning.
  14. Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. 2023. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. In Advances in Neural Information Processing Systems, volume 36, pages 50742–50768. find
  15. Kanika Goswami, Puneet Mathur, Ryan Rossi, and Franck Dernoncourt. 2025. Plotgen: Multi-agent llm-based scientific data visualization via multimodal retrieval feedback. In Companion Proceedings of the ACM on Web Conference 2025, WWW ’25, page 1672–1676, New York, NY, USA. Association for Computing M… find
  16. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. UniXcoder: Unified cross-modal pre-training for code representation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7212–7225, Dublin, Ireland. Associa…
  17. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. Deepseek-coder: When the large language model meets programming – the rise of code intelligence. Preprint, arXiv:2401.14196. find
  18. Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation. Preprint, arXiv:2311.16483.
  19. Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, and Xuanjing Huang. 2025. Distill visual chart reasoning ability from llms to mllms. Preprint, arXiv:2410.18798. find
  20. Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, and Zhuowen Tu. 2024. Bliva: A simple multimodal llm for better handling of text-rich visual questions. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3):2256–2264. find
  21. Klaus Krippendorff. 2011. Computing krippendorff’s alpha-reliability. Departmental Papers (ASC), page 43. find
  22. Jiachen Li, Xinyao Wang, Sijie Zhu, Chia-Wen Kuo, Lu Xu, Fan Chen, Jitesh Jain, Humphrey Shi, and Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, Jiaming Han, Siyuan Huang, Yichi Zhang, Xuming He, Hongsheng Li, and Yu Qiao. 2023. Sphinx… — [arXiv:2311.07575](https://arxiv.org/abs/2311.07575) find
  23. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26286–26296.
  24. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems.
  25. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. Deepseek-vl: Towards real-world vision-language understanding. Preprint, arXiv:2403.05525.
  26. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chun yue Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations.
  27. Meta. 2024. Introducing meta llama 3: The most capable openly available llm to date. find
  28. Paul Mooney. 2022. 2022 kaggle machine learning and data science survey. find
  29. Tianhao Niu, Yiming Cui, Baoxin Wang, Xiao Xu, Xin Yao, Qingfu Zhu, Dayong Wu, Shijin Wang, and Wanxiang Che. 2025. Chart2Code53: A large-scale diverse and complex dataset for enhancing chart-to-code generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processin… find
  30. OpenAI. 2024a. Gpt-4o mini: advancing cost-efficient intelligence. find
  31. OpenAI. 2024b. Hello gpt-4o. find
  32. OpenAI. 2025. Gpt-5 is here. find
  33. Baptiste Roziere, Marie-Anne Lachaux, Lowik Chanussot, and Guillaume Lample. 2020. Unsupervised translation of programming languages. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Curran Associates Inc.
  34. Chufan Shi, Cheng Yang, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, Gongye Liu, Xiaomei Nie, Deng Cai, and Yujiu Yang. 2025. Chartmimic: Evaluating LMM’s cross-modal reasoning capability via chart-to-code generation. In The Thirteenth International Co… find
  35. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  36. Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Chengzhong Xu. 2025. Hydralora: an asymmetric lora architecture for efficient fine-tuning. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associates Inc. find
  37. Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive multiview coding. In Computer Vision – ECCV 2020, pages 776–794, Cham. Springer International Publishing. find
  38. Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. 2024. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. In Advances in Neural…
  39. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, and 56 others. 2025. Internvl3.5: Advancing open-source multimodal…
  40. Chengyue Wu, Zhixuan Liang, Yixiao Ge, Qiushan Guo, Zeyu Lu, Jiahao Wang, Ying Shan, and Ping Luo. 2025. Plot2Code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. In Findings of the Association for Computational Linguistics: NAACL… find
  41. Taiqiang Wu, Jiahao Wang, Zhe Zhao, and Ngai Wong. 2024. Mixture-of-subspaces in low-rank adaptation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7880–7899, Miami, Florida, USA. Association for Computational Linguistics. find
  42. Zhengzhuo Xu, Bowen Qu, Yiyan Qi, SiNan Du, Chengjin Xu, Chun Yuan, and Jian Guo. 2025. Chartmoe: Mixture of diversely aligned expert connector for chart understanding. In The Thirteenth International Conference on Learning Representations. find
  43. Pengyu Yan, Mahesh Bhosale, Jay Lal, Bikhyat Adhikari, and David Doermann. 2024. Chartreformer: Natural language-driven chart image editing. In Document Analysis and Recognition - ICDAR 2024: 18th International Conference, Athens, Greece, August 30–September 4, 2024, Proceedings, Part I, page 453–46… find
  44. Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, and Maosong Sun. 2024. MatPlotAgent: Method and evaluation for LLM-based agentic scientific data visualization. In Findings of the Association for Computationa… find
  45. Xiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, and 3 others. 2024. Mmmu: A massive multi-discipline multimodal u…
  46. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023a. Sigmoid loss for language image pre-training. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11941–11952.
  47. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023b. Sigmoid loss for language image pre-training. Preprint, arXiv:2303.15343.
  48. Wenqi Zhang, Zhenglin Cheng, Yuanyu He, Mengna Wang, Yongliang Shen, Zeqi Tan, Guiyang Hou, Mingqian He, Yanna Ma, Weiming Lu, and Yueting Zhuang. 2024. Multimodal self-instruct: Synthetic abstract image and visual reasoning instruction using language model. In Proceedings of the 2024 Conference on… find
  49. Zhihan Zhang, Yixin Cao, and Lizi Liao. 2025a. Boosting chart-to-code generation in mllm via dual preference-guided refinement. In Proceedings of the 33rd ACM International Conference on Multimedia, MM ’25, page 11032–11041, New York, NY, USA. Association for Computing Machinery. find
  50. Zhihan Zhang, Yixin Cao, and Lizi Liao. 2025b. XFin-Bench: Benchmarking LLMs in complex financial problem solving and reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8715–8758, Vienna, Austria. Association for Computational Linguistics. find
  51. Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, and Maosong Sun. 2025. Chart-Coder: Advancing multimodal large language model for chart-to-code generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-aligned-multi-view-scripts-universal/