Elamur Research⌕
☰
home / papers / 2026 / Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation

Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation

Summary

Turning a chart image into a table is a key step for multimodal language models (MLMs) that must answer complex questions about charts. The authors observe that popular chart corpora such as PlotQA and ChartQA are heavily skewed in their y-axis properties, especially the digit length of the axis maximum. They ask whether this skew causes uneven model behaviour. To find out, they built FairChart2Table, a framework with two parts:

  • Benchmark: synthetic charts rendered with Bokeh. All non-y-axis elements are held fixed, and one y-axis factor is varied at a time.
  • Metrics: new error measures based on Tick-Based Error (TBE). These penalise mistakes large enough to see on the chart more heavily than small numeric deviations.

Several models were tested, including DePlot, ChartGemma, TinyChart, Pixtral, Gemini and GPT-4o. Clear biases appear for digit length, number of major ticks, value range and tick label format. Accuracy also drops as more data series (entities) are drawn. Chart type matters less once the underlying data are identical. Giving general-purpose MLMs the y-axis tick values in the prompt can noticeably improve results. The benchmark offers a controlled way to audit such biases in other models.

Key points

  • Dataset imbalance: measured by the coefficient of variation of images per digit length, PlotQA scores 1.72 and ChartQA 1.40. FairChart2Table is fully balanced at 0.00 and covers digit lengths 0–16.
  • Part A (digit length): built from 1,020 tables (1–6 entities, digit lengths 0–16), each rendered as a line, dot and bar chart, giving 3,060 images. Parts B–D reuse the 170 three-entity tables to vary:
    • B: major tick count (3 or 11, versus 6 in Part A)
    • C: y-axis range
    • D: tick label format (comma, scientific notation, abbreviations such as K/M/B/T)
  • TBE and metrics: TBE is the absolute prediction error divided by the minor tick interval. It underlies the metrics RMS TBE F1, RMS TBE F1-Sig, TBE-Raw and SES.
  • Entity count: the number of entities correlates strongly with line crossings (Pearson 0.9549), which plausibly explains why models confuse values at the same x position. Gemini and TinyChart were the most robust to this effect.
  • Y-axis hints: prompting with tick values raised Pixtral's scores by 0.94–26.60 points. For Gemini on abbreviated labels, the score rose from 46.54 to 61.32. GPT-4o improved in most settings, but not for the comma or scientific formats.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1765

@inproceedings{song2026assessing,
  title = {{Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation}},
  author = {Song, Seok Hwan and Efat, Azher Ahmed and Tavanapong, Wallapak},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1765},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1765}
}

References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. 2024. Pixtral 12b. arXiv preprint arXiv:2410.07073. ↗
  2. Mubashara Akhtar, Oana Cocarascu, and Elena Simperl. 2023. Reading and reasoning over chart images for evidence-based automated fact-checking. In Findings of the Association for Computational Linguistics: EACL 2023, pages 399–414, Dubrovnik, Croatia. Association for Computational Linguistics. find
  3. Ritwick Chaudhry, Sumit Shekhar, Utkarsh Gupta, Pranav Maneriker, Prann Bansal, and Ajay Joshi. 2020. Leaf-qa: Locate, encode attend for figure question answering. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 3501–3510. find
  4. Ashim Gupta, Vivek Gupta, Shuo Zhang, Yujie He, Ning Zhang, and Shalin Shah. 2024. Enhancing question answering on charts through effective pre-training tasks. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 185–192, Miami, Florida, US. Association for Computational Linguistics. find
  5. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. ↗ our card
  6. Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, and Enamul Hoque. 2024. Are large vision language models up to the challenge of chart comprehension and reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3334–3368, Miami, Florida, USA. Association for Computational Linguistics. find
  7. Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018. Dvqa: Understanding data visualizations via question answering. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5648–5656. find
  8. Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2017. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300. ↗
  9. Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022. Chart-to-text: A large-scale benchmark for chart summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4005–4023, Dublin, Ireland. Association for Computational Linguistics. find
  10. Wonjoong Kim, Sangwu Park, Yeonjun In, Seokwon Han, and Chanyoung Park. 2025. Simplot: Enhancing chart question answering by distilling essentials. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 573–593. find
  11. Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. 2023a. Deplot: One-shot visual language reasoning by plot-to-table translation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10381–10399. ↗
  12. Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Eisenschlos. 2023b. Matcha: Enhancing visual language pretraining with math reasoning and chart derendering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12756–12770. ↗
  13. Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, and Dong Yu. 2024. MMC: Advancing multimodal chart understanding with large-scale instruction tuning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1287–1310, Mexico City, Mexico. Association for Computational Linguistics. ↗
  14. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, pages 2263– 2279. find our card
  15. Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. 2023. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 14662–14684. find
  16. Ahmed Masry, Mehrad Shahmohammadi, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. 2024. ChartInstruct: Instruction tuning for chart comprehension and reasoning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 10387–10409, Bangkok, Thailand. Association for Computational Linguistics. find
  17. Ahmed Masry, Megh Thakkar, Aayush Bajaj, Aaryaman Kartha, Enamul Hoque, and Shafiq Joty. 2025. ChartGemma: Visual instruction-tuning for chart reasoning in the wild. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 625–643, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  18. Fanqing Meng, Wenqi Shao, Quanfeng Lu, Peng Gao, Kaipeng Zhang, Yu Qiao, and Ping Luo. 2024. Chartassistant: A universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7775– 7803. find
  19. Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1527–1536. find
  20. Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta, and Dan Roth. 2024. Unraveling the truth: Do VLMs really understand charts? a deep dive into consistency and robustness. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16696–16717, Miami, Florida, USA. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-assessing-y-axis-influence-bias/