Elamur Research⌕
☰
home / papers / 2022 / Barch: an English Dataset of Bar Chart Summaries

Barch: an English Dataset of Bar Chart Summaries

Paper · 2022 · Iza Škrjanec, Muhammad Salman Edhi and Vera Demberg

Summary

Chart-to-text generation needs charts paired with descriptions, but existing resources have problems. AutoChart uses template text of doubtful naturalness. Chart2Text uses crawled human text that often brings in outside knowledge, and its alignment skips inferential statements. Barch addresses this with human-written English summaries of bar charts. The authors fabricated data for vertical bar charts with 3–6 bars across many topics. They varied each chart's title so that it signals one of four intended messages, which lets researchers study how framing shapes what people write. Crowdworkers described each chart as if presenting it aloud to an audience, so that the text alone would convey the data. Summary spans were then aligned to chart entities (bars ranked by height, axes, title) and to analytical inferences such as differences, ratios, groups, slopes and approximate heights. The authors trained an LSTM baseline and two stronger neural generators on the data. They evaluated these models both automatically and with human judges, discussing each model's strengths and weaknesses. Barch therefore serves both the study of human summarization and the training of NLG systems.

Key points

  • 47 charts on 18 topics. Each title carries one of four intended messages: neutral, proportional (highlights the tallest bars), inverse (the lowest bars) or emphasis (one arbitrary bar).
  • Summaries were collected via Prolific and LingoTurk from 72 UK native English speakers holding at least a Bachelor's degree, paid 8.48 GBP per hour. After 16 poor summaries were removed, 1,063 remain, about 22 per chart.
  • Basic labels are height-ranked: firstX/firstY, secondX/secondY … lastX/lastY, plus X, Y and title.
  • Five analytical label types: addition, multiplication, grouping, slope and height approximation. Label names record which bars are involved, e.g. addition-1-4.
  • Models tested: an OpenNMT LSTM with copy attention, the C2T transformer and fine-tuned KGPT. Corpus and guidelines are on GitHub (izaskr/barch_dataset).

Interesting ideas

  • I1. Bars are labelled by height rank rather than by their names, so one template like "firstX is highest" applies to every chart and placeholders transfer across topics.
  • I2. Analytical statements (difference, ratio, grouping, approximation) are annotated as explicit relations between named data points, not left as free text.
  • I3. The same data is shown under four titles with different intended messages, isolating how framing alone changes content selection.
  • I4. Each input has many independent human summaries, revealing natural variation in content and ordering rather than a single gold reference.
  • I5. Summaries that were not factual were filtered out, and the alignment only covers content grounded in the input data, avoiding the outside-knowledge noise seen in crawled corpora.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Entities holding quantities could get rank-based roles (largest, smallest, second), so questions like "who has the most" are answered against ranks computed in the world of the text.
  • I2 (answer): Maths word-problem readers could use the same small inventory of comparison relations to map "twice as many" or "5 more than" onto operations over world quantities.
  • I3 (frames): The prag crate could treat a title or lead sentence as a focus cue that predicts which entities the following text will foreground, which helps salience and coreference ranking.
  • I4 (back): The text → graph → text generator could be evaluated against multiple references and learn which content orderings humans prefer when verbalizing a set of facts.
  • I5 (learn): Silver training pairs for generation could be checked so that every generated claim is traceable to a node in the meaning graph, rejecting ungrounded spans as a negative control.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Saarland UniversitySaarbrücken · Germany148 works · 18,287 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Iza Škrjanec, Muhammad Salman Edhi and Vera Demberg. 2022. Barch: an English Dataset of Bar Chart Summaries. Proceedings of the Thirteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{skrjanec2022barch,
  title = {{Barch: an English Dataset of Bar Chart Summaries}},
  author = {Iza Škrjanec, Muhammad Salman Edhi and Vera Demberg},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  doi = {10.63317/3anxvtxebcw2},
  url = {https://doi.org/10.63317/3anxvtxebcw2}
}

References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bahdanau, D., Cho, K., and Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. January. 3rd International Conference on Learning Representations, ICLR 2015 ; Conference date: 07-05-2015 Through 09-05-2015. ↗
  2. Chen, W., Su, Y., Yan, X., and Wang, W. Y. (2020). KGPT: Knowledge-grounded pre-training for data-to-text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8635–8648, Online, November. Association for Computational Linguistics.
  3. Clive, J., Cao, K., and Rei, M. (2021). Control prefixes for text generation. ArXiv, abs/2110.08329. ↗
  4. Demir, S., Carberry, S., and McCoy, K. F. (2012). Summarizing information graphics textually. Computational Linguistics, 38(3):527–574. ↗
  5. Ferres, L., Verkhogliad, P., Lindgaard, G., Boucher, L., Chretien, A., and Lachance, M. (2007). Improving accessibility to statistical graphs: The IGraph-Lite system. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, Assets ’07, page 67–74, New York, NY, USA. Association for Computing Machinery. find
  6. Greenbacker, C., Carberry, S., and McCoy, K. (2011). A corpus of human-written summaries of line graphs. In Proceedings of the UCNLG+Eval: Language Generation and Evaluation Workshop, pages 23–27, Edinburgh, Scotland, July. Association for Computational Linguistics. find
  7. Kingma, D. P. and Ba, J. (2015). Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015). ICLR. ↗
  8. Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A. M. (2017). OpenNMT: Open-source toolkit for neural machine translation. In Proc. ACL.
  9. Li, L., Ma, C., Yue, Y., and Hu, D. (2021). Improving encoder by auxiliary supervision tasks for table-to-text generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5979–5989, Online, August. Association for Computational Linguistics. find
  10. Moryossef, A., Goldberg, Y., and Dagan, I. (2019). Step-by-step: Separating planning from realization in neural data-to-text generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2267–2277. Association for Computational Linguistics. ↗
  11. Obeid, J. and Hoque, E. (2020). Chart-to-text: Generating natural language descriptions for charts by adapting the transformer model. In Proceedings of the 13th International Conference on Natural Language Generation, pages 138–147, Dublin, Ireland, December. Association for Computational Linguistics. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. find
  12. (2002). Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 311–318, USA. ACL.
  13. Puduppully, R., Dong, L., and Lapata, M. (2019). Data-to-text generation with content selection and planning. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, volume 33, page 6908–6915. ↗
  14. Pusse, F., Sayeed, A., and Demberg, V. (2016). Lingo-Turk: managing crowdsourced tasks for psycholinguistics. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 57–61, San Diego, California, June. Association for Computational Linguistics. find
  15. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language models are unsupervised multitask learners. find
  16. Talbot, J., Setlur, V., and Anand, A. (2014). Four experiments on the perception of bar charts. IEEE Transactions on Visualization and Computer Graphics, 20(12):2152–2160. ↗
  17. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems, pages 5998– 6008.
  18. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. (2020). Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. find
  19. Zhu, J., Ran, J., Lee, R. K.-W., Li, Z., and Choo, K. (2021). AutoChart: A dataset for chart-to-text generation task. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1636–1644, Held Online, September. INCOMA Ltd. 10. ↗
  20. Language Resource References Obeid, J. and Hoque, E. (2020). Chart-to-text: Generating natural language descriptions for charts by adapting the transformer model. In Proceedings of the 13th International Conference on Natural Language Generation, pages 138–147, Dublin, Ireland, December. Association for Computational Linguistics. find
  21. Zhu, J., Ran, J., Lee, R. K.-W., Li, Z., and Choo, K. (2021). AutoChart: A dataset for chart-to-text generation task. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1636–1644, Held Online, September. INCOMA Ltd. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/skrjanec-2022-barch-english-dataset-bar-chart/