Elamur Research
☰
home / papers / 2020 / DART: Open-Domain Structured Data Record to Text Generation

DART: Open-Domain Structured Data Record to Text Generation

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2007.02871

Summary

Data-to-text corpora tend to cover narrow domains such as restaurants or a few DBPedia categories. They also usually describe inputs as flat slot-value lists or plain table schemas, which hides how table fields relate to each other. DART addresses both problems with an open-domain corpus of about 82k pairs, each linking a set of triples to a sentence.

To build it, internal annotators assigned a parent to every column header in Wikipedia tables from WikiSQL and WikiTableQuestions, which turned each flat schema into a tree whose root holds the table context and title. Connected subtrees were then sampled from these trees and converted into subject-predicate-object triples. Crowd workers wrote descriptions of the selected cells, and internal annotators checked those descriptions.

The corpus also includes WikiSQL questions rewritten as declarative statements, along with WebNLG 2017 and Cleaned E2E, all with unified predicates. Strong pretrained generators such as BART and T5 perform worse on DART than on domain-specific benchmarks. Adding DART as training data gave new state-of-the-art results on WebNLG 2017. DART is therefore useful both as a harder benchmark and as a resource for transfer across domains.

Key points

  • DART contains 82,191 instances drawn from 5,623 tables, split into 62,659 train, 6,980 dev and 12,552 test examples.
  • Its sources are WikiSQL and WikiTableQuestions (annotated by internal staff and MTurk workers, plus declarative rewrites of questions), WebNLG 2017 (15 DBPedia categories) and Cleaned E2E (restaurants).
  • Each input is built by sampling a connected component of 2–5 nodes from the ontology tree. A traversal parameter p, drawn from 0.5–0.7, balances how wide or how deep the sampled subtree is.
  • Compared with WikiTableText, LogicNLG and ToTTo, DART's outputs are longer: 21.6 words and 1.5 sentences per realization on average.
  • Crowd annotation took 125 MTurk hours at roughly $15 per hour. Data and code are available at github.com/Yale-LILY/dart.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

121 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 116 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2007.02871 · arXiv 2007.02871

@misc{nan2020dart,
  title = {{DART: Open-Domain Structured Data Record to Text Generation}},
  author = {Nan, Linyong and Radev, Dragomir and Zhang, Rui and Rau, Amrit and Sivaprasad, Abhinand and Hsieh, Chiachun and Tang, Xiangru and Vyas, Aadit and Verma, Neha and Krishna, Pranav and Liu, Yangxiaokang and Irwanto, Nadia and Pan, Jessica and Rahman, Faiaz and Zaidi, Ahmad and Mutuma, Mutethia and Tarabar, Yasin and Gupta, Ankit and Yu, Tao and Tan, Yi Chern and Lin, Xi Victoria and Xiong, Caiming and Socher, Richard and Rajani, Nazneen Fatema},
  year = {2020},
  doi = {10.48550/arxiv.2007.02871},
  eprint = {2007.02871},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2007.02871}
}

References

50 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Junwei Bao, Duyu Tang, Nan Duan, Zhao Yan, Yuanhua Lv, Ming Zhou, and Tiejun Zhao. 2018. Table-to-text: Describing table region with natural language. In AAAI. find
  2. Thiago Castro Ferreira, Chris van der Lee, Emiel van Miltenburg, and Emiel Krahmer. 2019. Neural data-to-text generation: A comparison between pipeline and end-to-end architectures. In EMNLP. find
  3. Alison J Cawsey, Bonnie L Webber, and Ray B Jones. 1997. Natural language generation in health care. find
  4. David L Chen and Raymond J Mooney. 2008. Learning to sportscast: a test of grounded language acquisition. In ICML. find
  5. Wenhu Chen, Jianshu Chen, Yu Su, Zhiyu Chen, and William Yang Wang. 2020a. Logical natural language generation from open-domain tables. In ACL. find
  6. Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan, and William Yang Wang. 2020b. Logic2Text: High-fidelity natural language generation from logical forms. In Findings of EMNLP. find
  7. Zhiyu Chen, Harini Eavani, Wenhu Chen, Yinyin Liu, and William Yang Wang. 2020c. Few-shot nlg with pre-trained language model. In ACL. find
  8. Dorottya Demszky, Kelvin Guu, and Percy Liang. 2018. Transforming question answering datasets into natural language inference datasets. arXiv preprint arXiv:1809.02922. find
  9. Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-Wei Chang, Dipanjan Das, and William Cohen. 2019. Handling divergent reference texts when evaluating table-to-text generation. In ACL. find our card
  10. Ondřej Dušek, David M. Howcroft, and Verena Rieser. 2019. Semantic noise matters for neural natural language generation. In INLG. find
  11. Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. 2018. T-REx: A large scale alignment of natural language with knowledge base triples. In LREC. find
  12. Angela Fan, Claire Gardent, Chloé Braud, and Antoine Bordes. 2019. Using local knowledge graph construction to scale Seq2Seq models to multi-document inputs. In EMNLP-IJCNLP. find
  13. Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. The WebNLG challenge: Generating text from RDF data. In INLG. find our card
  14. Sebastian Gehrmann, Falcon Dai, Henry Elder, and Alexander Rush. 2018. End-to-end content and plan selection for data-to-text generation. In INLG. find
  15. Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. In ACL. find
  16. Valerie Hajdik, Jan Buys, Michael Wayne Goodman, and Emily M Bender. 2019. Neural text generation from rich semantic representations. In NAACL. find
  17. Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, and Jianfeng Gao. 2020. Few-shot natural language generation for task-oriented dialog. In arXiv. find
  18. Hamza Harkous, Isabel Groves, and Amir Saffari. 2020. Have your text and use it too! end-to-end neural data-to-text generation with semantic fidelity. arXiv preprint arXiv:2004.06577. find
  19. Mihir Kale. 2020. Text-to-text pre-training for data-to-text tasks. arXiv preprint arXiv:2005.10433. find
  20. Gregg Kellogg, Ivan Herman, and Jeremy Tandy. 2015. Generating RDF from tabular data on the web. W3C recommendation, W3C. Https://www.w3.org/TR/2015/REC-csv2rdf-20151217/. find
  21. Ioannis Konstas and Mirella Lapata. 2012. Unsupervised concept-to-text generation with hypergraphs. In NAACL. find
  22. Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with application to the biography domain. In EMNLP. find our card
  23. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In ACL. find our card
  24. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In ACL. find
  25. Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. CommonGen: A constrained text generation challenge for generative commonsense reasoning. In Findings of EMNLP. find
  26. Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2018. Table-to-text generation by structure-aware seq2seq learning. In AAAI. find
  27. Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019. Step-by-step: Separating planning from realization in neural data-to-text generation. In NAACL. find our card
  28. Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017a. Why we need new evaluation metrics for nlg. In EMNLP. find
  29. Ratish Puduppully, Li Dong, and Mirella Lapata. 2018. Data-to-text generation with content selection and planning. In AAAI. find
  30. Ratish Puduppully, Li Dong, and Mirella Lapata. 2019. Data-to-text generation with entity modeling. In ACL. find
  31. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Technical report, OpenAI. find
  32. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. find
  33. Ehud Reiter. 2007. An architecture for data-to-text systems. In Proceedings of the Eleventh European Workshop on Natural Language Generation (ENLG 07). find
  34. Ehud Reiter. 2020. Openai gpt system: What does it do? Technical report, Arria. find
  35. Ehud Reiter and Robert Dale. 2000. Building natural language generation systems. Cambridge university press. find
  36. Leonardo F. R. Ribeiro, Claire Gardent, and Iryna Gurevych. 2019. Enhancing AMR-to-text generation with dual graph representations. In EMNLP. find
  37. Leonardo F. R. Ribeiro, Martin Schmitt, Hinrich Schütze, and Iryna Gurevych. 2020. Investigating pretrained language models for graph-to-text generation. arXiv. find
  38. Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020. BLEURT: Learning robust metrics for text generation. In ACL. find
  39. Jekaterina Novikova, Ondrej Dusek, and Verena Rieser. 2017b. The E2E dataset: New challenges for end-to-end generation. In SIGDIAL. find our card
  40. Linfeng Song, Xiaochang Peng, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2017. Amr-to-text generation with synchronous node replacement grammar. In ACL. find our card
  41. Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A controlled table-to-text generation dataset. In EMNLP. find our card
  42. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. find
  43. Pavlos Vougiouklis, Hady ElSahar, Lucie-Aimée Kaffee, Christophe Gravier, Frédérique Laforest, Jonathon S. Hare, and Elena Simperl. 2018. Neural wikipedian: Generating textual summaries from knowledge base triples. Journal of Web Semantics, 52-53:1 – 15. find
  44. Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, David Vandyke, and Steve Young. 2016. Conditional generation and snapshot learning in neural dialogue systems. In EMNLP. find
  45. Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In EMNLP. find
  46. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In EMNLP. find our card
  47. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In ICLR. find
  48. Chao Zhao, Marilyn Walker, and Snigdha Chaturvedi. 2020. Bridging the structural gap between encoding and decoding for data-to-text generation. In ACL. find
  49. Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M Meyer, and Steffen Eger. 2019. MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance. In EMNLP. find our card
  50. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103. find