Elamur Research⌕
☰
home / papers / 2019 / Data-to-text Generation with Entity Modeling

Data-to-text Generation with Entity Modeling

Paper · 2019 · Ratish Puduppully, Li Dong and Mirella Lapata

Summary

Neural encoder-decoder systems that turn structured tables into text usually handle entities such as players and teams as ordinary vocabulary items. This ignores how much the coherence of descriptive writing depends on the way entities are introduced and revisited. Puduppully, Dong and Lapata propose ENT, an entity-centric architecture. It keeps a separate memory vector for every entity and updates it at each decoding step through a gated recurrent processor whose parameters are shared across entities. The decoder uses hierarchical attention: it first looks at the records belonging to each entity, then across the entity memories, and it copies values from the input table.

The authors test the model on the NBA benchmark ROTOWIRE and on MLB, a new baseball corpus they built. MLB is about five times larger, has longer summaries and adds play-by-play event records. ENT outperforms ED+CC, NCP+CC and a template system in both automatic and human evaluation. Unlike NCP+CC, it needs no extracted content plans, which makes it simpler and easier to move to new domains.

Key points

  • MLB dataset: pairs game statistics and play-by-play with human-written summaries. Gold summaries average 542 tokens, roughly twice the length of ROTOWIRE ones. Code and data are available at github.com/ratishsp/data2text-entity-py.
  • Encoding: MLB records are encoded with a bidirectional LSTM because play-by-play is chronological. ROTOWIRE records are encoded with an MLP only.
  • Human evaluation setup: Best-Worst scaling on 30 test summaries per dataset, each judged by 3 raters, covering Grammaticality, Coherence and Conciseness.
  • Human evaluation on MLB: ENT received the top Grammaticality (8.31) and Coherence (6.07) scores and was second on Conciseness (5.39, behind TEMPL). It was also rated above Gold on all three criteria, probably because the gold texts read as verbose.
  • Human evaluation on ROTOWIRE: ENT matched Gold on Coherence (3.70 vs. 3.33) and beat NCP+CC on Grammaticality and Coherence. It also produced fewer contradicting facts than NCP+CC on MLB.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom255 works · 68,771 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 works

🧭 Topics 1Research subjects and languages the work is about.Text generation · 145 works

Cite

Cite as Ratish Puduppully, Li Dong and Mirella Lapata. 2019. Data-to-text Generation with Entity Modeling. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1195

@inproceedings{puduppully2019data,
  title = {{Data-to-text Generation with Entity Modeling}},
  author = {Ratish Puduppully, Li Dong and Mirella Lapata},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1195},
  url = {https://doi.org/10.18653/v1/p19-1195}
}

References

38 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, California.
  2. Antoine Bosselut, Omer Levy, Ari Holtzman, Corin Ennis, Dieter Fox, and Yejin Choi. 2018. Simulating action dynamics with neural process networks. In Proceedings of the 6th International Conference on Learning Representations, ICLR 2018, Vancouver, Canada. find
  3. Marc Cavazza and Fred Charles. 2005. Dialogue generation in character-based interactive storytelling. In Proceedings of the 1st Artificial Intelligence and Interactive Digital Entertainment Conference, pages 21–26, Marina del Rey, California. find
  4. Marc Cavazza, Fred Charles, and Steven J Mead. 2002. Character-based interactive storytelling. IEEE Intelligent Systems, 17(4):17–24. find
  5. Wallace L. Chafe. 1976. Givenness, contrastiveness, definiteness, subjects, topics, a nd point of view. In Charles N. Li, editor, Subject and topic, pages 25– 55. Academic Press, New York. find
  6. Elizabeth Clark, Yangfeng Ji, and Noah A. Smith. 2018. Neural text generation in stories using entity representations as context. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers),… find
  7. Herbert H. Clark and Susan E. Haviland. 1977. Comprehension and the given- new conract. In Roy O. Freedle, editor, Discourse production and comprehension, pages 1–39. Ablex, Norwood, NewJersey. find
  8. John C. Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121–2159. find
  9. Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. J. Artif. Intell. Res., 61:65–170. find
  10. Rodger Kibble and Richard Power. 2004. Optimizing referential coherence in text generation. Computational Linguistics, 30(4):401–416. find
  11. Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi. 2016. Globally coherent text generation with neural checklist models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 329–339, Austin, Texas.
  12. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. In Proceedings of ACL 2017, System Demonstrations, pages 67–72, Vancouver, Canada.
  13. Susumu Kuno. 1972. Functional sentence perspective. Linguistic Inquiry, 3:269–320. find
  14. Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with application to the biography domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1203–1213, Austin, Texas.
  15. Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225.
  16. Jordan J Louviere, Terry N Flynn, and Anthony Alfred John Marley. 2015. Best-worst scaling: Theory, methods and applications. Cambridge University Press. find
  17. Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. 2016. Pointing the unknown words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 140– 149, Berlin, Germany.
  18. Jordan J Louviere and George G Woodworth. 1991. Best-worst scaling: A model for the largest difference judgments. University of Alberta: Working Paper. find
  19. M. A. K. Halliday and Ruqaiya Hasan. 1976. Cohesion in English. Longman, London. find
  20. Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2017. Tracking the world state with recurrent entity networks. In Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Toulon, France. Open-Review.net. find
  21. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal.
  22. William C. Mann and Sandra A. Thomson. 1988. Rhetorical structure theory. Text, 8(3):243–281. find
  23. Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, and Noah A. Smith. 2017. Dynamic entity representations in neural language models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1831– 1840, Copenhagen, Denmark.
  24. Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. 2016. What to talk about and how? selective generation using lstms with coarse-to-fine alignment. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page…
  25. Nikiforos Karamanis, Massimo Poesio, Chris Mellish, and Jon Oberlander. 2004. Evaluating centering-based metrics of coherence. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04). find
  26. Bryan Orme. 2009. Maxdiff analysis: Simple counting, individual-level logit, and hb. Sawtooth Software. find
  27. Lauri Karttunen. 1976. Discourse referents. In James D. McCawley, editor, Syntax and Semantics: Notes from the Linguistic Underground, volume 7, pages 363–86. Academic Press, New York. find
  28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
  29. Laura Perez-Beltrachini and Mirella Lapata. 2018. Bootstrapping generators from noisy data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1516–1527, New Orleans, Louisia… find
  30. Ellen Prince. 1981. Toward a taxonomy of given-new information. In Peter Cole, editor, Radical Pragmatics, pages 223–255. Academic Press, New York/London. find
  31. Ratish Puduppully, Li Dong, and Mirella Lapata. 2019. Data-to-text generation with content selection and planning. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, Hawaii. find
  32. Ehud Reiter and Robert Dale. 1997. Building applied natural language generation systems. Natural Language Engineering, 3(1):57–87. find
  33. Ehud Reiter and Robert Dale. 2000. Building natural language generation systems. Cambridge University Press, New York, NY. find
  34. Jacques Robin. 1994. Revision-based generation of Natural Language Summaries providing historical Background. Ph.D. thesis, Ph. D. thesis, Columbia University. find
  35. Donia Scott and Clarisse Sieckenius de Souza. 1990. Getting the message across in RST-based text generation. In Robert Dale, Chris Mellish, and Michael Zock, editors, Current Research in Natural Language Generation, pages 47–73. Academic Press, New York. find
  36. Kumiko Tanaka-Ishii, Koiti Hasida, and Itsuki Noda. 1998. Reactive content selection in the generation of real-time soccer commentary. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 2,… find
  37. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263, Copenhagen, Denmark.
  38. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 14…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/puduppully-2019-data-text-generation-entity-modeling/