Elamur Research⌕
☰
home / papers / 2019 / Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue

Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue

Paper · 2019 · Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani et al.

Summary

Neural sequence-to-sequence generators for task-oriented dialogue usually take flat slot-value meaning representations (MRs) as input. Such MRs say nothing about discourse structure, so generators cannot be told when to contrast, justify or group information, and they often omit or hallucinate content. Balakrishnan et al. revive an idea from rule-based NLG: hierarchical MRs built from arguments, dialog acts (e.g. INFORM, RECOMMEND) and nestable discourse relations (JOIN, JUSTIFY, CONTRAST). The model produces a bracketed tree alongside the text. This lets the authors apply constrained decoding, which forces the output tree to cover exactly the input content in a valid structure. They also release two resources: a new crowd-annotated weather-domain dataset collected in a conversational setting, and an E2E dataset enriched with tree-structured MRs. On both datasets, tree-based inputs combined with constrained decoding improve semantic correctness, control over CONTRAST, and diversity over flat baselines. They also make models more data-efficient and help them generalize to compositional structures. The work shows that adding explicit structure to neural NLG yields more controllable and reliable dialogue responses.

Key points

  • Weak contrast in E2E systems: In the E2E challenge, only 43 of 10,080 system outputs (0.4%) contained contrastive tokens, although the test set has about 300 contrastive references. This motivates explicit discourse relations in the input.
  • Data efficiency: S2S-Constr (tree MRs plus constrained decoding) reaches over 90% tree accuracy with only 2K training examples and over 95% with 5K, on both the E2E and weather data.
  • Compositional generalization: The E2E training set was split into 26,896 flat and 24,530 compositional examples. When 5% of the compositional examples were added to training, a model also trained on weather data reached 76% tree accuracy, compared with 53.72% for the E2E-only model.
  • Diversity: On E2E, S2S-Constr produces 134 unique tokens, 1,128 unique trigrams and a Shannon entropy of 5.86. The flat-input baselines TGen (83 / 597 / 5.41) and SLUG (74 / 507 / 5.35) score lower.
  • Error analysis and release: In 35 cases where tree accuracy and human judgments disagreed, 63% were traced to noise in the automatic E2E annotation. Code and datasets are available at github.com/facebookresearch/TreeNLG.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MetaMenlo Park · United States158 works · 211,459 citations🎓The Ohio State UniversityColumbus · United States57 works · 17,793 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 3Research subjects and languages the work is about.Dialogue systems · 401 worksDiscourse analysis · 394 worksText generation · 145 works

Cite

Cite as Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani, Michael White and Rajen Subba. 2019. Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1080

@inproceedings{balakrishnan2019constrained,
  title = {{Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue}},
  author = {Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani, Michael White and Rajen Subba},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1080},
  url = {https://doi.org/10.18653/v1/p19-1080}
}

References

49 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martin Kay. 1996. Chart generation. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, pages 200–204, Santa Cruz, California, USA. Association for Computational Linguistics.
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
  3. Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi. 2016. Globally coherent text generation with neural checklist models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 329–339. Association for Computational Linguistics.
  4. Giuseppe Carenini and Johanna D. Moore. 2006. Generating and evaluating evaluative arguments. Artificial Intelligence, 170:925–952. find
  5. John Carroll, Ann Copestake, Dan Flickinger, and Victor Poznański. 1999. An efficient chart generator for (semi-) lexicalist grammars. In Proc. EWNLG-99. find
  6. Diederik P. Kingma and Jimmy Ba. 2015. A method for stochastic optimization. abs/1412.6980. find
  7. John Carroll and Stefan Oepen. 2005. High efficiency realization for a wide-coverage unification grammar. In Proc. IJCNLP-05. find
  8. Adam: CoRR, Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia. Association for Computational Linguistics.
  9. Vera Demberg, Andi Winterboer, and Johanna D Moore. 2011. A strategy for information presentation in spoken dialog systems. Computational Linguistics, 37(3):489–539. find
  10. Ondřej Dušek and Filip Jurcicek. 2016. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 45–51. Association for Computational Linguistics.
  11. Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke Zettlemoyer. 2017. Neural AMR: Sequence-to-sequence models for parsing and generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 146–157, Vancouver, Ca…
  12. Ondrej Dušek and Filip Jurcıcek. 2016. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In The 54th Annual Meeting of the Association for Computational Linguistics, page 45.
  13. Oliver Lemon, Johanna Moore, Mary Ellen Foster, and Michael White. 2004. Generating tailored, comparative descriptions in spoken dialogue. In Proc. of FLAIRS. AAAI. find
  14. Ondřej Dušek, Jekaterina Novikova, and Verena Rieser. 2018. Findings of the E2E NLG challenge. In Proceedings of the 11th International Conference on Natural Language Generation, pages 322–328. Association for Computational Linguistics. find
  15. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 9…
  16. Ondřej Dušek, Jekaterina Novikova, and Verena Rieser. 2019. Evaluating the state-of-the-art of end-to-end natural language generation: The E2E NLG Challenge. arXiv preprint arXiv:1901.11528. find
  17. William C. Mann and Sandra A. Thompson. 1988. Rhetorical structure theory: Towards a functional theory of text organization. TEXT, 8(3):243–281. find
  18. Henry Elder, Sebastian Gehrmann, Alexander O’Connor, and Qun Liu. 2018. E2E NLG challenge submission: Towards controllable generation of diverse natural language. In Proceedings of the 11th International Conference on Natural Language Generation, pages 457–462. find
  19. Christopher D. Manning and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA, USA. find
  20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  21. Hongyuan Mei, Mohit Bansal, and R. Matthew Walter. 2016. What to talk about and how? selective generation using lstms with coarse-to-fine alignment. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page…
  22. Juraj Juraska, Panagiotis Karagiannis, Kevin K. Bowden, and Marilyn A. Walker. 2018. A deep ensemble model with slot alignment for sequence-to-sequence natural language generation. In NAACL-HLT. find
  23. Johanna D. Moore and Cécile Paris. 1993. Planning text for advisory dialogues: Capturing intentional and rhetorical information. Computational Linguistics, 19:651–694. find
  24. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional Sequence to Sequence Learning. ArXiv e-prints.
  25. Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019. Step-by-step: Separating planning from realization in neural data-to-text generation. arXiv preprint arXiv:1904.03396.
  26. Amanda Stent, Marilyn A. Walker, Steve Whittaker, and Preetam Maloor. 2002. User-tailored generation for spoken dialogue: an experiment. In INTER-SPEECH. find
  27. Hiroko Nakanishi, Yusuke Miyao, and Jun’ichi Tsujii. 2005. Probabilistic methods for disambiguation of an HPSG-based chart generator. In Proc. IWPT-05. find
  28. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. find
  29. Jekaterina Novikova, Ondrej Dusek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In EMNLP.
  30. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Advances in Neural Information Processing Systems, pages 2692–2700. find
  31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proc. ACL-02.
  32. Marilyn Walker, Amanda Stent, Francois Mairesse, and Rashmi Prasad. 2007. Individual and domain adaptation in sentence planning for dialogue. Journal of Artificial Intelligence Research (JAIR), 30:413–456. find
  33. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  34. Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, M. Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, and Steve Young. 2016. Multi-domain neural network language generation for spoken dialogue systems. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computat…
  35. Matt Post and David Vilar. 2018. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. In NAACL-HLT.
  36. Ratish Puduppully, Li Dong, and Mirella Lapata. 2018. Data-to-text generation with content selection and planning. arXiv preprint arXiv:1809.00582. — [arXiv:1809.00582](https://arxiv.org/abs/1809.00582) find
  37. Tsung-Hsien Wen, Milica Gasic, Nikola Mrkšić, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1711–1721. A…
  38. Owen Rambow, Srinivas Bangalore, and Marilyn Walker. 2001. Natural language generation in dialog systems. In Proceedings of the First International Conference on Human Language Technology Research, HLT ’01, pages 1–4, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  39. Michael White. 2006. Efficient realization of coordinate structures in combinatory categorial grammar. Research on Language and Computation, 4(1):39– 75. find
  40. Lena Reed, Shereen Oraby, and Marilyn Walker. 2018. Can neural generators for dialogue learn sentence planning and discourse structuring? In Proceedings of the 11th International Conference on Natural Language Generation, pages 284–295. Association for Computational Linguistics. find
  41. Michael White, Robert A. J. Clark, and Johanna D. Moore. 2010. Generating tailored, comparative descriptions with contextually appropriate intonation. Computational Linguistics, 36(2):159–201. find
  42. Ehud Reiter. 2018. A structured review of the validity of BLEU. Computational Linguistics, 44:393–401.
  43. Michael White and Rajakrishnan Rajkumar. 2009. Perceptron reranking for CCG realization. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 410–419, Singapore. Association for Computational Linguistics. find
  44. Ehud Reiter and Robert Dale. 2000. Building Natural-Language Generation Systems. Cambridge University Press. find
  45. Stuart Shieber. 1988. A uniform architecture for parsing and generation. In Proc. COLING-88. find
  46. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263. Association for Computational Linguistics.
  47. Charese Smiley, Elnaz Davoodi, Dezhao Song, and Frank Schilder. 2018. The E2E NLG challenge: A tale of two systems. In INLG. find
  48. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. find
  49. Sam Wiseman, Stuart M. Shieber, and Alexander M. Rush. 2018. Learning neural templates for text generation. In EMNLP.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/balakrishnan-2019-constrained-decoding-neural-nlg-compositional/