Elamur Research⌕
☰
home / papers / 2020 / Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs

Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs

Summary

Neural dialogue generators often produce bland, repetitive or off-topic replies. Earlier knowledge-aware systems use external knowledge only for concepts that appear explicitly in the input and treat it as a fixed memory. Real conversations, however, drift toward neighbouring and more distant ideas. The authors propose ConceptFlow, which links words in a post to ConceptNet concepts and expands them into a graph for each conversation. The central part holds the mentioned concepts and their direct neighbours, and the outer part reaches two hops away. The central graph is encoded with the GraftNet graph neural network, and outer hops are summarized through attention. A GRU decoder then uses a gate to choose whether each output token is an ordinary word, a central concept or a two-hop concept. On Reddit dialogue data, ConceptFlow beat seq2seq baselines, earlier commonsense-based models such as CCM, and two fine-tuned GPT-2 systems in both automatic and human evaluation. It did so with about 70% fewer parameters than the GPT-2 systems. The work shows that explicitly modelling topic shifts through a knowledge graph is an efficient way to get more informative responses.

Key points

  • Graph construction: concepts mentioned in the post (zero-hop) are expanded with one-hop and two-hop ConceptNet neighbours. Zero-hop and one-hop concepts form the central graph, and one-hop and two-hop concepts form the outer graph.
  • Encoding and attention: central concepts are encoded with GraftNet (Sun et al., 2018). Graph attention guides the traversal toward relevant distant concepts.
  • Evaluation: experiments use Reddit conversations, with baselines including seq2seq, CCM and other static-memory knowledge models, plus two fine-tuned GPT-2 variants.
  • Analysis:
    • Many Reddit dialogues follow paths in ConceptNet.
    • Adding two-hop concepts improves informativeness and reduces repetition.
    • The learned attention favours concepts that appear in the reference response.
  • Code: the implementation is released at github.com/thunlp/ConceptFlow (ACL 2020).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Reasoning · Knowledge graphs & ontologies

Institutions

🏢MicrosoftRedmond · United States193 works · 110 465 citations🎓Tsinghua UniversityBeijing · China92 works · 77 045 citations🎓Brown UniversityProvidence · United States40 works · 21 721 citations🔬State Key Laboratory of Intelligent Technology and SystemsBeijing14 works · 8 004 citations

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.184

@inproceedings{zhang2020grounded,
  title = {{Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs}},
  author = {Zhang, Houyu and Liu, Zhenghao and Xiong, Chenyan and Liu, Zhiyuan},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.184},
  url = {https://doi.org/10.18653/v1/2020.acl-main.184}
}

References

43 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR. find our card
  2. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pages 2787–2795. find
  3. Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. 2017. A survey on dialogue systems: Recent advances and new frontiers. SIGKDD Explorations, pages 25–35. find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL, pages 4171– 4186. find our card
  5. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, pages 138–145. find
  6. Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Proceedings of NeurIPS, pages 13042–13054. find our card
  7. Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, and Mari Ostendorf. 2018. Sounding Board: A user-centric and content-driven social chatbot. In Proceedings of NAACL, pages 96–100. find
  8. Michel Galley, Chris Brockett, Xiang Gao, Bill Dolan, and Jianfeng Gao. 2018. End-to-end conversation modeling: Moving beyond chitchat. find
  9. Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. 2019. Jointly optimizing diversity and relevance in neural response generation. In Proceedings of NAACL, pages 1229–1238. find
  10. Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018. A knowledge-grounded neural conversation model. In Proceedings of AAAI, pages 5110–5117. find
  11. Barbara J. Grosz and Candace L. Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, pages 175–204. find our card
  12. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of ACL, pages 1631–1640. find our card
  13. Hiroaki Hayashi, Zecong Hu, Chenyan Xiong, and Graham Neubig. 2019. Latent relation language models. arXiv preprint arXiv:1908.07690. — [arXiv:1908.07690](https://arxiv.org/abs/1908.07690) find
  14. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751. find
  15. Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231. find our card
  16. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of NAACL, pages 110–119. find our card
  17. Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016b. Deep reinforcement learning for dialogue generation. In Proceedings of EMNLP, pages 1192–1202. find our card
  18. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81. find our card
  19. Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. 2018. Knowledge diffusion for neural dialogue generation. In Proceedings of the ACL, pages 1489–1498. find
  20. Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. 2019. Barack’s wife hillary: Using knowledge graphs for fact-aware language modeling. In Proceedings of ACL, pages 5962–5971. find
  21. Yinong Long, Jianan Wang, Zhen Xu, Zongsheng Wang, Baoxun Wang, and Zhuoran Wang. 2017. A knowledge enhanced generative conversational service agent. In DSTC6 Workshop. find
  22. Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. OpenDialKG: Explainable conversational reasoning with attention-based walks over knowledge graphs. In Proceedings of ACL, pages 845–854. find our card
  23. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of ACL, pages 311–318. find our card
  24. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP, pages 1532– 1543. find our card
  25. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. In Proceedings of Technical report, OpenAI. find
  26. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog. find
  27. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019. Neural text generation with unlikelihood training. arXiv preprint arXiv:1908.04319. find our card
  28. Corbin Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, and Paul Bennett. 2020. Leading conversational search by suggesting useful questions. In Proceedings of The Web Conference 2020, pages 1160–1170. find
  29. Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. In Proceedings of ACL, pages 808–819. find
  30. Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of AAAI, pages 3776–3784. find our card
  31. Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In Proceedings of AAAI, pages 3351–3357. find
  32. Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In Proceedings of ACL, pages 1577–1586. find our card
  33. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of AAAI, pages 4444–4451. find
  34. Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of EMNLP, pages 4231–4242. find
  35. Zhen Xu, Bingquan Liu, Baoxun Wang, Chengjie Sun, and Xiaolong Wang. 2017. Incorporating loose-structured knowledge into conversation modeling via recall-gate LSTM. In 2017 International Joint Conference on Neural Networks, IJCNN, pages 3506–3513. find
  36. Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. In Proceedings of NeurIPS, pages 1810–1820. find
  37. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS, pages 3104–3112. find
  38. Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018a. Commonsense knowledge aware conversation generation with graph attention. In Proceedings of IJCAI, pages 4623–4629. find
  39. Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric Xing, and Zhiting Hu. 2019. Target-guided open-domain conversation. In Proceedings of ACL, pages 5624–5634. find
  40. Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. 2018b. A dataset for document grounded conversations. In Proceedings of EMNLP, pages 708–713. find our card
  41. Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869. find our card
  42. Wenya Zhu, Kaixiang Mo, Yu Zhang, Zhangbin Zhu, Xuezheng Peng, and Qiang Yang. 2017. Flexible end-to-end dialogue system for knowledge grounded conversation. arXiv preprint arXiv:1709.04264. — [arXiv:1709.04264](https://arxiv.org/abs/1709.04264) find
  43. Pavlos Vougiouklis, Jonathon Hare, and Elena Simperl. 2016. A neural network approach for knowledge-driven response generation. In Proceedings of COL-ING, pages 3370–3380. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2020-grounded-conversation-generation-as-guided/