OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P19-1081.pdf

Summary

Open-domain dialog agents must decide which related entities to bring up next. A large knowledge graph offers far too many factually valid options, and most would feel unnatural in conversation. The authors treat choosing the next entity as learning walks over a KG conditioned on dialog context. They build OpenDialKG, a crowdsourced set of 15K role-play conversations between a user and an assistant. Every entity mention in it is linked to the KG path that connects it to earlier turns. They also propose DialKG Walker. It encodes the current entities, the sentence and the dialog history, then fuses them with modality attention. An autoregressive decoder attends over relations to trace paths from the mentioned entities. Candidates are ranked by combining these path scores with a zeroshot relevance score in KG embedding space. In automatic and human evaluations, the model outperforms state-of-the-art and rule-based baselines both in-domain and cross-domain. Each prediction comes with an explicit walk path, which makes the model's reasoning interpretable.

Key points

OpenDialKG corpus: about 91K utterances in 15K human-to-human dialogs, collected with ParlAI. It covers scenarios such as chit-chat and recommendations, and each entity mention is manually grounded to a KG path. The authors describe it as the first fully annotated parallel Dialog ↔ KG resource of this kind.
Underlying KG: a common-fact graph with roughly 100K entities and 1.1M facts. Pruning this search space is the central challenge.
Input encoding: entities use KG embeddings (Bordes et al., 2013). Sentences use an attention-based Bi-LSTM with GloVe vectors trained on 6B tokens. Dialog history uses a hierarchical Bi-LSTM with self-attention. The three inputs are combined via modality attention.
Graph decoder and training: the decoder applies relation-level attention at each hop and is supervised with ground-truth walk paths. A hinge rank loss in KG embedding space provides zeroshot scoring for entities unseen in training.
Qualitative findings: the model explores multi-hop relations more often than baselines, which yields more diverse responses. Even when its predicted entity differs from the reference, the predicted relation path is often correct and the entity is still plausible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

143 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 139 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1081

@inproceedings{moon2019opendialkg,
  title = {{OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs}},
  author = {Moon, Seungwhan and Shah, Pararth and Kumar, Anuj and Subba, Rajen},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1081},
  url = {https://doi.org/10.18653/v1/p19-1081}
}

References

31 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Dimitri Kartsaklis, Mohammad Taher Pilehvar, and Nigel Collier. 2018. Mapping text to knowledge graph entities using multi-sense lstms. EMNLP. find
Hannah Bast, Florian Baurle, Bjorn Buchhold, and Elmar Haussmann. 2014. Easy access to the freebase dataset. In WWW. find
Ni Lao, Tom Mitchell, and William W. Cohen. 2011. Random walk inference and learning in a large scale knowledge base. In EMNLP. find
Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. ICLR. find
Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. ACL. find our card
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS. find
Yinong Long, Jianan Wang, Zhen Xu, Zongsheng Wang, Baoxun Wang, and Zhuoran Wang. 2017. A knowledge enhanced generative conversational service agent. In NIPS DSTC6 Workshop. find
Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, and Tom M Mitchell. 2010. Toward an architecture for never-ending language learning. In AAAI. find
A. H. Miller, W. Feng, A. Fisch, J. Lu, D. Batra, A. Bordes, D. Parikh, and J. Weston. 2017. Parlai: A dialog research software platform. EMNLP. find our card
CY Chen, D Yu, W Wen, YM Yang, J Zhang, M Zhou, K Jesse, A Chau, A Bhowmick, S Iyer, G Sreenivasulu, R Cheng, A Bhandare, and Z Yu. 2018. Gunrock: Building a human-like social bot by leveraging large scale real user data. In 2nd Alexa Prize. find
Seungwhan Moon and Jaime Carbonell. 2017. Completely heterogeneous transfer learning with attention: What and what not to transfer. IJCAI. find
Yun-Nung Chen, William Yang Wang, and Alexander Rudnicky. 2015. Jointly modeling inter-slot relations by random walk on knowledge graphs for unsupervised spoken language understanding. In NAACL. find
Seungwhan Moon, Suyoun Kim, and Haohan Wang. 2015. Multimodal transfer deep learning with applications in audio-visual recognition. In NIPS MMML Workshop. find
Seungwhan Moon, Leonard Neves, and Vitor Carvalho. 2018a. Multimodal named entity recognition for short social media posts. NAACL. find
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. find
Supervised Seungwhan Moon, Leonard Neves, and Vitor Carvalho. 2018b. Zeroshot multimodal named entity disambiguation for noisy social media posts. ACL. find
Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic embeddings of knowledge graphs. AAAI. find
Prasanna Parthasarathi and Joelle Pineau. 2018. Extending neural generative conversational model using external knowledge sources. EMNLP. find
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP. find
Siva Reddy, Danqi Chen, and Christopher D Manning. 2018. Coqa: A conversational question answering challenge. arXiv preprint arXiv:1808.07042. find our card
Yasser Salem, Jun Hong, and Weiru Liu. 2014. History-guided conversational recommendation. In WWW. find
Pararth Shah, Dilek Hakkani-Tur, Bing Liu, and Gokhan Tur. 2018. Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning. In NAACL. find
Yueming Sun and Yi Zhang. 2018. Conversational recommender system. SIGIR. find
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS. find
Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela. 2018. Talk the walk: Navigating new york city through grounded dialogue. ECCV. find
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In AAAI. find
Wei Wei, Quoc Le, Andrew Dai, and Jia Li. 2018. Airdialogue: An environment for goal-oriented dialogue research. In EMNLP. find
Jason D Williams, Kavosh Asadi, and Geoffrey Zweig. 2017. Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. ACL. find
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In NAACL. find our card
Tom Young, Erik Cambria, Iti Chaturvedi, Minlie Huang, Hao Zhou, and Subham Biswas. 2018. Augmenting end-to-end dialog systems with commonsense knowledge. AAAI. find
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? ACL. find our card