Elamur Research⌕
☰
home / papers / 2019 / Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

Paper · 2019 · Sewon Min, Danqi Chen, Luke Zettlemoyer et al.

Summary

Open-domain question answering usually depends on text corpora, which cover many facts, while knowledge bases support structured inference but are less complete. This paper by Min, Chen, Zettlemoyer and Hajishirzi shows how a knowledge base can guide both passage retrieval and passage reading. Wikipedia serves as the corpus and Wikidata as the knowledge base.

The pipeline has two parts:

  • GraphRetriever builds a graph of passages. It starts from seed passages found through entity linking and TF-IDF, then grows the graph using Wikidata relations and same-article links.
  • GraphReader is a BERT-based reader. Instead of scoring each passage separately, it refines passage representations by passing information along the graph edges and their relation labels.

Tested on WebQuestions, Natural Questions and TriviaQA, the system beats non-graph baselines by 2–11% absolute. It matches or surpasses the state of the art without costly end-to-end training. Ablations show that both the retriever and the reader contribute to the gains. The work shows that knowledge-base structure can improve text-based QA, both by finding more relevant evidence and by combining information across passages.

Key points

  • Graph construction: Passages are vertices. Edges are either Wikidata relations (e.g., part of, performer) or child/parent links between an article's first passage and its other passages.
  • Retrieval procedure: The seed set combines entity-linked articles with the top-K TF-IDF articles. Expansion runs iteratively until n passages are collected (n = 40 or 80), and supporting passages within an article are ranked with BM25.
  • Reader design: GraphReader encodes each question–passage pair with BERT and encodes relations separately. It then applies M fusion layers to propagate information across linked passages.
  • Baseline comparison: Concatenation baselines (ParReader++ over passage pairs, limited to 300 tokens) did not compete with GraphReader. The likely cause is information lost when passages are truncated.
  • Qualitative examples: In one case, the link connecting the "Director of the United States Mint" passage to the "United States Mint" passage surfaced evidence that text matching missed. In another, the is located in relation helped resolve confusion between similar place names.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States402 works · 212,762 citations🏢MetaMenlo Park · United States214 works · 241,492 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States304 works · 106,138 citations🎓Princeton UniversityPrinceton · United States113 works · 48,612 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

🧭 Topics 2Research subjects and languages the work is about.Question answering · 487 worksInformation retrieval & search · 552 works

Cite

Cite as Sewon Min, Danqi Chen, Luke Zettlemoyer and Hannaneh Hajishirzi. 2019. Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering. arXiv:1911.03868.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1911.03868 · arXiv 1911.03868

@misc{min2019knowledge,
  title = {{Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering}},
  author = {Sewon Min, Danqi Chen, Luke Zettlemoyer and Hannaneh Hajishirzi},
  year = {2019},
  doi = {10.48550/arxiv.1911.03868},
  eprint = {1911.03868},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1911.03868}
}

References

43 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chris Alberti, Kenton Lee, and Michael Collins. 2019. A BERT baseline for the Natural Questions. arXiv preprint arXiv:1901.08634. — [arXiv:1901.08634](https://arxiv.org/abs/1901.08634) find
  2. Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. 2020. Learning to retrieve reasoning paths over wikipedia graph for question answering. In ICLR.
  3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In EMNLP.
  4. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019. Question answering by reasoning across documents with graph convolutional networks. In NAACL.
  5. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In ACL.
  6. Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehension. In ACL.
  7. Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. 2019. Multi-step retriever-reader interaction for scalable open-domain question answering. In ICLR.
  8. Rajarshi Das, Manzil Zaheer, Siva Reddy, and Andrew McCallum. 2017. Question answering on knowledge bases and text using universal schema and memory networks. In ACL. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL.
  10. Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. 2019. Cognitive graph for multi-hop reading comprehension at scale. In ACL. find
  11. Paolo Ferragina and Ugo Scaiella. 2011. Fast and accurate annotation of short texts with wikipedia pages. IEEE software. find
  12. David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A Kalyanpur, Adam Lally, J William Murdock, Eric Nyberg, John Prager, et al. 2010. Building Watson: An overview of the DeepQA project. AI magazine, 31(3):59–79. find
  13. Ameya Godbole, Dilip Kavarthapu, Rajarshi Das, Zhiyu Gong, Abhishek Singhal, Hamed Zamani, Mo Yu, Tian Gao, Xiaoxiao Guo, Manzil Zaheer, et al. 2019. Multi-step entity-centric information retrieval for multi-hop question answering. In Workshop on Machine Reading for Question Answering EMNLP. find
  14. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In ACL.
  15. Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
  16. Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, and Luke Zettlemoyer. 2013. Scaling semantic parsers with on-the-fly ontology matching. In EMNLP.
  17. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Change, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  18. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In ACL.
  19. Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. 2018. Denoising distantly supervised open-domain question answering. In ACL.
  20. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In EMNLP.
  21. Todor Mihaylov and Anette Frank. 2018. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge. In ACL.
  22. Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019a. A discrete hard EM approach for weakly supervised question answering. In EMNLP.
  23. Sewon Min, Eric Wallace, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019b. Compositional questions do not necessitate multi-hop reasoning. In ACL.
  24. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. find
  25. Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends R in Information Retrieval. find
  26. Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P Parikh, Ali Farhadi, and Hannaneh Hajishirzi. 2019. Real-time open-domain question answering with dense-sparse phrase index. In ACL.
  27. Linfeng Song, Zhiguo Wang, Mo Yu, Yue Zhang, Radu Florian, and Daniel Gildea. 2018. Exploring graph-structured passage representation for multi-hop reading comprehension with graph neural networks. arXiv preprint arXiv:1809.02040. — [arXiv:1809.02040](https://arxiv.org/abs/1809.02040) find
  28. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research.
  29. Haitian Sun, Tania Bedrax-Weiss, and William W Cohen. 2019. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. In EMNLP. find
  30. Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William W Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In EMNLP. find
  31. Swabha Swayamdipta, Ankur P Parikh, and Tom Kwiatkowski. 2018. Multi-mention learning for reading comprehension with neural cascades. In ICLR. find
  32. Ellen M Voorhees et al. 1999. The TREC-8 question answering track report. In Trec.
  33. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledge base. find
  34. Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerry Tesauro, Bowen Zhou, and Jing Jiang. 2018. R 3 : Reinforced ranker-reader for open-domain question answering. In AAAI. find
  35. Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, and Bing Xiang. 2019. Multi-passage BERT: A globally normalized BERT model for open-domain question answering. In ACL.
  36. Dirk Weissenborn, Tomáš Kočiskỳ, and Chris Dyer. 2017. Dynamic integration of background knowledge in neural NLU systems. arXiv preprint arXiv:1706.02596. — [arXiv:1706.02596](https://arxiv.org/abs/1706.02596) find
  37. Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2017. Constructing Datasets for Multi-hop Reading Comprehension Across Documents. In TACL.
  38. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace’s Transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  39. Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2019a. Improving question answering over incomplete kbs with knowledge-aware reader. In ACL. find
  40. Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Hong Wang, Shiyu Chang, Murray Campbell, and William Yang Wang. 2019b. Simple yet effective bridge reasoning for open-domain multi-hop question answering. In 2nd Workshop on Machine Reading for Question Answering. find
  41. Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019. End-to-end open-domain question answering with BERTserini. In NAACL (Demonstrations).
  42. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In EMNLP.
  43. Scott Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic parsing via staged query graph generation: Question answering with knowledge base. In ACL.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/min-2019-knowledge-guided-text-retrieval-reading/