Dense Passage Retrieval for Open-Domain Question Answering

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Computing & GPUs

Cited by

1982 papers of our library cite it (of the 140673 whose bibliographies we read); 95 of them have a page here, 1887 not yet.

• Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training
• Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
• On the Opportunities and Risks of Foundation Models
• LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
• Adapting Language Models to Compress Contexts
• Promptagator: Few-shot Dense Retrieval From 8 Examples
• Mind2Web: Towards a Generalist Agent for the Web
• CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
• SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking
• SimCSE: Simple Contrastive Learning of Sentence Embeddings
• Precise Zero-Shot Dense Retrieval without Relevance Labels
• Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
• Enabling Large Language Models to Generate Text with Citations
• RARR: Researching and Revising What Language Models Say, Using Language Models
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators
• Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
• Unsupervised Dense Information Retrieval with Contrastive Learning
• Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Evaluating Open-Domain Question Answering in the Era of Large Language Models
• Large Language Models Struggle to Learn Long-Tail Knowledge
• RealTime QA: What's the Answer Right Now?
• UNIFIEDQA: Crossing Format Boundaries with a Single QA System
• Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
• Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering
• Internet-Augmented Dialogue Generation
• Hurdles to Progress in Long-form Question Answering
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
• Internet-augmented language models through few-shot prompting for open-domain question answering
• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
• PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• RA-DIT: Retrieval-Augmented Dual Instruction Tuning
• What Makes Good In-Context Examples for GPT-3?
• Entity-Based Knowledge Conflicts in Question Answering
• Sparse, Dense, and Attentional Representations for Text Retrieval
• Fine-Tuning LLaMA for Multi-Stage Text Retrieval
• Query Rewriting in Retrieval-Augmented Large Language Models
• Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
• Generation-Augmented Retrieval for Open-domain Question Answering
• Teaching language models to support answers with verified quotes
• Generative Representational Instruction Tuning
• WebGPT: Browser-assisted question-answering with human feedback
• Text and Code Embeddings by Contrastive Pre-Training
• Large Dual Encoders Are Generalizable Retrievers
• QuALITY: Question Answering with Long Input Texts, Yes!
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
• KILT: a Benchmark for Knowledge Intensive Language Tasks
• PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations
• RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
• Learning To Retrieve Prompts for In-Context Learning
• Improving Passage Retrieval with Zero-Shot Question Generation
• ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
• REPLUG: Retrieval-Augmented Black-Box Language Models
• Retrieval Augmentation Reduces Hallucination in Conversation
• Prompting GPT-3 To Be Reliable
• ASQA: Factoid Questions Meet Long-Form Answers
• One Embedder, Any Task: Instruction-Finetuned Text Embeddings
• Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents
• Transformer Memory as a Differentiable Search Index
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• LaMDA: Language Models for Dialog Applications
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Query2doc: Query Expansion with Large Language Models
• Improving Text Embeddings with Large Language Models
• Multilingual E5 Text Embeddings: A Technical Report
• Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
• Scalable Zero-shot Entity Linking with Dense Entity Retrieval
• C-Pack: Packed Resources For General Chinese Embeddings
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• Benchmarking Retrieval-Augmented Generation for Medicine
• Retrieval meets Long Context Large Language Models
• LeanDojo: Theorem Proving with Retrieval-Augmented Language Models
• It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story Books
• LinkBERT: Pretraining Language Models with Document Links
• Making Retrieval-Augmented Language Models Robust to Irrelevant Context
• Generate rather than Retrieve: Large Language Models are Strong Context Generators
• IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositions
• BARTScore: Evaluating Generated Text as Text Generation
• MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
• Large Language Models for Information Retrieval: A Survey
• ToolQA: A Dataset for LLM Question Answering with External Tools

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.550 · arXiv 2004.04906

@inproceedings{karpukhin2020dense,
  title = {{Dense Passage Retrieval for Open-Domain Question Answering}},
  author = {Karpukhin, Vladimir and Oguz, Barlas and Min, Sewon and Lewis, Patrick and Wu, Ledell and Edunov, Sergey and Chen, Danqi and Yih, Wen-tau},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.550},
  eprint = {2004.04906},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.550}
}

References

47 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. 2020. Learning to retrieve reasoning paths over Wikipedia graph for question answering. In International Conference on Learning Representations (ICLR). find our card
2. Petr Baudiš and Jan Šedivỳ. 2015. Modeling of the question answering task in the yodaqa system. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 222– 228. Springer. find
3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from clickthrough data. In ACM International Conference on Information and Knowledge Management (CIKM), pages 2333–2338. question-answer pairs. In Empirical Methods in Natural Language Processing (EMNLP). find
4. Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. 1994. Signature verification using a “Siamese” time delay neural network. In NIPS, pages 737–744. find
5. Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning, pages 89–96. find our card
6. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In International Conference on Learning Representations (ICLR). find
7. Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering. ArXiv, abs/2007.01282. find our card
8. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Association for Computational Linguistics (ACL), pages 1870–1879. find our card
9. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with GPUs. ArXiv, abs/1702.08734. find our card
10. Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. 2019. Multi-step retriever-reader interaction for scalable open-domain question answering. In International Conference on Learning Representations (ICLR). find
11. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Association for Computational Linguistics (ACL), pages 1601–1611. find our card
12. Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. 1990. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391–407. find
13. Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and effective passage search via contextualized late interaction over BERT. In ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 39–48. find our card
14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Association for Computational Linguistics (NAACL). find
15. Brian Kulis. 2013. Metric learning: A survey. Foundations and Trends in Machine Learning, 5(4):287– 364. find
16. David A Ferrucci. 2012. Introduction to “This is Watson”. IBM Journal of Research and Development, 56(3.4):1–1. find
17. Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. 2019. Learning dense representations for entity retrieval. In Computational Natural Language Learning (CoNLL). find
18. Ruiqi Guo, Sanjiv Kumar, Krzysztof Choromanski, and David Simcha. 2016. Quantization based fast inner product search. In Artificial Intelligence and Statistics, pages 482–490. find
19. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-augmented language model pre-training. ArXiv, abs/2002.08909. find
20. Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-hsuan Sung, László Lukács, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart reply. ArXiv, abs/1705.00652. — [arXiv:1705.00652](https://arxiv.org/abs/1705.00652) find
21. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for Web search using Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red-field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey,… find
22. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Association for Computational Linguistics (ACL), pages 6086–6096. find our card
23. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Association for Computational Linguistics (A… find our card
24. Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information… find
25. Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. 2018. Denoising distantly supervised open-domain question answering. In Association for Computational Linguistics (ACL), pages 1736–1745. find
26. Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019a. A discrete hard EM approach for weakly supervised question answering. In Empirical Methods in Natural Language Processing (EMNLP). find
27. Sewon Min, Danqi Chen, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019b. Knowledge guided text retrieval and reading for open domain question answering. ArXiv, abs/1911.03868. find
28. Dan Moldovan, Marius Paşca, Sanda Harabagiu, and Mihai Surdeanu. 2003. Performance issues and error analysis in an open-domain question answering system. ACM Transactions on Information Systems (TOIS), 21(2):133–154. find
29. Stephen Mussmann and Stefano Ermon. 2016. Learning and inference via maximum inner product search. In International Conference on Machine Learning (ICML), pages 2587–2596. find
30. Yixin Nie, Songhe Wang, and Mohit Bansal. 2019. Revealing the importance of semantic retrieval for machine reading at scale. In Empirical Methods in Natural Language Processing (EMNLP). find
31. Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. ArXiv, abs/1901.04085. find our card
32. Anshumali Shrivastava and Ping Li. 2014. Asymmetric LSH (ALSH) for sublinear time maximum inner product search (MIPS). In Advances in Neural Information Processing Systems (NIPS), pages 2321– 2329. find
33. Ellen M Voorhees. 1999. The TREC-8 question answering track report. In TREC, volume 99, pages 77–82. find
34. Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerry Tesauro, Bowen Zhou, and Jing Jiang. 2018. Rˆ3: Reinforced ranker-reader for open-domain question answering. In Conference on Artificial Intelligence (AAAI). find
35. Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, and Bing Xiang. 2019. Multi-passage BERT: A globally normalized bert model for open-domain question answering. In Empirical Methods in Natural Language Processing (EMNLP). find
36. Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gardner, Yoav Goldberg, Daniel Deutch, and Jonathan Berant. 2020. Break it down: A question understanding benchmark. Transactions of the Association of Computational Linguistics (TACL). find our card
37. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020a. Approximate nearest neighbor negative contrastive learning for dense text retrieval. ArXiv, abs/2007.00808. find our card
38. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. ArXiv, abs/1910.10683. find
39. Wenhan Xiong, Hankang Wang, and William Yang Wang. 2020b. Progressively pretrained dense corpus index for open-domain question answering. ArXiv, abs/2005.00038. find
40. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Empirical Methods in Natural Language Processing (EMNLP), pages 2383–2392. find our card
41. Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019a. End-to-end open-domain question answering with bertserini. In North American Association for Computational Linguistics (NAACL), pages 72–77. find our card
42. Parikshit Ram and Alexander G Gray. 2012. Maximum inner-product search using cone trees. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 931–939. find
43. Wei Yang, Yuqing Xie, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019b. Data augmentation for bert fine-tuning in open-domain question answering. ArXiv, abs/1904.06652. — [arXiv:1904.06652](https://arxiv.org/abs/1904.06652) find
44. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? ArXiv, abs/2002.08910. find
45. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. find our card
46. Wen-tau Yih, Kristina Toutanova, John C Platt, and Christopher Meek. 2011. Learning discriminative projections for text similarity measures. In Computational Natural Language Learning (CoNLL), pages 247–256. find
47. Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, and Hannaneh Hajishirzi. 2019. Real-time open-domain question answering with dense-sparse phrase index. In Association for Computational Linguistics (ACL). find