TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

1601 papers of our library cite it (of the 140673 whose bibliographies we read); 134 of them have a page here, the first 100 below, 1467 not yet.

• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• Phi-4 Technical Report
• Muppet: Massive Multi-task Representations with Pre-Finetuning
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
• PaLM 2 Technical Report
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
• The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• STARC: Structured Annotations for Reading Comprehension
• GPT-NeoX-20B: An Open-Source Autoregressive Language Model
• Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
• InternLM2 Technical Report
• PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
• A Survey on Evaluation of Large Language Models
• HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
• QuAC: Question Answering in Context
• Simple and Effective Multi-Paragraph Reading Comprehension
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
• DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
• DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
• Moshi: a speech-text foundation model for real-time dialogue
• Event Extraction by Answering (Almost) Natural Questions
• GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• English Machine Reading Comprehension Datasets: A Survey
• TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
• ELI5: Long Form Question Answering
• HeQ: a Large and Diverse Hebrew Reading Comprehension Benchmark
• Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
• Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
• Retrieval-Augmented Generation for Large Language Models: A Survey
• Improving alignment of dialogue agents via targeted human judgements
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
• LongT5: Efficient Text-To-Text Transformer for Long Sequences
• DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
• TrustLLM: Trustworthiness in Large Language Models
• Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
• Unsupervised Dense Information Retrieval with Contrastive Learning
• Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity
• What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• SpanBERT: Improving Pre-training by Representing and Predicting Spans
• Large Language Models Struggle to Learn Long-Tail Knowledge
• Dense Passage Retrieval for Open-Domain Question Answering
• RealTime QA: What's the Answer Right Now?
• How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
• Kimi K2: Open Agentic Intelligence
• Natural Questions: A Benchmark for Question Answering Research
• Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds
• Latent Retrieval for Weakly Supervised Open Domain Question Answering
• MLQA: Evaluating Cross-lingual Extractive Question Answering
• PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them
• Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
• Large Language Models with Controllable Working Memory
• DataComp-LM: In search of the next generation of training sets for language models
• Search-o1: Agentic Search-Enhanced Large Reasoning Models
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• RA-DIT: Retrieval-Augmented Dual Instruction Tuning
• LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
• What Makes Good In-Context Examples for GPT-3?
• AgentBench: Evaluating LLMs as Agents
• LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
• Generation-Augmented Retrieval for Open-domain Question Answering
• DocVQA: A Dataset for VQA on Document Images
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Teaching language models to support answers with verified quotes
• Generative Representational Instruction Tuning
• WebGPT: Browser-assisted question-answering with human feedback
• A Comprehensive Overview of Large Language Models
• Text and Code Embeddings by Contrastive Pre-Training
• Passage Re-ranking with BERT
• QuALITY: Question Answering with Long Input Texts, Yes!
• KILT: a Benchmark for Knowledge Intensive Language Tasks
• PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations
• DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
• Know What You Don't Know: Unanswerable Questions for SQuAD
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• A guide to the dataset explosion in QA, NLI, and commonsense reasoning
• Improving Passage Retrieval with Zero-Shot Question Generation
• DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension
• Multitask Prompted Training Enables Zero-Shot Task Generalization
• ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
• Toolformer: Language Models Can Teach Themselves to Use Tools
• Towards Understanding Sycophancy in Language Models
• REPLUG: Retrieval-Augmented Black-Box Language Models
• Retrieval Augmentation Reduces Hallucination in Conversation
• BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
• Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• ASQA: Factoid Questions Meet Long-Form Answers
• One Embedder, Any Task: Instruction-Finetuned Text Embeddings
• Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p17-1147 · arXiv 1705.03551

@inproceedings{joshi2017triviaqa,
  title = {{TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension}},
  author = {Joshi, Mandar and Choi, Eunsol and Weld, Daniel and Zettlemoyer, Luke},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1147},
  eprint = {1705.03551},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p17-1147}
}

References

33 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeti… find
2. Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. 2015. Large-scale simple question answering with memory networks. CoRR abs/1506.02075. https://arxiv.org/abs/1506.02075. ↗
3. Jordan Boyd-Graber, Brianna Satinoff, He He, and Hal Daumé III. 2012. Besting the quiz master: Crowdsourcing incremental classification games. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for… find
4. Qingqing Cai and Alexander Yates. 2013. Large-scale semantic parsing via schema matching and lexicon extension. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Sofia, Bulgaria, pages 423–43… find
5. Danqi Chen, Jason Bolton, and Christopher D. Manning. 2016. A thorough examination of the cnn/daily mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin,… find our card
6. Matthew Dunn, Levent Sagun, Mike Higgins, Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017. Searchqa: A new q&a dataset augmented with context from a search engine. CoRR https://arxiv.org/abs/1704.05179. ↗ our card
7. Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2014. Open question answering over curated and extracted knowledge bases. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, KDD ’14, pages 1156–1165. https://doi.org/10.114… — [doi:10.1145/2623330.2623677](https://doi.org/10.1145/2623330.2623677) find
8. Paolo Ferragina and Ugo Scaiella. 2010. Tagme: On-the-fly annotation of short text fragments (by wikipedia entities). In Proceedings of the 19th ACM International Conference on Information and Knowledge Management. ACM, New York, NY, USA, CIKM ’10, pages 1625–1628. https://doi.org/10.1145/1871437.18… ↗ our card
9. David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, Nico Schlaefer, and Chris Welty. 2010. Building watson: An overview of the deepqa project. AI MAGAZINE 31(3):59–79. find
10. He He, Jordan Boyd-Graber, Kevin Kwok, and Hal Daumé III. 2016. Opponent modeling in deep reinforcement learning. In Maria Flo-rina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning. PMLR, New York, New York, USA, volume 48 of Proceedings… find
11. Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems. http://arxiv.org/abs/1506.03340. ↗ our card
12. Technologies. Association for Computational Linguistics, Portland, Oregon, USA, pages 541–550. http://www.aclweb.org/anthology/P11-1055. Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daumé III. 2014. A neural network for factoid question answering over paragraphs. In P… ↗
13. Mandar Joshi, Uma Sawant, and Soumen Chakrabarti. 2014. Knowledge graph and corpus driven segmentation and answer inference for telegraphic entity-seeking queries. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguis… find
14. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. Race: Large-scale reading comprehension dataset from examinations. CoRR https://arxiv.org/abs/1704.04683. ↗ our card
15. Peng Li, Wei Li, Zhengyan He, Xuguang Wang, Ying Cao, Jie Zhou, and Wei Xu. 2016. Dataset and neural recurrent sequence labeling model for open-domain factoid question answering. CoRR https://arxiv.org/abs/1607.06275. — [arXiv:1607.06275](https://arxiv.org/abs/1607.06275) ↗
16. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In Workshop in Advances in Neural Information Processing Systems. https://arxiv.org/pdf/1611.09268.pdf. ↗ our card
17. Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2016. Who did what: A large-scale person-centered cloze dataset. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2230… find
18. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2015. The goldilocks principle: Reading children’s books with explicit memory representations. CoRR https://arxiv.org/abs/1511.02301. ↗ our card
19. Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez. 2016. The lambada dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association f… find our card
20. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Panupong Pasupat and… find
21. Conference on Natural Language Processing (Volume 2: Short Papers). Association for Computational Linguistics, Beijing, China, pages 700–706. http://www.aclweb.org/anthology/P15-2115. ↗
22. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2383–2392.… find our card
23. Qiang Wu, Christopher J. Burges, Krysta M. Svore, and Jianfeng Gao. 2010. Adapting boosting for information retrieval measures. Inf. Retr. 13(3):254–270. https://doi.org/10.1007/s10791-009-9112-1. — [doi:10.1007/s10791-009-9112-1](https://doi.org/10.1007/s10791-009-9112-1) ↗
24. Matthew Richardson, Christopher J.C. Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Seattle, Washingto… find our card
25. Sebastian Riedel, Limin Yao, and Andrew Mc-Callum. 2010. Modeling relations and their mentions without labeled text. In Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases: Part III. Springer-Verlag, Berlin, Heidelberg, ECML PKDD’10, pages 148–163. ht… find
26. Uma Sawant and Soumen Chakrabarti. 2013. Learning joint query interpretation and response ranking. In Proceedings of the 22Nd International Conference on World Wide Web. ACM, New York, NY, USA, WWW ’13, pages 1099–1110. https://doi.org/10.1145/2488388.2488484. — [doi:10.1145/2488388.2488484](https://doi.org/10.1145/2488388.2488484) ↗
27. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning. https://arxiv.org/abs/1502.… ↗ our card
28. Yi Yang, Wen-tau Yih, and Christopher Meek. 2015. Wikiqa: A challenge dataset for open-domain question answering. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 2013–2018. http://aclweb.org… ↗ our card
29. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Associat… find our card
30. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In Proceedings of the International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1611.01603. ↗ our card
31. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2016. Newsqa: A machine comprehension dataset. CoRR https://arxiv.org/abs/1611.09830. ↗ our card
32. Ellen M. Voorhees and Dawn M. Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, USA, SIGIR ’00, pages 200–207. https://doi.org/10.1145/345508.345577. — [doi:10.1145/345508.345577](https://doi.org/10.1145/345508.345577) ↗
33. Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2015. Machine comprehension with syntax, frames, and semantics. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint find