Elamur Research⌕
☰
home / papers / 2026 / HistoriQA-ThirdRepublic: Multi-Hop Question Answering Corpus for Historical Research, Parliamentary Debates from the French Third Republic (1870-1940)

HistoriQA-ThirdRepublic: Multi-Hop Question Answering Corpus for Historical Research, Parliamentary Debates from the French Third Republic (1870-1940)

Summary

Current RAG and LLM benchmarks mostly use short passages, single documents or multiple-choice formats. They therefore miss the reasoning historians need: linking sources, tracking change over time and assembling scattered evidence from rare archival material. HistoriQA-ThirdRepublic addresses this gap with a French-language QA resource. It pairs digitized parliamentary proceedings of the French Third Republic, focused on a subcorpus around 1887, with contemporary press coverage such as L'Intransigeant. The theoretical framing is agenda-setting.

The questions were generated automatically through controlled prompting. Multi-hop candidates were found by combining embedding similarity, temporal constraints, clustering and metadata. A historian checked every item for factual and contextual soundness. The authors also study how the questions are spread across semantic space, classify their reasoning types and benchmark RAG retrieval and generation. They further compare LLM-as-judge scores against human annotations. The result is a realistic, domain-specific testbed for historical multi-hop QA, and the method can be transferred to other languages and national archives.

Key points

  • The corpus has 1782 questions: 897 single-hop and 885 multi-hop. All were validated by a historian.
  • Sources are French parliamentary debates (Chamber of Deputies and Senate, digitized for 1880–1940) aligned with newspapers of the same dates, e.g. L'Intransigeant's "La Chambre" column.
  • The multi-hop generation prompt requires each question to need both excerpts and allows an explicit "AUCUNE QUESTION" refusal. Every item is labelled with one of five reasoning types (factual, chronological, causal, comparative, critical/inferential) and a difficulty level.
  • The evaluation covers retrieval and generation stages, historian-based quality judgments, and agreement between LLM judges and humans.
  • The dataset is released at github.com/atomegoyan/historiqa-thirdrepublic (LREC 2026).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3xibvwpihfoe

@inproceedings{pellet2026historiqa,
  title = {{HistoriQA-ThirdRepublic: Multi-Hop Question Answering Corpus for Historical Research, Parliamentary Debates from the French Third Republic (1870-1940)}},
  author = {Pellet, Aurelien and Puren, Marie Anna and PEREZ, Julien},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3xibvwpihfoe},
  url = {https://doi.org/10.63317/3xibvwpihfoe}
}

References

21 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Source-oracle LLM Annotation Human Annotation Correct Incorrect Total 20 30 Correct Incorrect 0 14 20 30 20 16 Total Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chia Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, and Na Zou. 2025. Main-rag: Multi-… find
  2. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao,… find
  3. Yufei Guo, Muzhe Guo, Juntao Su, Zhou Yang, Mengqiu Zhu, Hongfei Li, Mengyang Qiu, and Shuo Shuo Liu. 2024. Bias in large language models: Origin, evaluation, and mitigation. find
  4. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. find our card
  5. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, Barcelona, Spain (Online). International Comm… find our card
  6. Seonjeong Hwang, Yunsu Kim, and Gary Geunbae Lee. 2024. Explainable multi-hop question generation: An end-to-end approach without intermediate question labeling. find
  7. Dominique Kalifa, Philippe Régnier, and Marie Ève Thérenty, editors. 2011. La civilisation du journal : histoire culturelle et littéraire de la presse française au XIXe siècle. Nouveau Monde Éditions, Paris. find
  8. Quinn Leng, Jacob Portes, Sam Havens, Matei Zaharia, and Michael Carbin. 2024. Long context rag performance of llms. find
  9. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc… find
  10. Maodong Li, Longyin Zhang, and Fang Kong. 2025a. Multi-hop question generation via dual-perspective keyword guidance. In Findings of the Association for Computational Linguistics: ACL 2025, page 10096–10112, Vienna, Austria. Association for Computational Linguistics. find
  11. Rui Li, Liyang He, Qi Liu, Zheng Zhang, Heng Yu, Yuyang Ye, Linbo Zhu, and Yu Su. 2025b. Unirag: Unified query understanding method for retrieval augmented generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page 14163–14178… find
  12. Zefeng Lin, Weidong Chen, Yan Song, and Yongdong Zhang. 2024. Prompting few-shot multi-hop question generation via comprehending type-aware semantics. In Findings of the Association for Computational Linguistics: NAACL 2024, page 3730–3740, Mexico City, Mexico. Association for Computational Linguist… find
  13. Maxwell E. McCombs and Donald L. Shaw. 1972. The agenda-setting function of mass media. The Public Opinion Quarterly, 36(2):176–187. find
  14. Benjamin Morel. 2024. Le Parlement, temple de la République : de 1789 à nos jours. Passés composés, Paris. find
  15. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… find our card
  16. Aurélien Pellet, Julien Perez, and Marie Puren. 2025. Évaluation automatique du retour à la source dans un contexte historique long et bruité : les débats parlementaires de la Troisième République française. In PFIA 2025, Dijon, France. find
  17. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Dmitry Dodonov, Tung Nguyen, Jaeho Lee,… find our card
  18. Julian Schnitzler, Xanh Ho, Jiahao Huang, Florian Boudin, Saku Sugawara, and Akiko Aizawa. 2024. Morehopqa: More than multi-hop reasoning. find
  19. Bradon Smith and Anton Troynikov. 2024. Evaluating Chunking Strategies for Retrieval. find
  20. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369… find our card
  21. Ziyuan Zhuang, Zhiyang Zhang, Sitao Cheng, Fangkai Yang, Jia Liu, Shujian Huang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Zhang. 2024. Efficientrag: Efficient retriever for multi-hop question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proces… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pellet-2026-historiqa-thirdrepublic-multi-hop-question/