Elamur Research⌕
☰
home / papers / 2026 / SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series

SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series

Summary

Most video reasoning benchmarks test understanding of short clips or neighbouring frames. They leave open whether models can follow a story that unfolds over many hours. SagaQA addresses this with abstractive questions about full-length TV series. Each question is tied to a block of 20 consecutive episodes (roughly 20 hours of footage). Answering it requires chaining events from several distant episodes and combining dialogue with visual cues.

The authors created the questions by prompting an LLM with episode summaries from SummScreen3D for the soap opera As the World Turns. They kept only pairs that passed LLM-based checks for being multi-episode, multi-hop and multimodal. They then grouped agentic video planners into Parallel, Sequential and Hybrid types and compared them.

Hybrid planners explore candidate segments widely and then refine step by step, and they ground answers in the correct episodes most reliably. However, abstractive answer metrics stay nearly flat across settings. This suggests the main bottleneck is synthesizing evidence, not retrieving it.

Key points

  • The benchmark has 200 QA pairs drawn from As the World Turns (2007–2010). Each sample is (Q, E20, A, E*), where E* marks the episodes relevant to the answer. Code and data are released on GitHub (galannp/sagaqa).
  • A question needs about 4 reasoning hops and 4.2 episodes on average. The widest gap between linked episodes has a median of 13 episodes (SD 5.4).
  • Source annotations come from SummScreen3D, a video-to-text summarization corpus of 5,421 soap-opera episodes with transcripts, videos and multimodal summaries.
  • Evaluation uses Episode Precision, Recall and F1 alongside ROUGE and METEOR. Hybrid planners reach the highest F1. Sequential planners tend to repeat similar queries, and parallel planners lack depth for multi-hop chains.
  • The findings hold when Qwen3-30B-Instruct is replaced with Mistral-Small-3.1-24B in VideoRAG-based methods.
  • Stated limitations:
    • Grounding is at episode level, not clip level.
    • The data covers the soap-opera genre only.
    • The fixed 20-episode window does not test memory across a whole series.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigdial-1.22

@inproceedings{pennec2026sagaqa,
  title = {{SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series}},
  author = {Pennec, Galann and Liu, Zhengyuan and Asher, Nicholas and Muller, Philippe and Chen, Nancy},
  booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2026},
  doi = {10.18653/v1/2026.sigdial-1.22},
  url = {https://doi.org/10.18653/v1/2026.sigdial-1.22}
}

References

45 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, and Li Fei-Fei. 2024. Hourvideo: 1-hour video-language understanding. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Informa… find
  2. Yukang Chen, Wei Huang, Baifeng Shi, Qinghao Hu, Hanrong Ye, Ligeng Zhu, Zhijian Liu, Pavlo Molchanov, Jan Kautz, Xiaojuan Qi, Sifei Liu, Hongxu Yin, Yao Lu, and Song Han. 2025. Scaling RL to long videos. CoRR, abs/2507.07966. find
  3. Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, and Ying Shan. 2025. Video-holmes: Can MLLM think like holmes for complex video reasoning? CoRR, abs/2505.21374. — [arXiv:2505.21374](https://arxiv.org/abs/2505.21374) find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, Krishna Har… find our card
  5. Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. 2025. Video-R1: Reinforcing video reasoning in mllms. CoRR, abs/2503.21776. — [arXiv:2503.21776](https://arxiv.org/abs/2503.21776) find
  6. Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun. 2024. Video-MME: The first-ever comprehensive evaluation… find our card
  7. Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, and Si Liu. 2025. Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, C… find
  8. Jian Hu, Zixu Cheng, Chenyang Si, Wei Li, and Shaogang Gong. 2025. Cos: Chain-of-shot prompting for long video understanding. CoRR, abs/2502.06428. — [arXiv:2502.06428](https://arxiv.org/abs/2502.06428) find
  9. De-An Huang, Subhashree Radhakrishnan, Zhiding Yu, and Jan Kautz. 2025. FRAG: frame selection augmented generation for long video and long document understanding. CoRR, abs/2504.17447. — [arXiv:2504.17447](https://arxiv.org/abs/2504.17447) find
  10. Bruno Korbar, Yongqin Xian, Alessio Tonioni, Andrew Zisserman, and Federico Tombari. 2024. Text-conditioned resampler for long form video understanding. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXXVI, volume 15144 of Le… find
  11. Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Lou, Limin Wang, and Yu Qiao. 2024. Mvbench: A comprehensive multi-modal video understanding benchmark. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, Jun… find our card
  12. Xuchen Li, Xuzhao Li, Shiyu Hu, Kaiqi Huang, and Wentao Zhang. 2025. CausalStep: A benchmark for explicit stepwise causal reasoning in videos. CoRR, abs/2507.16878. — [arXiv:2507.16878](https://arxiv.org/abs/2507.16878) find
  13. Zhengyang Liang, Yan Shu, Xiangrui Liu, Minghao Qin, Kaixin Liang, Nicu Sebe, Zheng Liu, and Lizi Liao. 2026. Video-browser: Towards agentic open-web video browsing. find
  14. Chengwen Liu, Xiaomin Yu, Zhuoyue Chang, Zhe Huang, Shuo Zhang, Heng Lian, Kunyi Wang, Rui Xu, Sen Hu, Jianheng Hou, Hao Peng, Chengwei Qin, Xiaobin Hu, Hong Peng, Ronghao Chen, and Huacan Wang. 2026. Watching, reasoning, and searching: A video deep research benchmark on open web for agentic video r… — [arXiv:2601.06943](https://arxiv.org/abs/2601.06943) find
  15. Pinelopi Papalampidi and Mirella Lapata. 2023. Hierarchical3D adapters for long video-to-text summarization. In Findings of the Association for Computational Linguistics: EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 1267–1290. Association for Computational Linguistics. find
  16. Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryoo, Donghyun Kim, and Michael S. Ryoo. 2024. Too many frames, not all useful: Efficient strategies for long-form video QA. CoRR, abs/2406.09396. find
  17. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, Findings of ACL, pages 5687–5711. Ass… find
  18. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, Proceedings of Machine Learning Research,… find our card
  19. Aniket Rege, Arka Sadhu, Yuliang Li, Kejie Li, Ramya Korlakai Vinayak, Yuning Chai, Yong Jae Lee, and Hyo Jin Kim. 2026. Agentic very long video understanding. CoRR, abs/2601.18157. find
  20. Xubin Ren, Lingrui Xu, Long Xia, Shuaiqiang Wang, Dawei Yin, and Chao Huang. 2025. Videorag: Retrieval-augmented generation with extreme long-context videos. CoRR, abs/2502.01549. — [arXiv:2502.01549](https://arxiv.org/abs/2502.01549) find
  21. Gemma Team. 2024. Gemma 2: Improving open language models at a practical size. CoRR, abs/2408.00118. find
  22. Qwen Team. 2025. Qwen3 technical report. CoRR, abs/2505.09388. find our card
  23. Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. 2023. Egoschema: A diagnostic benchmark for very long-form video language understanding. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans,… find
  24. Mingyang Mao, Mariela M. Perez-Cabarcas, Utteja Kallakuri, Nicholas R. Waytowich, Xiaomin Lin, and Tinoosh Mohsenin. 2025. Multi-rag: A multimodal retrieval-augmented generation system for adaptive video understanding. CoRR, abs/2505.23990. — [arXiv:2505.23990](https://arxiv.org/abs/2505.23990) find
  25. Mistral AI. 2025. Mistral small 3.1. https://mistral.ai/news/mistral-small-3-1. Accessed: June 3, 2026. ↗
  26. Shulin Tian, Ruiqi Wang, Hongming Guo, Penghao Wu, Yuhao Dong, Xiuying Wang, Jingkang Yang, Hao Zhang, Hongyuan Zhu, and Ziwei Liu. 2025. Ego-r1: Chain-of-tool-thought for ultra-long egocentric video reasoning. CoRR, abs/2506.13654. — [arXiv:2506.13654](https://arxiv.org/abs/2506.13654) find
  27. Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. 2024a. LVBench: An extreme long video understanding benchmark. CoRR, abs/2406.08035. find
  28. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, JingJing Xie,… find our card
  29. Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. 2024b. VideoAgent: Long-form video understanding with large language model as agent. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXX, volume 15138 of Lecture Not… find
  30. Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. 2024c. VideoTree: Adaptive tree-based video representation for LLM reasoning on long videos. CoRR, abs/2405.19209. — [arXiv:2405.19209](https://arxiv.org/abs/2405.19209) find
  31. Zeyu Xu, Junkang Zhang, Qiang Wang, and Yi Liu. 2025. E-VRAG: enhancing long video understanding with resource-efficient retrieval augmented generation. CoRR, abs/2508.01546. — [arXiv:2508.01546](https://arxiv.org/abs/2508.01546) find
  32. Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, Bei Ouyang, Zhengyu Lin, Marco Cominelli, Zhongang Cai, Bo Li, Yuanhan Zhang, Peiyuan Zhang, Fangzhou Hong, Joerg Widmer, Francesco Gringoli, Lei Yang, and Ziwei Li… find
  33. Te Yang, Xiangyu Zhu, Bo Wang, Quan Chen, Peng Jiang, and Zhen Lei. 2025b. EEA: exploration-exploitation agent for long video understanding. CoRR, abs/2512.03500. — [arXiv:2512.03500](https://arxiv.org/abs/2512.03500) find
  34. Zeyuan Yang, Delin Chen, Xueyang Yu, Maohao Shen, and Chuang Gan. 2024. VCA: video curious agent for long video understanding. CoRR, abs/2412.10471. — [arXiv:2412.10471](https://arxiv.org/abs/2412.10471) find
  35. Zuhao Yang, Sudong Wang, Kaichen Zhang, Keming Wu, Sicong Leng, Yifan Zhang, Bo Li, Chengwei Qin, Shijian Lu, Xingxuan Li, and Lidong Bing. 2025c. Longvt: Incentivizing "thinking with long videos" via native tool calling. CoRR, abs/2511.20785. — [arXiv:2511.20785](https://arxiv.org/abs/2511.20785) find
  36. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023,… find
  37. Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristóbal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, Jiajun Wu, and Manling Li. 2025. Rethinking temporal search for long-form video understanding. In IEEE/CVF Conference on Computer Vision and P… find
  38. Jiashuo Yu, Yue Wu, Meng Chu, Zhifei Ren, Zizheng Huang, Pei Chu, Ruijie Zhang, Yinan He, Qirui Li, Songze Li, Zhenxiang Li, Zhongying Tu, Conghui He, Yu Qiao, Yali Wang, Yi Wang, and Limin Wang. 2025a. VRBench: A benchmark for multi-step reasoning in long narrative videos. CoRR, abs/2506.10857. — [arXiv:2506.10857](https://arxiv.org/abs/2506.10857) find
  39. Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. 2023. Self-chained image-language model for video localization and question answering. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA… find
  40. Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, Hao Zhang, and Qianru Sun. 2025b. Frame-Voyager: Learning to query frames for video large language models. In The Thirteenth International Conference on Learning Represe… find
  41. Huaying Yuan, Zheng Liu, Junjie Zhou, Hongjin Qian, Ji-Rong Wen, and Zhicheng Dou. 2025. Videodeepresearch: Long video understanding with agentic tool using. CoRR, abs/2506.10821. — [arXiv:2506.10821](https://arxiv.org/abs/2506.10821) find
  42. Congzhi Zhang, Zhibin Wang, Yinchao Ma, Jiawei Peng, Yihan Wang, Qiang Zhou, Jun Song, and Bo Zheng. 2025a. Rewatch-r1: Boosting complex video reasoning in large vision-language models through agentic data synthesis. CoRR, abs/2509.23652. — [arXiv:2509.23652](https://arxiv.org/abs/2509.23652) find
  43. Haoji Zhang, Xin Gu, Jiawen Li, Chixiang Ma, Sule Bai, Chubin Zhang, Bowen Zhang, Zhichao Zhou, Dongliang He, and Yansong Tang. 2025b. Thinking with videos: Multimodal tool-augmented reinforcement learning for long video reasoning. CoRR, abs/2508.04416. — [arXiv:2508.04416](https://arxiv.org/abs/2508.04416) find
  44. Zhuo Zhi, Qiangqiang Wu, Minghe shen, Wenbo Li, Yinchuan Li, Kun Shao, and Kaiwen Zhou. 2025. VideoAgent2: Enhancing the LLM-based agent system for long-form video understanding by uncertainty-aware CoT. CoRR, abs/2504.04471. — [arXiv:2504.04471](https://arxiv.org/abs/2504.04471) find
  45. Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. MLVU: A comprehensive benchmark for multi-task long video understanding. CoRR, abs/2406.04264. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pennec-2026-sagaqa-multi-hop-reasoning-benchmark/