Elamur Research⌕
☰
home / papers / 2026 / TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation

TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation

Paper · 2026 · Pengyu Yan, Akhil V S S Gorugantu, Mahesh Bhosale et al.

Summary

The paper tackles multi-video event understanding: producing factual, properly cited claims from many long, heterogeneous news videos in response to a query and user persona. Large vision–language models struggle here because they run out of frame budget and favor visually striking content over brief but informative moments such as tickers, captions or scoreboards. TRACE takes a "ground first, reason later" approach. It runs YOLOv12 object detection and OCR on sampled frames to build a text-searchable timeline for each video. A text-only LLM then searches that timeline to find query-relevant moments and writes guidance. An LVLM (Qwen3-VL) generates claims from uniform frames plus the targeted keyframes, and a final stage merges duplicate claims across videos and propagates citations. On the MAGMaR 2026 validation split, macro MiRAGE F1 rises from 0.705 to 0.811, driven mainly by better citation recall. The method also tops the official MAGMaR leaderboard and transfers to WikiVideo. The work shows that cheap, interpretable text signals can stand in for brute-force long-context video processing.

Key points

  • Evaluation and baseline: TRACE is tested on the MAGMaR 2026 Oracle Track (validation subset of 8 topics) and WikiVideo (52 queries). It is compared against unguided Qwen3-VL-8B and Qwen3-VL-30B.
  • MAGMaR validation results:
    • Citation recall rises from 0.440 to 0.628, and CiteF1 from 0.609 to 0.753.
    • Citation precision stays high at 0.939.
    • InfoF1 improves from 0.800 to 0.869.
  • WikiVideo results:
    • Avg. F1 reaches 0.879, versus 0.878 for Qwen3-VL-8B and 0.854 for Qwen3-VL-30B.
    • Citation recall goes from 0.792 to 0.838.
    • The smaller margin is attributed to shorter videos and baselines that already score near the ceiling.
  • Ablation:
    • The best setup combines 100 uniform frames, guidance-targeted keyframes, and embedding-similarity deduplication with LLM verification (EMBED-SIM).
    • Every variant scores at least 0.802 Avg. F1, so the grounding stage itself accounts for most of the gain.
  • Limitations:
    • The detector only recognizes COCO-80 classes.
    • Errors in the sequential grounding stage cannot be recovered downstream.
    • Proposed future work includes open-vocabulary detection and end-to-end optimization.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York UniversityNew York · United States201 works · 100,180 citations🎓University at Buffalo, State University of New YorkBuffalo · United States12 works · 318 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Pengyu Yan, Akhil V S S Gorugantu, Mahesh Bhosale, Abdul Wasi, Vishvesh Trivedi and David Doermann. 2026. TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation. Proceedings of the 2nd Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.magmar-main.14

@inproceedings{yan2026trace,
  title = {{TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation}},
  author = {Pengyu Yan, Akhil V S S Gorugantu, Mahesh Bhosale, Abdul Wasi, Vishvesh Trivedi and David Doermann},
  booktitle = {Proceedings of the 2nd Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.magmar-main.14},
  url = {https://doi.org/10.18653/v1/2026.magmar-main.14}
}

References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai and 1 others. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection. ICLR.
  2. Ali Furkan Biten and 1 others. 2019. Scene text visual question answering. In ICCV. find
  3. Sebastian Borgeaud and 1 others. 2022. Improving language models by retrieving from trillions of tokens. ICML. find
  4. Yuning Du and 1 others. 2020. Pp-ocr: A practical ultra lightweight ocr system. arXiv preprint arXiv:2009.09941. — [arXiv:2009.09941](https://arxiv.org/abs/2009.09941) find
  5. Chaoyou Fu and 1 others. 2024. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075.
  6. Tanmay Gupta and Aniruddha Kembhavi. 2023. Visual programming: Compositional visual reasoning without training. In CVPR. find
  7. Reno Kriz, Kate Sanders, David Etter, Kenton Murray, Cameron Carpenter, Carl Van Ess-Dykema, Eugene Yang, Hamed Sayyed, Robin Carmody, Mahsa Roberts, and Benjamin Van Durme. 2025. MultiVENT 2.0: A massive multilingual benchmark for event-centric video retrieval. arXiv preprint arXiv:2410.11619. Veri… — [arXiv:2410.11619](https://arxiv.org/abs/2410.11619) find
  8. Fanqing Ma and 1 others. 2024. Llama-vid: An image is worth 2 tokens in large language models. arXiv preprint arXiv:2311.17043. find
  9. Alexander Martin, Kate Sanders, William Walden, Dengjia Zhang, Reno Kriz, Angela Cao, Adarsh Pyarelal, Eugene Yang, and Benjamin Van Durme. 2025a. WikiVideo: Article generation from multiple videos. arXiv preprint arXiv:2504.00939.
  10. Alexander Martin, William Walden, Reno Kriz, Dengjia Zhang, Kate Sanders, Eugene Yang, Chihsheng Jin, and Benjamin Van Durme. 2025b. Seeing through the mirage: Evaluating multimodal retrieval augmented generation. Preprint, arXiv:2510.24870. — [arXiv:2510.24870](https://arxiv.org/abs/2510.24870) find
  11. Anand Mishra and 1 others. 2019. Ocr-vqa: Visual question answering by reading text in images. IC-DAR. find
  12. Nanyun Peng and 1 others. 2023. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824.
  13. Jihao Qiu, Yuan Zhang, Xi Tang, Lingxi Xie, Tianren Ma, Pengyu Yan, David Doermann, Qixiang Ye, and Yunjie Tian. 2024. Artemis: Towards referential understanding in complex videos. arXiv preprint arXiv:2406.00258. — [arXiv:2406.00258](https://arxiv.org/abs/2406.00258) find
  14. Kate Sanders, David Etter, Reno Kriz, and Benjamin Van Durme. 2023. Multivent: Multilingual videos of events and aligned natural text. Advances in Neural Information Processing Systems, 36:51065–51079. find
  15. Yongliang Shen and 1 others. 2023. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. NeurIPS. find
  16. Jie Lei and 1 others. 2021a. Moment-detr: End-to-end video moment retrieval and highlight detection. In NeurIPS. find
  17. Jie Lei and 1 others. 2021b. Qvhighlights: Detecting moments and highlights in videos via natural language queries. In NeurIPS. find
  18. Patrick Lewis and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In NeurIPS.
  19. KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2024. VideoChat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355.
  20. Liunian Harold Li and 1 others. 2022. Glip: Grounded language-image pre-training. In CVPR. find
  21. Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. 2024. Video-LLaVA: Learning united visual representation by alignment before projection. In Proceedings of EMNLP.
  22. Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. 2023. Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499.
  23. Amanpreet Singh and 1 others. 2019. Towards vqa models that read. In CVPR. find
  24. Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, Yan Lu, Jenq-Neng Hwang, and Gaoang Wang. 2024. MovieChat: From dense token to sparse memory for long video understanding. In Proceedings of CVPR. find
  25. Dídac Suris and 1 others. 2023. Vipergpt: Visual inference via python execution for reasoning. ICCV.
  26. Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. 2025. Adaptive keyframe sampling for long video understanding. arXiv preprint arXiv:2502.21271. — [arXiv:2502.21271](https://arxiv.org/abs/2502.21271) find
  27. Hunyuan Vision Team, Pengyuan Lyu, Xingyu Wan, Gengluo Li, Shangpin Peng, Weinong Wang, Liang Wu, Huawen Shen, Yu Zhou, Canhui Tang, Qi Yang, Qiming Peng, Bin Luo, Hower Yang, Xinsong Zhang, Jinnian Zhang, Houwen Peng, Hongming Yang, Senhao Xie, and 12 others. 2025. Hunyuanocr technical report.
  28. Qwen Team. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  29. Qwen Team. 2025b. Qwen3 technical report. Preprint, arXiv:2505.09388.
  30. Tencent Hunyuan Team. 2025. HunyuanOCR: A multilingual OCR model from tencent Hunyuan. Tencent Hunyuan Team. Model card available at https://huggingface.co/tencent/HunyuanOCR; verify final citation form. ↗
  31. Yunjie Tian, Qixiang Ye, and David Doermann. 2025. YOLOv12: Attention-centric real-time object detectors. arXiv preprint arXiv:2502.12524. find
  32. Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. 2024. LongVideoBench: A benchmark for long-context interleaved video-language understanding. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track. find
  33. Pengyu Yan and 1 others. 2024. Chartreformer: Natural language-driven chart image editing. arXiv preprint arXiv:2403.00209. — [arXiv:2403.00209](https://arxiv.org/abs/2403.00209) find
  34. Zhengyuan Yang and 1 others. 2023. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381. find
  35. Dengjia Zhang, Alexander Martin, William Jurayj, Kenton Murray, Benjamin Van Durme, and Reno Kriz. 2026. Unified multimodal uncertain inference. Preprint, arXiv:2604.08701. — [arXiv:2604.08701](https://arxiv.org/abs/2604.08701) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yan-2026-trace-evidence-grounding-guided-multi/