Elamur Research⌕
☰
home / papers / 2026 / Take It All: Ensemble Retrieval for Multimodal Evidence Aggregation

Take It All: Ensemble Retrieval for Multimodal Evidence Aggregation

Paper · 2026 · Max Upravitelev, Veronika Solopova, Premtim Sahitaj et al.

Summary

Fact-checking claims that pair text with images, such as miscaptioned photos or fabricated infographics, requires analyzing both modalities. Visual processing, however, makes systems slow and costly to run. The authors present ADA-AGGR, a pipeline for the FEVER 9 AVerImaTeC shared task. Instead of letting a model choose which retrieval tools to call, it runs every retriever on every claim, which avoids errors caused by poor tool selection. The pipeline combines:

  • question generation with a VLM,
  • hybrid text retrieval (BM25 candidates re-ranked by dense embeddings),
  • image-only and image-plus-text retrieval based on ColPali embeddings.

The pooled results become a fixed set of ten evidence pieces, which feed a final verdict model. A variant built around Gemini 3 Pro placed second on both the dev and test leaderboards. The authors then ran extensive ablations with open-weight Qwen3 models to study the trade-off between accuracy and speed. Heavily shrinking the images given to the VLM cut average runtime by up to 6.28× with comparable scores. An LLM-based error analysis found that retrieved text drives most correct verdicts, while images help on average but can sometimes mislead.

Key points

  • ADA-AGGR ranked 2nd of 17 teams on the AVerImaTeC dev leaderboard and 2nd on the test leaderboard (Gemini 3 Pro variant). The code is released on GitHub (XplaiNLP).
  • Evidence budget: BM25 retrieves 1,000 candidates, and the top 5 after dense re-ranking are kept as text evidence. Two ColPali retrievers (image-only and multimodal) each return their top 3 images, which are merged and de-duplicated, for 10 evidence items in total.
  • Open-weight configurations used Qwen3-VL 2B/8B (base and fine-tuned) for the generation steps and Qwen3 embedding models for text retrieval. Some smaller components lowered runtime without hurting results.
  • Shrinking input images as much as eightfold made the pipeline up to 6.28× faster at similar quality. Even the fastest setup still needs about 75 seconds per claim.
  • Suggested future work includes batching, vllm-style inference engines, smaller fine-tuned models, and further reduction of visual input. The system is meant as decision support for human fact-checkers, not a replacement.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬German Research Centre for Artificial IntelligenceKaiserslautern · Germany77 works · 29,479 citations🎓Technische Universität BerlinBerlin · Germany22 works · 957 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 2Research subjects and languages the work is about.Vision & language · 657 worksInformation retrieval & search · 473 works

Cite

Cite as Max Upravitelev, Veronika Solopova, Premtim Sahitaj, Ariana Sahitaj, Charlott Jakob, Sebastian Möller and Vera Schmitt. 2026. Take It All: Ensemble Retrieval for Multimodal Evidence Aggregation. Proceedings of the Ninth Fact Extraction and VERification Workshop (FEVER).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.fever-1.7

@inproceedings{upravitelev2026take,
  title = {{Take It All: Ensemble Retrieval for Multimodal Evidence Aggregation}},
  author = {Max Upravitelev, Veronika Solopova, Premtim Sahitaj, Ariana Sahitaj, Charlott Jakob, Sebastian Möller and Vera Schmitt},
  booktitle = {Proceedings of the Ninth Fact Extraction and VERification Workshop (FEVER)},
  year = {2026},
  doi = {10.18653/v1/2026.fever-1.7},
  url = {https://doi.org/10.18653/v1/2026.fever-1.7}
}

References

33 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Raia Abu Ahmad, Aida Usmanova, and Georg Rehm. 2025. The ClimateCheck dataset: Mapping social media claims about climate change to corresponding scholarly articles. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 42–56, Vienna, Austria. Association for Computa… find
  2. Mubashara Akhtar, Rami Aly, Yulong Chen, Zhenyun Deng, Michael Schlichtkrull, Chenxi Whitehouse, and Andreas Vlachos. 2025. The 2nd automated verification of textual claims (AVeriTeC) shared task: Open-weights, reproducible and efficient systems. In Proceedings of the Eighth Fact Extraction and VERi… find
  3. Google DeepMind. 2025. Gemini 3. Google DeepMind Model Release. Accessed: 2025-11-30. find
  4. Rajarshi Haldar and Julia Hockenmaier. 2025. Rating roulette: Self-inconsistency in LLM-as-a-judge frameworks. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 24986– 25004, Suzhou, China. Association for Computational Linguistics. find
  5. Mubashara Akhtar, Michael Schlichtkrull, Zhijiang Guo, Oana Cocarascu, Elena Simperl, and Andreas Vlachos. 2023. Multimodal automated fact-checking: A survey. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5430–5448, Singapore. Association for Computational Linguisti… find
  6. Michael Hameleers, Thomas E Powell, Toni GLA Van Der Meer, and Lieke Bos. 2020. A picture paints a thousand lies? the effects and mechanisms of multimodal disinformation and rebuttals disseminated via social media. Political communication, 37(2):281– 301. find
  7. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025. Qwen3-vl technical report. Preprint, ar…
  8. Jushaan Singh Kalra, Xinran Zhao, To Eun Kim, Fengyu Cai, Fernando Diaz, and Tongshuang Wu. 2025. MoR: Better handling diverse queries with a mixture of sparse, dense, and human retrievers. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 11982–12001,… find
  9. Rui Cao, Zifeng Ding, Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2025. Averimatec: A dataset for automatic verification of image-text claims with evidence from the web. arXiv preprint arXiv:2505.17978. — [arXiv:2505.17978](https://arxiv.org/abs/2505.17978) find
  10. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 39–48, New York, NY, USA. Associati…
  11. Recep Firat Cekinel, Pinar Karagoz, and Çağrı Çöltekin. 2025. Multimodal fact-checking with vision language models: A probing classifier based solution with embedding strategies. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4622–4633. find
  12. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  13. Jaemin Cho, Debanjan Mahata, Ozan Irsoy, Yujie He, and Mohit Bansal. 2025. M3docvqa: Multi-modal multi-page multi-document understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 6178– 6188. find
  14. Mingda Li, Xinyu Li, Yifan Chen, Wenfeng Xuan, and Weinan Zhang. 2024a. Unraveling and mitigating retriever inconsistencies in retrieval-augmented large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4833– 4850, Bangkok, Thailand. Association for Compu… find
  15. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 other…
  16. Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024b. Retrieval augmented generation or long-context LLMs? a comprehensive study and hybrid approach. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 881– 893,… find
  17. Michael Han Daniel Han and Unsloth team. 2023. Unsloth. find
  18. Quentin Macé, António Loison, and Manuel Faysse. 2025. Vidore benchmark v2: Raising the bar for visual retrieval. Preprint, arXiv:2505.17166. — [arXiv:2505.17166](https://arxiv.org/abs/2505.17166) find
  19. Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, CELINE HUDELOT, and Pierre Colombo. 2025. Colpali: Efficient document retrieval with vision language models. In International Conference on Learning Representations, volume 2025, pages 61424–61449. find
  20. Preslav Nakov, David Corney, Maram Hasanain, Firoj Alam, Tamer Elsayed, Alberto Barrón-Cedeño, Paolo Papotti, Shaden Shaar, and Giovanni Da San Martino. 2021. Automated Fact-Checking for Assisting Human Fact-Checkers. In Proceedings of the Thirtieth International Joint Conference on Artificial Intel… find
  21. Saikrishna Sanniboina, Shiv Trivedi, and Sreenidhi Vijayaraghavan. 2024. Lore: Logit-ranked retriever ensemble for enhancing open-domain question answering. Preprint, arXiv:2410.10042. — [arXiv:2410.10042](https://arxiv.org/abs/2410.10042) find
  22. Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, and Andreas Vlachos. 2024. The automated verification of textual claims (AVeriTeC) shared task. In Proceedings of th… find
  23. Vera Schmitt, Isabel Bezzaoui, Charlott Jakob, Premtim Sahitaj, Qianli Wang, Arthur Hilbert, Max Upravitelev, Jonas Fegert, Sebastian Möller, and Veronika Solopova. 2025. Beyond transparency: Evaluating explainability in ai-supported fact-checking. In Proceedings of the 4th ACM International Worksho… find
  24. Mir Nafis Sharear Shopnil, Sharad Duwal, Abhishek Tyagi, and Adiba Mahbub Proma. 2025. Mirage: Agentic framework for multimodal misinformation detection with web-grounded reasoning. Preprint, arXiv:2510.17590. — [arXiv:2510.17590](https://arxiv.org/abs/2510.17590) find
  25. Veronika Solopova, Robert Nickel, and Dorothea Kolossa. 2025. Lagging Behind: Challenges of Adapting Automated Propaganda Detection to the Chronological and Geographic Evolution of Language, chapter Chapter 9. find
  26. Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi, and Dinesh Manocha. 2025. VisDoM: Multi-document QA with visually rich elements using multimodal retrieval-augmented generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associati… find
  27. Max Upravitelev, Premtim Sahitaj, Arthur Hilbert, Veronika Solopova, Jing Yang, Nils Feldhus, Tatiana Anikina, Simon Ostermann, and Vera Schmitt. 2025b. Exploring semantic filtering heuristics for efficient claim verification. In Proceedings of the Eighth Fact Extraction and VERification Workshop (F… find
  28. Navve Wasserman, Roi Pony, Oshri Naparstek, Adi Raz Goldfarb, Eli Schwartz, Udi Barzelay, and Leonid Karlinsky. 2025. REAL-MM-RAG: A real-world multi-modal retrieval benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3…
  29. Cailin Winston and René Just. 2025. A taxonomy of failures in tool-augmented llms. In 2025 IEEE/ACM International Conference on Automation of Software Test (AST), pages 125–135. find
  30. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transf…
  31. Jing Yang and Anderson Rocha. 2024. Take it easy: Label-adaptive self-rationalization for fact verification and explanation generation. In 2024 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1–6. IEEE. find
  32. Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, and Kunwoo Park. 2024. HerO at AVeriTeC: The herd of open large language models for verifying real-world claims. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 130– 136, Miami, Florida, USA. Association for Computation… find
  33. Max Upravitelev, Nicolau Duran-Silva, Christian Woerle, Giuseppe Guarino, Salar Mohtaj, Jing Yang, Veronika Solopova, and Vera Schmitt. 2025a. Comparing LLMs and BERT-based classifiers for resource-sensitive claim verification in social media. In Proceedings of the Fifth Workshop on Scholarly Docume… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/upravitelev-2026-take-it-all-ensemble-retrieval/