Elamur Research⌕
☰
home / papers / 2026 / Tree-of-Evidence: Efficient “System 2” Search for Faithful Multimodal Grounding

Tree-of-Evidence: Efficient “System 2” Search for Faithful Multimodal Grounding

Paper · 2026 · Micky C. Nnamdi, Benoit Louis Marteau, Yishan Zhong et al.

Summary

Large multimodal models used in high-stakes settings such as intensive care are hard to audit. Attention maps and post-hoc attributions often fail to reflect how these models actually combine modalities. The authors propose Tree-of-Evidence (ToE), which recasts explanation as a discrete search for the smallest set of input pieces that still reproduces the model's prediction and confidence.

ToE has two parts:

  • Evidence Bottlenecks: lightweight MLP selectors trained with straight-through top-k masking. They score coarse units, namely hourly windows of ICU vitals/labs and chunks of radiology reports.
  • Beam search: run at inference over these units. It balances agreement with the original decision, stability of the predicted probability, and sparsity.

CXR and ECG information is not searched; it enters as fixed patient context. Across six tasks on three datasets in clinical and industrial domains, ToE keeps nearly all predictive performance using only a handful of units. It also gives more faithful explanations than standard attribution methods and much larger LLMs. This makes it a practical tool for auditing multimodal predictors and flagging unreliable outputs.

Key points

  • Evaluation covers four MIMIC-IV clinical tasks, cross-center testing on eICU (208 hospitals), and non-clinical fault detection on LEMMA-RCA. Notes are encoded with BioClinicalBERT.
  • With as few as five evidence units, ToE retains over 98% of full-model AUROC.
  • Under sparse budgets, ToE's explanations match the model's decisions more often and deviate less from its outputs than those of LIME, SHAP, gradient saliency, Top-k ranking, and concept-bottleneck models. Unlike concept-bottleneck models, it needs no concept annotations.
  • The ~109M-parameter system outperforms LLMs of up to 70B parameters. It adds roughly 13 ms of overhead, and beam search comes within 0.001 AUROC of exhaustive search at small k on MIMIC-IV.
  • Search exhaustion is 4–26× more frequent for false positives than for true positives, so it can support selective abstention.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Micky C. Nnamdi, Benoit Louis Marteau, Yishan Zhong, J. Ben Tamo and May Dongmei Wang. 2026. Tree-of-Evidence: Efficient “System 2” Search for Faithful Multimodal Grounding. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1460

@inproceedings{nnamdi2026tree,
  title = {{Tree-of-Evidence: Efficient “System 2” Search for Faithful Multimodal Grounding}},
  author = {Micky C. Nnamdi, Benoit Louis Marteau, Yishan Zhong, J. Ben Tamo and May Dongmei Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1460},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1460}
}

References

38 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Julián N Acosta, Guido J Falcone, Pranav Rajpurkar, and Eric J Topol. 2022. Multimodal biomedical ai. Nature medicine, 28(9):1773–1784. find
  2. Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott. 2019. Publicly available clinical bert embeddings. In Proceedings of the 2nd clinical natural language processing workshop, pages 72–78.
  3. Chun Sik Chan, Huanqi Kong, and Liang Guanqing. 2022. A comparative study of faithfulness metrics for model interpretability methods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5029–5038, Dublin, Ireland. Association for… find
  4. Zheyi Chen, Liuchang Xu, Hongting Zheng, Luyao Chen, Amr Tolba, Liang Zhao, Keping Yu, and Hailin Feng. 2024. Evolution and prospects of foundation models: From large language models to large multimodal models. Computers, Materials & Continua, 80(2). find
  5. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2020. Eraser: A benchmark to evaluate rationalized nlp models. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4443–4458.
  6. Joakim Edin, Andreas Geert Motzfeldt, Casper L. Christensen, Tuukka Ruotsalo, Lars Maaløe, and Maria Maistro. 2025. Normalized AOPC: Fixing misleading faithfulness metrics for feature attributions explainability. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguist… find
  7. Sara Bersche Golas, Takuma Shibahara, Stephen Agboola, Hiroko Otaki, Jumpei Sato, Tatsuya Nakae, Toru Hisamitsu, Go Kojima, Jennifer Felsted, Sujay Kakarmath, and 1 others. 2018. A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analy… find
  8. Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. 2000. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. circulatio… find
  9. Brian Gow, Tom Pollard, Larry A Nathanson, Alistair Johnson, Benjamin Moody, Chrystinne Fernandes, Nathaniel Greenbaum, Jonathan W Waks, Parastou Eslami, Tanner Carbonati, Ashish Chaudhari, Elizabeth Herbst, Dana Moukheiber, Seth Berkowitz, Roger Mark, and Steven Horng. 2023. MIMIC-IV-ECG: Diagnosti… find
  10. Dawei Huang, Chuan Yan, Qing Li, and Xiaojiang Peng. 2024. From large language models to large multimodal models: A literature review. Applied Sciences, 14(12):5068. find
  11. Shih-Cheng Huang, Anuj Pareek, Saeed Seyyedi, Imon Banerjee, and Matthew P Lungren. 2020. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ digital medicine, 3(1):136. find
  12. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4198–4205.
  13. Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543–3556.
  14. A Johnson, L Bulgarelli, T Pollard, B Gow, B Moody, S Horng, LA Celi, and R Mark. 2024. Mimic-iv (version 3.1). physionet. rrid: Scr_007345. find
  15. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, and 1 others. 2023. Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10(1):1.
  16. Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. In International conference on machine learning, pages 5338–5348. PMLR. find
  17. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  18. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107–117, Austin, Texas. Association for Computational Linguistics.
  19. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
  20. Bhargavi Paranjape, Mandar Joshi, John Thickstun, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. An information bottleneck approach for controlling conciseness in rationale extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1938– 19…
  21. Tom J Pollard, Alistair EW Johnson, Jesse D Raffa, Leo A Celi, Roger G Mark, and Omar Badawi. 2018. The eicu collaborative research database, a freely available multi-center database for critical care research. Scientific data, 5(1):180178. find
  22. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135– 1144.
  23. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, volume 32. find
  24. Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. 2019. Latent ordinary differential equations for irregularly-sampled time series. Advances in neural information processing systems, 32. find
  25. Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215.
  26. Tomohisa Seki, Yoshimasa Kawazoe, and Kazuhiko Ohe. 2021. Machine learning-based prediction of in-hospital mortality using admission laboratory data: A retrospective, single-site study using electronic health record data. PloS one, 16(2):e0246640. find
  27. Lecheng Zheng, Zhengzhang Chen, Dongjie Wang, Chengyuan Deng, Reon Matsuoka, and Haifeng Chen. 2024. Lemma-rca: A large multi-modal multi-domain dataset for root cause analysis. arXiv preprint arXiv:2406.05375. — [arXiv:2406.05375](https://arxiv.org/abs/2406.05375) find
  28. Yilun Zhou and Julie Shah. 2023. The solvability of interpretability evaluation metrics. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2399–2415. find
  29. Vivswan Shitole, Fuxin Li, Minsuk Kahng, Prasad Tadepalli, and Alan Fern. 2021. One explanation is not enough: structured attention graphs for image classification. Advances in Neural Information Processing Systems, 34:11352–11363. find
  30. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find
  31. Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ira Ktena, and 1 others. 2024. Towards generalist biomedical ai. Nejm Ai, 1(3):AIoa2300138. find
  32. Moritz Vandenhirtz, Sonia Laguna, Ričards Marcinke-vičs, and Julia Vogt. 2024. Stochastic concept bottleneck models. Advances in Neural Information Processing Systems, 37:51787–51810. find
  33. Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech., 31:841. find
  34. Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11–20, Hong Kong, China. Association for…
  35. Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A Pfeffer, Jason Fries, and Nigam H Shah. 2023. The shaky foundations of large language models and foundation models for electronic health records. npj digital medicine, 6(1):135. find
  36. Xi Xu, Jianqiang Li, Zhichao Zhu, Linna Zhao, Huina Wang, Changwei Song, Yining Chen, Qing Zhao, Jijiang Yang, and Yan Pei. 2024. A comprehensive review on synergy of multi-modal data and ai technologies in medical diagnosis. Bioengineering, 11(3):219. find
  37. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. find
  38. Evidence Size by Mortality Outcome Table A1: ToE Comprehensiveness (↑) across evidence budgets and four MIMIC-IV tasks (mean ± std, 5 seeds). E4: Post-Obs Mort. 1 3 5 8 12 16 20 24 0.0413 ± 0.0050 0.0885 ± 0.0104 0.1112 ± 0.0143 0.1293 ± 0.0177 0.1417 ± 0.0199 0.1489 ± 0.0216 0.1549 ± 0.0233 0.1612… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nnamdi-2026-tree-evidence-efficient-system-2/