Elamur Research⌕
☰
home / papers / 2026 / Modeling Clinical Uncertainty in Radiology Reports: From Explicit Uncertainty Markers to Implicit Reasoning Pathways

Modeling Clinical Uncertainty in Radiology Reports: From Explicit Uncertainty Markers to Implicit Reasoning Pathways

Summary

Chest X-ray reports carry two kinds of uncertainty that structuring pipelines usually miss. The first is explicit: radiologists hedge with words like "likely" or "cannot be excluded", and how strong that hedge is depends on context. The second is implicit: reports often state only a diagnosis and leave out the supporting observations, so an unmentioned finding may be absent or simply unrecorded.

The authors handle both on the Lunguage benchmark of structured MIMIC-CXR reports.

  • Explicit uncertainty: Gemini extracts hedging phrases tied to each tentative finding. An LLM then compares uncertainty expressions in pairs, producing a relative ranking of phrases that clinicians validated. This ranking is converted into a continuous probability between 0 and 1, replacing the usual positive/uncertain/negative labels.
  • Implicit uncertainty: clinicians defined diagnostic pathways for 14 common conditions. These rules add the expected sub-findings to each stated diagnosis, carry over that diagnosis's probability, and resolve conflicts with existing findings.

The outcome is Lunguage++, an uncertainty-aware resource. It supports image classifiers that account for uncertainty and more faithful modelling of diagnostic reasoning.

Key points

  • Base data: Lunguage, with 1,473 MIMIC-CXR reports containing 14,049 structured findings, of which 2,066 finding-sentence pairs are labelled tentative.
  • The hedging vocabulary has 42 phrases, each extracted at least ten times. The most frequent are "or", "likely", "may", "suggesting" and "cannot be excluded".
  • There are 33 expert pathways across 14 diagnoses (e.g., pneumonia, emphysema, CHF, fracture, tuberculosis). They cover sub-findings that experts consider highly likely (>80%) for each diagnosis.
  • Rule-based conflict resolution handles clashes when expanded findings overlap existing ones. Original findings take priority over expansions, and otherwise the entry with the highest probability is kept.
  • Code is on GitHub (prabaey/lunguage_uncertainty), and Lunguage++ will be released through PhysioNet.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2i8nrgcnb52p

@inproceedings{rabaey2026modeling,
  title = {{Modeling Clinical Uncertainty in Radiology Reports: From Explicit Uncertainty Markers to Implicit Reasoning Pathways}},
  author = {Rabaey, Paloma and Moon, Jong Hak and Lee, Jung-Oh and Kim, Min Gwan and Yoon, Hangyul and Demeester, Thomas and Choi, Edward},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2i8nrgcnb52p},
  url = {https://doi.org/10.63317/2i8nrgcnb52p}
}

References

27 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Claude sonnet. https://www.anthropic.com/transparency/model-report. ↗
  2. Leonard Berlin. 2000. Pitfalls of the vague radiology report. American Journal of Roentgenology, 174(6):1511–1518. find
  3. Michael A Bruno, Jonelle Petscavage-Thomas, and Hani H Abujudeh. 2017. Communicating uncertainty in the radiology report. American Journal of Roentgenology, 209(5):1006–1008. find
  4. Andrew L Callen, Sara M Dupont, Adi Price, Ben Laguna, David McCoy, Bao Do, Jason Talbott, Marc Kohli, and Jared Narvid. 2020. Between always and never: evaluating uncertainty in radiology reports using natural language processing. Journal of Digital Imaging, 33(5):1194–1201. find
  5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv prep… find our card
  6. Julia Gärtner, Pascal O Berberat, Martina Kadmon, and Sigrid Harendza. 2020. Implicit expression of uncertainty–suggestion of an empirically derived framework. BMC Medical Education, 20(1):83. find
  7. Lawrence R Goodman. 2014. Felson’s principles of chest roentgenology, a programmed text. Elsevier Health Sciences. find
  8. Richard B Gunderman. 2009. Biases in radiologic reasoning. American Journal of Roentgenology, 192(3):561–564. find
  9. Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. Trueskill™: a bayesian skill rating system. Advances in neural information processing systems, 19. find
  10. Jonathan L Hobby, BD Tom, C Todd, PW Bearcroft, and Adrian K Dixon. 2000. Communication of doubt and certainty in radiological reports. The British Journal of Radiology, 73(873):999–1001. find
  11. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. find our card
  12. Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. 2019. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on art… find
  13. Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P Lungren, Andrew Y Ng, et al. 2021. Radgraph: Extracting clinical entities and relations from radiology reports. arXiv preprint arXiv:2106.14463. — [arXiv:2106.14463](https://arxiv.org/abs/2106.14463) find
  14. Alistair Johnson, Matt Lungren, Yifan Peng, Zhiyong Lu, Roger Mark, Seth Berkowitz, and Steven Horng. 2019a. Mimic-cxr-jpg-chest radiographs with structured labels. PhysioNet, 101:215–220. find
  15. Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. 2019b. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317. find
  16. Sameer Khanna, Adam Dejl, Kibo Yoon, Steven QH Truong, Hanh Duong, Agustina Saenz, and Pranav Rajpurkar. 2023. Radgraph2: Modeling disease progression in radiology reports via hierarchical information extraction. In Machine learning for healthcare conference, pages 381– 402. PMLR. find
  17. Cindy S Lee, Paul G Nagy, Sallie J Weaver, and David E Newman-Toker. 2013. Cognitive and system factors contributing to diagnostic errors in radiology. American Journal of Roentgenology, 201(3):611–617. find
  18. Jong Hak Moon, Geon Choi, Paloma Rabaey, Min Gwam Kim, Hyuk Gi Hong, Jung Oh Lee, Hangyul Yoon, Eunwoo Doe, Jiyoun Kim, Harshita Sharma, Daniel Coelho de Castro, Javier Alvarez Valle, and Edward Choi. 2026. Lunguage: A Benchmark for Structured and Sequential Chest X-ray Interpretation. PhysioNet. Ve… find
  19. Judea Pearl. 2009. Causality. Cambridge university press. find
  20. Bruce I Reiner. 2018. Quantitative analysis of uncertainty in medical reporting: creating a standardized and objective methodology. Journal of digital imaging, 31(2):145–149. find
  21. François Remy, Kris Demuynck, and Thomas Demeester. 2023. Biolord-2023: Semantic textual representations fusing llm and clinical knowledge graph insights. arXiv preprint arXiv:2311.16075. — [arXiv:2311.16075](https://arxiv.org/abs/2311.16075) find
  22. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. 2025. Medgemma technical report. arXiv preprint arXiv:2507.05201. find
  23. Atul B Shinagare, Ronilda Lacson, Giles W Boland, Aijia Wang, Stuart G Silverman, William W Mayo-Smith, and Ramin Khorasani. 2019. Radiologist preferences, agreement, and variability in phrases used to convey diagnostic certainty in radiology reports. Journal of the American College of Radiology, 16… find
  24. Mark Turner, Julia Ive, and Sumithra Velupillai. 2021. Linguistic uncertainty in clinical nlp: A taxonomy, dataset and approach. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 129–141. Springer. find
  25. W Richard Webb and Charles B Higgins. 2011. Thoracic imaging: pulmonary and cardiovascular radiology. Lippincott Williams & Wilkins. find
  26. Joy T Wu, Nkechinyere N Agu, Ismini Lourentzou, Arjun Sharma, Joseph A Paguio, Jasper S Yao, Edward C Dee, William Mitchell, Satyananda Kashyap, Andrea Giovannini, et al. 2021. Chest imagenome dataset for clinical reasoning. arXiv preprint arXiv:2108.00316. — [arXiv:2108.00316](https://arxiv.org/abs/2108.00316) find
  27. Mengliang Zhang, Xinyue Hu, Lin Gu, Tatsuya Harada, Kazuma Kobayashi, Ronald Summers, and Yingying Zhu. 2023. Cad-chest: Comprehensive annotation of diseases based on mimic-cxr radiology report. PhysioNet. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rabaey-2026-modeling-clinical-uncertainty-radiology-reports/