Elamur Research
☰
home / papers / 2023 / Capabilities of GPT-4 on Medical Challenge Problems

Capabilities of GPT-4 on Medical Challenge Problems

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2303.13375

Summary

This study measures how well a general-purpose LLM handles medical exam questions without medical fine-tuning or elaborate prompting. The authors tested the text-only GPT-4 against GPT-3.5 and published Flan-PaLM 540B / Med-PaLM numbers on six multiple-choice benchmarks:

  • two official NBME sources of USMLE practice material (the Sample Exam and the Self-Assessments)
  • four public datasets from the MultiMedQA suite: MedQA, PubMedQA, MedMCQA and MMLU

They used a simple prompt borrowed from earlier Med-PaLM work, in zero-shot and 5-shot variants, and the model produced only a single answer letter. The study also looked at:

  • questions that refer to images
  • signs that exam content was memorized during training
  • how well the model's answer probabilities match its actual correctness
  • a qualitative case study of explanations and counterfactual reasoning

GPT-4 beat the USMLE passing threshold by more than 20 points and clearly outscored GPT-3.5. It also beat Flan-PaLM 540B on nearly every dataset and was markedly better calibrated. The work matters as a strong baseline for LLMs in medicine. The authors still stress caution about errors, safety and real-world validation before clinical use.

Key points

  • Zero-shot GPT-4 scored 86.65% on the USMLE Self-Assessments and 86.7% on the Sample Exam, compared with 53.61% and 58.78% for GPT-3.5.
  • Prompts followed the MultiMedQA template, with small changes for chat models. The logit_bias API parameter restricted output to valid option letters, and 5-shot results used leave-one-out cross-validation.
  • GPT-4 outperformed Flan-PaLM 540B on every MultiMedQA dataset except one, without chain-of-thought, retrieval or ensembling.
  • GPT-4's probabilities were much better calibrated than GPT-3.5's, which matters for high-stakes medical decisions.
  • The paper discusses uses in medical education, decision support and reducing administrative work. It also covers risks: accuracy, fairness, effects on career choices, and the limits of benchmark-based evaluation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

192 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 179 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.13375 · arXiv 2303.13375

@misc{nori2023capabilities,
  title = {{Capabilities of GPT-4 on Medical Challenge Problems}},
  author = {Nori, Harsha and King, Nicholas and McKinney, Scott Mayer and Carignan, Dean and Horvitz, Eric},
  year = {2023},
  doi = {10.48550/arxiv.2303.13375},
  eprint = {2303.13375},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.13375}
}

References

70 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reductions approach to fair classification. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, p… find
  2. Shunit Agmon, Plia Gillis, Eric Horvitz, and Kira Radinsky. Gender-sensitive word embeddings for healthcare. Journal of the American Medical Informatics Association, 29(3):415– 423, 2022. find
  3. Roy Adams, Katharine E Henry, Anirudh Sridharan, Hossein Soleimani, Andong Zhan, Nishi Rawat, Lauren Johnson, David N Hager, Sara E Cosgrove, Andrew Markowski, et al. Prospective, multi-site study of patient outcomes after implementation of the trews machine learning-based early warning system for s… find
  4. Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. Guidelines for human-AI interaction. In Proceedings of the 2019 CHI conference on Human Factors in Computing Systems, pages 1–13, 2019. find
  5. Mohsen Bayati, Mark Braverman, Michael Gillam, Karen M Mack, George Ruiz, Mark S Smith, and Eric Horvitz. Data-driven decisions for reducing readmissions for heart failure: General methodology and case study. PloS one, 9(10):e109264, 2014. find
  6. Rick Barraza, Russell Eames, Yan Esteve Balducci, Josh Hinds, Scott Hoogerwerf, Eric Horvitz, Ece Kamar, Jacquelyn Krones, Josh Lovejoy, Parham Mohadjer, et al. Error terrain analysis for machine learning: Tool and visualizations. In ICLR workshop on debugging machine learning models, 2019. find
  7. Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018. find
  8. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al. On the opportunities and risks of foundation models, 2021. find our card
  9. Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning: Limitations and Opportunities. fairmlbook.org, 2019. http://www.fairmlbook.org. ↗
  10. Yuntao Bai, Andy Jones, Kamal Ndousse, et al. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback, April 2022. find
  11. David W Bates, David M Levine, Hojjat Salmasian, Ania Syrowatka, David M Shahian, Stuart Lipsitz, Jonathan P Zebrowski, Laura C Myers, Merranda S Logan, Christopher G Roy, et al. The safety of inpatient health care. New England Journal of Medicine, 388(2):142–153, 2023. find
  12. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. find
  13. Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI conference on human computation and crowdsourcing, volume 7, pages 2–11, 2019. find
  14. Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S Weld, Walter S Lasecki, and Eric Horvitz. Updates in human-AI teams: Understanding and addressing the performance/-compatibility tradeoff. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2429–2437, 2019. find
  15. Bruce G Buchanan and Edward H Shortliffe. Rule based expert systems: the mycin experiments of the stanford heuristic programming project (the Addison-Wesley series in artificial intelligence). Addison-Wesley Longman Publishing Co., Inc., 1984. find
  16. Sebastien Bubeck and Mark Sellke. A universal law of robustness via isoperimetry. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 28811–28822. Curran Associates, Inc., 2021. find
  17. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017. find
  18. Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721–1730, 20… find
  19. Danton S Char, Nigam H Shah, and David Magnus. Implementing machine learning in health care—addressing ethical challenges. The New England journal of medicine, 378(11):981, 2018. find
  20. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. find
  21. Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. nature, 542(7639):115–118, 2017. find
  22. Gabriel J Escobar, Vincent X Liu, Alejandro Schuler, Brian Lawson, John D Greene, and Patricia Kipnis. Automated identification of adults at risk for in-hospital clinical deterioration. New England Journal of Medicine, 383(20):1951–1960, 2020. find
  23. Riccardo Fogliato, Shreya Chappidi, Matthew Lungren, Paul Fisher, Diane Wilson, Michael Fitzke, Mark Parkinson, Eric Horvitz, Kori Inkpen, and Besmira Nushi. Who goes first? Influences of human-ai workflow on decision making in clinical imaging. In 2022 ACM Conference on Fairness, Accountability, an… find
  24. G Anthony Gorry and G Octo Barnett. Experience with a model of sequential diagnosis. Computers and Biomedical Research, 1(5):490–507, 1968. find
  25. Christi A Grimm. Adverse events in hospitals: A quarter of medicare patients experienced harm in october 2018. Office of Inspector General, I. General, page 117, 2022. find
  26. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. find our card
  27. William J Hall, Mimi V Chapman, Kent M Lee, Yesenia M Merino, Tainayah W Thomas, B Keith Payne, Eugenia Eng, Steven H Day, and Tamera Coyne-Beasley. Implicit racial/ethnic bias among health care professionals and its influence on health care outcomes: a systematic review. American journal of public… find
  28. David E. Heckerman, Eric Horvitz, and Bharat N. Nathwani. Toward normative expert systems: Part I the Pathfinder project. Methods of Information in Medicine, 31:90 – 105, 1992. find
  29. Katharine E Henry, David N Hager, Peter J Pronovost, and Suchi Saria. A targeted real-time early warning score (trewscore) for septic shock. Science translational medicine, 7(299):299ra122–299ra122, 2015. find
  30. Eric Horvitz. Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 159–166, 1999. find
  31. Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009. find
  32. Ayanna Howard, Cha Zhang, and Eric Horvitz. Addressing bias in machine learning algorithms: A pilot study on emotion recognition for intelligent systems. In 2017 IEEE Workshop on Advanced Robotics and its Social Impacts, pages 1–7. IEEE, 2017. find
  33. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146, 2019. find our card
  34. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021. find our card
  35. Abigail Z Jacobs and Hanna Wallach. Measurement and fairness. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 375–385, 2021. find
  36. Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLOS Digital Health… find
  37. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022. find our card
  38. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. find
  39. Ankit Kulshrestha and Ilya Safro. Fairlearn: Configurable and interpretable algorithmic fairness. arXiv preprint arXiv:2111.08878, 2021. — [arXiv:2111.08878](https://arxiv.org/abs/2111.08878) find
  40. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, et al. Holistic evaluation of language models, 2022. find
  41. Valentin Liévin, Christoffer Egeberg Hother, and Ole Winther. Can large language models reason about medical questions? arXiv preprint arXiv:2207.08143, 2022. — [arXiv:2207.08143](https://arxiv.org/abs/2207.08143) find
  42. Robert S Ledley and Lee B Lusted. Reasoning foundations of medical diagnosis: Symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science, 130(3366):9–21, 1959. find
  43. Hussein Mozannar, Gagan Bansal, Adam Fourney, and Eric Horvitz. Reading between the lines: Modeling user behavior and costs in ai-assisted programming. arXiv preprint arXiv:2210.14306, 2022. — [arXiv:2210.14306](https://arxiv.org/abs/2210.14306) find
  44. Microsoft. Error Analysis. https://www.erroranalysis.ai/, 2023. [Online; accessed 18-March-2023]. ↗
  45. Scott Mayer McKinney, Marcin Sieniek, Varun Godbole, Jonathan Godwin, Natasha Antropova, Hutan Ashrafian, Trevor Back, Mary Chesus, Greg S Corrado, Ara Darzi, et al. International evaluation of an AI system for breast cancer screening. Nature, 577(7788):89– 94, 2020. find
  46. Besmira Nushi and Rahee Ghosh Peshawaria. Responsible AI Mitigations and Tracker: New open-source tools for guiding mitigations in Responsible AI. https://bit.ly/3ZXc0Fq, 2023. [Online; accessed 18-March-2023]. ↗
  47. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. find our card
  48. Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223, 2019. — [arXiv:1909.09223](https://arxiv.org/abs/1909.09223) find
  49. Alexandru Niculescu-Mizil and Rich Caruana. Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning, pages 625–632, 2005. find
  50. David E Newman-Toker, Susan M Peterson, Shervin Badihian, Ahmed Hassoon, Najlla Nassery, Donna Parizadeh, Lisa M Wilson, Yuanxi Jia, Rodney Omron, Saraniya Thar-marajah, et al. Diagnostic errors in the emergency department: A systematic review, 2022. find
  51. OpenAI. Gpt-4 technical report, 2023. find our card
  52. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464):447–453, 2019. find
  53. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022. find
  54. Ramesh S Patil, Peter Szolovits, and William B Schwartz. Causal understanding of patient illness in medical diagnosis. In IJCAI, volume 81, pages 893–899, 1981. find
  55. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on Health, Inference, and Learning, pages 248–260. PMLR, 2022. find our card
  56. Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, et al. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225, 2017. — [arXiv:1711.05225](https://arxiv.org/abs/1711.05225) find
  57. Kristen Reeder and Hwan Lee. Impact of artificial intelligence on us medical students’ choice of radiology. Clinical Imaging, 81:67–71, 2022. find
  58. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. arXiv preprint arXiv, 2018. find
  59. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138, 2022. find our card
  60. Harini Suresh, Nathan Hunt, Alistair Johnson, Leo Anthony Celi, Peter Szolovits, and Marzyeh Ghassemi. Clinical intervention prediction and understanding with deep neural networks. In Machine Learning for Healthcare Conference, pages 322–337. PMLR, 2017. find
  61. Edward H Shortliffe. Mycin: A knowledge-based computer program applied to infectious diseases. In Proceedings of the Annual Symposium on Computer Application in Medical Care, page 66. American Medical Informatics Association, 1977. find
  62. Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, and Eric Horvitz. An empirical analysis of backward compatibility in machine learning systems. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3272–3280, 2020. find
  63. Daniel Susskind and Richard Susskind. The future of the professions. Proceedings of the American Philosophical Society, 162(2):125–138, 2018. find
  64. Eric Topol. Deep medicine: how artificial intelligence can make healthcare human again. Hachette UK, 2019. find
  65. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. find
  66. Jenna Wiens, John Guttag, and Eric Horvitz. Patient risk stratification with time-varying parameters: a multitask learning approach. The Journal of Machine Learning Research, 17(1):2797–2819, 2016. find
  67. Bryan Wilder, Eric Horvitz, and Ece Kamar. Learning to complement humans. arXiv preprint arXiv:2005.00582, 2020. — [arXiv:2005.00582](https://arxiv.org/abs/2005.00582) find
  68. Jenna Wiens, Suchi Saria, Mark Sendak, Marzyeh Ghassemi, Vincent X Liu, Finale Doshi-Velez, Kenneth Jung, Katherine Heller, David Kale, Mohammed Saeed, et al. Do no harm: a roadmap for responsible machine learning for health care. Nature medicine, 25(9):1337– 1340, 2019. find
  69. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. find our card
  70. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. find our card