Elamur Research
☰
home / papers / 2018 / Explainable Prediction of Medical Codes from Clinical Text

Explainable Prediction of Medical Codes from Clinical Text

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N18-1100.pdf

Summary

Assigning ICD codes to clinical notes is slow and error-prone when done by hand. Automating it is hard because the label space contains thousands of codes and clinical writing is noisy. The authors propose CAML (Convolutional Attention for Multi-Label classification). A CNN encodes a discharge summary. Then a separate attention distribution for each code picks out the text n-grams most relevant to that code, instead of using a single pooling step shared by all labels. A variant, DR-CAML, adds a regularizer that pulls each code's output weights toward an embedding of its textual description, which helps rarely seen codes.

Experiments use ICU discharge summaries from MIMIC-III (all codes and the 50 most frequent) and from MIMIC-II. CAML beats earlier systems in every setting, including SVM-based, HA-GRU and LSTM approaches. On full MIMIC-III it reaches precision@8 of 0.71 and Micro-F1 of 0.54.

A physician also rated the text snippets each system highlighted as explanations, and CAML's attention-selected snippets were judged useful. This matters for decision support, where coders need to see why a code was suggested.

Key points

  • MIMIC-III full setting:
    • 8,921 distinct ICD-9 codes (6,918 diagnosis, 2,003 procedure).
    • Training set of 47,724 summaries from 36,998 patients, plus 1,632 for validation and 3,372 for testing.
    • Data are split by patient so no patient appears in both training and test.
  • Additional settings:
    • A top-50-codes subset with 8,067 / 1,574 / 1,730 summaries for train / validation / test.
    • MIMIC-II with 5,031 labels and the Perotte et al. split of 20,533 training and 2,282 test examples.
  • Preprocessing and embeddings: purely numeric tokens are dropped, text is lowercased, rare words are mapped to UNK, and word2vec embeddings have dimension 100.
  • Human evaluation on 100 examples (informative / highly informative):
    • CAML: 46 / 22
    • Code-description similarity: 48 / 20
    • Logistic regression: 41 / 18
    • Max-pooling CNN: 36 / 13
  • Released resources and efficiency: code, data splits and pretrained models are available at github.com/jamesmullenbach/caml-mimic. The convolutional models matched or beat the recurrent ones (e.g. Bi-GRU) while being much cheaper to compute.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

151 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 148 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1100

@inproceedings{mullenbach2018explainable,
  title = {{Explainable Prediction of Medical Codes from Clinical Text}},
  author = {Mullenbach, James and Wiegreffe, Sarah and Duke, Jon and Sun, Jimeng and Eisenstein, Jacob},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1100},
  url = {https://doi.org/10.18653/v1/n18-1100}
}

References

39 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning. pages 2091–2100. find
  2. Anand Avati, Kenneth Jung, Stephanie Harman, Lance Downing, Andrew Ng, and Nigam H. Shah. 2017. Improving palliative care with deep learning. arXiv preprint arXiv:1711.06402 . — [arXiv:1711.06402](https://arxiv.org/abs/1711.06402) find
  3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations. find our card
  4. Tal Baumel, Jumana Nassour-Kassis, Michael Elhadad, and Noémie Elhadad. 2018. Multi-label classification of patient notes a case study on ICD code assignment. In AAAI Workshop on Health Intelligence. find
  5. Elena Birman-Deych, Amy D. Waterman, Yan Yan, David S. Nilasena, Martha J. Radford, and Brian F Gage. 2005. Accuracy of ICD-9-CM codes for identifying cardiovascular and stroke risk factors. Medical care 43(5):480–485. find
  6. Edward Choi, Mohammad Taha Bahadori, Andy Schuetz, Walter F Stewart, and Jimeng Sun. 2016. Doctor AI: Predicting clinical events via recurrent neural networks. In Machine Learning for Healthcare Conference. pages 301–318. find
  7. Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017. Language modeling with gated convolutional networks. In Proceedings of the 34th International Conference on Machine Learning. pages 933–941. http://proceedings. mlr.press/v70/dauphin17a.html. find our card
  8. Luciano R.S. de Lima, Alberto H.F. Laender, and Berthier A. Ribeiro-Neto. 1998. A hierarchical approach to the automatic categorization of medical documents. In Proceedings of the seventh international conference on Information and knowledge management. ACM, pages 132–139. find
  9. Joshua C. Denny, Marylyn D. Ritchie, Melissa A. Basford, Jill M. Pulley, Lisa Bastarache, Kristin Brown-Gentry, Deede Wang, Dan R. Masys, Dan M. Roden, and Dana C. Crawford. 2010. PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene–disease associations. Bioinformatics 26(9)… find
  10. Cıcero Nogueira dos Santos, Ming Tan, Bing Xiang, and Bowen Zhou. 2016. Attentive pooling networks. CoRR, abs/1602.03609 . — [arXiv:1602.03609](https://arxiv.org/abs/1602.03609) find
  11. Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Liwei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data 3. find our card
  12. Aaditya Prakash, Siyuan Zhao, Sadid A Hasan, Vivek V Datla, Kathy Lee, Ashequl Qadir, Joey Liu, and Oladimeji Farri. 2017. Condensed memory networks for clinical diagnostic inferencing. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. pages 3274–3280. find
  13. Ramakanth Kavuluru, Anthony Rios, and Yuan Lu. 2015. An empirical evaluation of supervised learning approaches in assigning diagnosis codes to electronic medical records. Artificial intelligence in medicine 65(2):155–166. find
  14. Rajesh Ranganath, Adler J. Perotte, Noémie Elhadad, and David M. Blei. 2015. The survival filter: Joint survival analysis with a latent time series. In UAI. pages 742–751. find
  15. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pages 1746–1751. find our card
  16. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations. find
  17. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 107– 117. https://aclweb.org/anthology/ D16-1011. ↗
  18. Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016. Visualizing and understanding neural models in NLP. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics… find our card
  19. Andrew McCallum. 1999. Multi-label text classification with a mixture model trained by EM. In AAAI workshop on Text Learning. pages 1–7. find
  20. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. pages 3111–3119. find
  21. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, pages 1135–1144. find our card
  22. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. 2016. Reasoning about entailment with neural attention. In International Conference on Learning Representations. find our card
  23. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 379–389. http://aclw… find
  24. Elyne Scheurwegs, Boris Cule, Kim Luyckx, Léon Luyten, and Walter Daelemans. 2017. Selecting relevant features from the electronic health record for clinical code prediction. Journal of Biomedical Informatics 74:92–103. find
  25. Elyne Scheurwegs, Kim Luyckx, Léon Luyten, Walter Daelemans, and Tim Van den Bulcke. 2015. Data integration of structured and unstructured sources for assigning clinical codes to patient stays. Journal of the American Medical Informatics Association 23(e1):e11–e19. find
  26. Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, and Eric P Xing. 2017. Towards automated ICD coding using deep learning. arXiv preprint arXiv:1711.04075 . — [arXiv:1711.04075](https://arxiv.org/abs/1711.04075) find
  27. Aurélie Névéol, Robert N Anderson, K Bretonnel Cohen, Cyril Grouin, Thomas Lavergne, Grégoire Rey, Aude Robert, Claire Rondet, and Pierre Zweigenbaum. 2017. CLEF ehealth 2017 multilingual information extraction task overview: ICD10 coding of death certificates in english and french. In CLEF 2017 Eva… find
  28. Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, Curran Associates, Inc., pages 2951–2959. find
  29. Adler Perotte, Rimma Pivovarov, Karthik Natarajan, Nicole Weiskopf, Frank Wood, and Noémie Elhadad. 2013. Diagnosis code assignment: models and evaluation metrics. Journal of the American Medical Informatics Association 21(2):231–237. find
  30. Michael Subotin and Anthony R. Davis. 2014. A system for predicting ICD-10-PCS codes from electronic health records. In Proceedings of the 2014 Workshop on Biomedical Natural Language Processing. find
  31. Michael Subotin and Anthony R. Davis. 2016. A method for modeling co-occurrence propensity of clinical codes with application to ICD-10-PCS autocoding. Journal of the American Medical Informatics Association 23(5):866–871. find
  32. Kevin Swersky, Jasper Snoek, and Ryan P Adams. 2013. Multi-task bayesian optimization. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, Curran Associates, Inc., pages 2004–2012. find
  33. Ankit Vani, Yacine Jernite, and David Sontag. 2017. Grounded recurrent neural networks. arXiv preprint arXiv:1705.08557 . — [arXiv:1705.08557](https://arxiv.org/abs/1705.08557) find
  34. Sen Wang, Xiaojun Chang, Xue Li, Guodong Long, Lina Yao, and Quan Z Sheng. 2016. Diagnosis code assignment using sparsity-based disease correlation embedding. IEEE Transactions on Knowledge and Data Engineering 28(12):3191–3202. find
  35. World Health Organization. 2016. International statistical classification of diseases and related health problems 10th revision. http: //apps.who.int/classifications/ icd10/browse/2016/en. find
  36. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find our card
  37. Wenpeng Yin and Hinrich Schütze. 2017. Attentive convolution. arXiv preprint arXiv:1710.00519 . — [arXiv:1710.00519](https://arxiv.org/abs/1710.00519) find
  38. Wenpeng Yin, Hinrich Schütze, Bing Xiang, and Bowen Zhou. 2016. ABCNN: Attention-based convolutional neural network for modeling sentence pairs. Transactions of the Association for Computational Linguistics 4:259–272. find our card
  39. Danchen Zhang, Daqing He, Sanqiang Zhao, and Lei Li. 2017. Enhancing automatic ICD-9-CM code assignment for medical texts with pubmed. find