Elamur Research⌕
☰
home / papers / 2018 / Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces

Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces

Summary

Clinical coding datasets such as MIMIC follow a power-law distribution. A handful of ICD-9 codes appear thousands of times, while most are rare or never appear in training data. Common metrics like micro-F1 or P@k hide how models handle these rare codes. Rios and Kavuluru address both the evaluation and the modeling side of this problem. They adapt the generalized zero-shot protocol of Xian et al. (2017) to multi-label classification, scoring frequent, few-shot and zero-shot labels separately. Their model, ZAGCNN, extends the label-wise attention CNN (ACNN) of Mullenbach et al. (2018). Instead of learning an independent vector for every code, it builds attention and output vectors from each code's textual descriptor. A two-layer GCNN then refines these vectors over the ICD-9 hierarchy, so the network can score codes it never saw during training. On discharge summaries from MIMIC II and MIMIC III, the method substantially improves recall for rare and unseen codes, with only a small loss on frequent ones. This matters because missed rare codes cause billing errors.

Key points

  • In MIMIC III, over half of roughly 17,000 ICD-9 diagnosis/procedure codes never occur, and about 5,000 occur only 1–10 times.
  • The architecture assumes the labels form a DAG and each label has a short natural-language descriptor. It combines a CNN over EMR text, label-wise attention, and a 2-layer GCNN (Kipf and Welling, 2017) over the label graph.
  • Compared with prior work, R@10 on few-shot labels rises by 6.2% (MIMIC II) and 4.8% (MIMIC III). On zero-shot labels the gains are 17.3% and 19%. ZAGCNN beats the strongest ESZSL baseline by about 16–20% on zero-shot labels.
  • ACNN remains slightly better on frequent labels (by about 0.3% R@10 on MIMIC III) and also on overall P@10 and macro-F1. The authors show that its macro-F1 advantage comes from frequent labels, which reveals a trade-off between frequent and rare labels.
  • When all labels are ranked together, a correct few-shot code reaches the top 16 for only 6% of the test documents that contain one. The authors therefore propose future work on novelty detection and on transferring knowledge from MeSH/UMLS.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

 Institutions

🎓University of KentuckyLexington · United States2 works · 259 citations

Cite

Cite as Anthony Rios and Ramakanth Kavuluru. 2018. Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1352

@inproceedings{rios2018few,
  title = {{Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces}},
  author = {Anthony Rios and Ramakanth Kavuluru},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1352},
  url = {https://doi.org/10.18653/v1/d18-1352}
}

References

47 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. 2016. Label-embedding for image classification. IEEE transactions on pattern analysis and machine intelligence, 38(7):1425–1438. find
  2. Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning, pages 2091–2100. find
  3. Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Simaan. 2017. Graph convolutional encoders for syntax-aware neural machine translation. In Conference on Empirical Methods in Natural Language Processing, pages 1957– 1967. our card
  4. Tal Baumel, Jumana Nassour-Kassis, Michael Elhadad, and Noemie Elhadad. 2018. Multi-label classification of patient notes a case study on icd code assignment. In AAAI Joint Workshop on Health Intelligence. find
  5. Kush Bhatia, Himanshu Jain, Purushottam Kar, Manik Varma, and Prateek Jain. 2015. Sparse local embeddings for extreme multi-label classification. In Advances in Neural Information Processing Systems, pages 730–738. find
  6. Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl 1):D267– D270. find
  7. Meihao Chen, Zhuoru Lin, and Kyunghyun Cho. 2017. Graph convolutional networks for classification with a structured label space. arXiv preprint arXiv:1710.04908. — [arXiv:1710.04908](https://arxiv.org/abs/1710.04908) find
  8. Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pages 3844–3852. our card
  9. Alistair EW Johnson, Tom J Pollard, Lu Shen, Liwei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data, 3. our card
  10. V Jouhet, G Defossez, A Burgun, P le Beux, P Levillain, P Ingrand, and V Claveau. 2012. Automated classification of free-text pathology reports for registration of incident cases of cancer. Methods of Information in Medicine, 51(3):242–251. find
  11. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modeling sentences. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 655–665. find
  12. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751. our card
  13. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR). find
  14. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR). our card
  15. Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2. find
  16. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1506–1515. our card
  17. Thomas Mensink, Efstratios Gavves, and Cees GM Snoek. 2014. Costa: Co-occurrence statistics for zero-shot classification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2441–2448. find
  18. Tomas Mikolov, Andrea Frome, Samy Bengio, Jonathon Shlens, Yoram Singer, Greg S Corrado, Jeffrey Dean, and Mohammad Norouzi. 2013. Zero-shot learning by convex combination of semantic embeddings. In International Conference on Learning Representations (ICLR). find
  19. James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explainable prediction of medical codes from clinical text. In North American Chapter of the Association for Computational Linguistics (NAACL). our card
  20. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning (ICML), pages 807–814. find
  21. Jinseok Nam, Jungi Kim, Eneldo Loza Mencı́a, Iryna Gurevych, and Johannes Fürnkranz. 2014. Large-scale multi-label text classification - revisiting neural networks. In Machine Learning and Knowledge Discovery in Databases, ECML PKDD, pages 437– 452. find
  22. NCHS. 1978. International classification of diseases, ninth revision, clinical modification (icd-9-cm). Hy-attsville (MD): National Center for Health Statistics. find
  23. Yannis Papanikolaou, Grigorios Tsoumakas, Manos Laliotis, Nikos Markantonatos, and Ioannis P Vlahavas. 2015. Auth-atypon at bioasq 3: Large-scale semantic indexing in biomedicine. In CLEF (Working Notes). find
  24. Ioannis Partalas, Aris Kosmopoulos, Nicolas Baskiotis, Thierry Artières, George Paliouras, Éric Gaussier, Ion Androutsopoulos, Massih-Reza Amini, and Patrick Gallinari. 2015. LSHTC: A benchmark for large-scale text classification. CoRR, abs/1503.08581. find
  25. Hao Peng, Jianxin Li, Yu He, Yaopeng Liu, Mengjiao Bao, Lihong Wang, Yangqiu Song, and Qiang Yang. 2018. Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In World Wide Web Conference on World Wide Web, pages 1063–1072. find
  26. Adler Perotte, Rimma Pivovarov, Karthik Natarajan, Nicole Weiskopf, Frank Wood, and Noémie Elhadad. 2013. Diagnosis code assignment: models and evaluation metrics. Journal of the American Medical Informatics Association, 21(2):231–237. find
  27. S. Pyysalo, F. Ginter, H. Moen, T. Salakoski, and S. Ananiadou. 2013. Distributional semantics resources for biomedical text processing. In LBM, pages 39–44. find
  28. Nikhil Rao, Hsiang-Fu Yu, Pradeep K Ravikumar, and Inderjit S Dhillon. 2015. Collaborative filtering with graph information: Consistency and scalable methods. In Advances in neural information processing systems, pages 2107–2115. find
  29. A Rios and R Kavuluru. 2015. Analyzing the moving parts of a large-scale multi-label text classification pipeline: Experiences in indexing biomedical articles. In IEEE International Conference on Healthcare Informatics, volume 2015, pages 1–7. find
  30. Anthony Rios and Ramakanth Kavuluru. 2018. Emr coding with semi-parametric multi-head matching networks. In North American Chapter of the Association for Computational Linguistics (NAACL), volume 1, pages 2081–2091. find
  31. Bernardino Romera-Paredes and Philip Torr. 2015. An embarrassingly simple approach to zero-shot learning. In International Conference on Machine Learning, pages 2152–2161. find
  32. Timothy N Rubin, America Chambers, Padhraic Smyth, and Mark Steyvers. 2012. Statistical topic models for multi-label document classification. Machine Learning, 88(1-2):157–208. find
  33. Ubai Sandouk and Ke Chen. 2016. Multi-label zero-shot learning via concept embedding. arXiv preprint arXiv:1606.00282. — [arXiv:1606.00282](https://arxiv.org/abs/1606.00282) find
  34. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In Proceedings of 15th European Semantic Web Conference (ESWC 2018). find
  35. Lei Shu, Hu Xu, and Bing Liu. 2017. Doc: Deep open classification of text documents. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2911–2916. find
  36. Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4080–4090. find
  37. Emma Strubell and Andrew McCallum. 2017. Dependency parsing with dilated iterated graph CNNs. In Workshop on Structured Prediction for Natural Language Processing, SPNLP@EMNLP, pages 1–6. find
  38. Lei Tang, Suju Rajan, and Vijay K Narayanan. 2009. Large scale multi-label classification via metala-beler. In Proceedings of the 18th international conference on World wide web, pages 211–220. ACM. find
  39. George Tsatsaronis, Michael Schroeder, Georgios Paliouras, Yannis Almirantis, Ion Androutsopoulos, Eric Gaussier, Patrick Gallinari, Thierry Artieres, Michael R Alvers, Matthias Zschunke, et al. 2012. Bioasq: A challenge on large-scale biomedical semantic indexing and question answering. In AAAI fal… find
  40. Grigorios Tsoumakas, Ioannis Katakis, and Ioannis P. Vlahavas. 2010. Mining multi-label data. In Data Mining and Knowledge Discovery Handbook, pages 667–685. find
  41. Ankit Vani, Yacine Jernite, and David Sontag. 2017. Grounded recurrent neural networks. arXiv preprint arXiv:1705.08557. — [arXiv:1705.08557](https://arxiv.org/abs/1705.08557) find
  42. Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et al. 2016. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pages 3630–3638. find
  43. Xiaolong Wang, Yufei Ye, and Abhinav Gupta. 2018. Zero-shot recognition via semantic embeddings and knowledge graphs. arXiv preprint arXiv:1803.08035. — [arXiv:1803.08035](https://arxiv.org/abs/1803.08035) find
  44. Jason Weston, Sumit Chopra, and Keith Adams. 2014. \# tagspace: Semantic embeddings from hashtags. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1822–1827. find
  45. Yongqin Xian, Bernt Schiele, and Zeynep Akata. 2017. Zero-shot learning-the good, the bad and the ugly. In IEEE Conference on Computer Vision and Pattern Recognition, pages 3077–3086. IEEE Computer Society. find
  46. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In North American Chapter of the Association for Computational Linguistics (NAACL), pages 1480– 1489. our card
  47. Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit S. Dhillon. 2014. Large-scale multi-label learning with missing labels. In Proceedings of the 31th International Conference on Machine Learning, pages 593–601. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rios-2018-few-shot-zero-shot-multi/