Elamur Research⌕
☰
home / papers / 2026 / Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches

Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches

Paper · 2026 · Noopur Zambare, Kiana Aghakasiri, Carissa Lin et al.

Summary

Clinical notes must have personal identifiers removed before they can be shared, and recent studies increasingly use very large LLMs for this. The authors ask two questions. How much do such models cost at inference time compared with lighter alternatives? And do de-identifiers stay reliable across note formats, genders, and identifiers following non-Anglophone naming conventions? They benchmark fine-tuned BERT, ClinicalBERT and ModernBERT, small Llama and Qwen models (about 1–8B parameters), Llama 3.3-70B, Qwen2.5-72B and rule-based tools. The test data are MIMIC-III discharge summaries and a private set of rheumatology referral letters, with synthetic identifiers from several locales inserted via the Faker library. The large LLMs process text roughly an order of magnitude more slowly, while compact models perform comparably. Encoders fine-tuned on a few hundred notes can even beat the large models on culturally diverse identifiers. No gender-related gap appeared. The released BERT-MultiCulture-DEID models were trained on multi-locale identifiers and generalize better across cultures. They give hospitals a cheaper and fairer de-identification option.

Key points

  • Models compared: BERT, ClinicalBERT and ModernBERT were tested alongside Llama (1B/3B/8B) and Qwen (1.5B/3B/7B) and the large Llama 3.3-70B and Qwen2.5-72B. Efficiency was measured as words processed per second against recall.
  • Data:
    • 4,000 MIMIC-III discharge summaries: 2,000 for training (subsets of 250/500/1,000/2,000), 1,000 for validation and 1,000 held out for testing.
    • A proprietary rheumatology referral-letter corpus, used to test transfer across formats.
  • Cultural test sets: Identifiers were drawn from Mandarin, Hindi, Spanish, French and Bengali, plus regional English variants. Gendered names were also checked, and no performance disparity by gender was found.
  • Fine-tuning results: Going from 250 to 1,000 training notes improved performance only slightly. After training on all language variants, most models showed a generalization gap under 1%.
  • Released resource: BERT-MultiCulture-DEID is a set of BERT, ClinicalBERT and ModernBERT models fine-tuned on MIMIC with multi-variant identifiers. It is intended as a more robust alternative to the widely used OBI de-identification model and is available via Zenodo (doi 10.5281/zenodo.18342291).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AlbertaEdmonton · Canada51 works · 5,017 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Noopur Zambare, Kiana Aghakasiri, Carissa Lin, Carrie Ye, J Ross Mitchell and Mohamed Abdalla. 2026. Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.222

@inproceedings{zambare2026fair,
  title = {{Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches}},
  author = {Noopur Zambare, Kiana Aghakasiri, Carissa Lin, Carrie Ye, J Ross Mitchell and Mohamed Abdalla},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.222},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.222}
}

References

47 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kiana Aghakasiri, Noopur Zambare, JoAnn Thai, Carrie Ye, Mayur Mehta, J Ross Mitchell, and Mohamed Abdalla. 2025. Not what the doctor ordered: Surveying llm-based de-identification and quantifying clinical information loss. arXiv preprint arXiv:2509.14464. find
  2. Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. 2019. Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323.
  3. Bayan Altalla’, Sameera Abdalla, Ahmad Altamimi, Layla Bitar, Amal Al Omari, Ramiz Kardan, and Iyad Sultan. 2025. Evaluating gpt models for clinical note de-identification. Scientific Reports, 15(1):3852. find
  4. Amazon Web Services. 2018. Amazon comprehend medical. https://aws.amazon.com/comprehend/ medical/. ↗
  5. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  7. Mikhail Budnikov, Anna Bykova, and Ivan P Yamshchikov. 2025. Generalization potential of large language models. Neural Computing and Applications, 37(4):1973–1997. find
  8. Centers for Medicare & Medicaid Services. 1996. The Health Insurance Portability and Accountability Act of 1996 (HIPAA). Online at http://www.cms.hhs.gov/hipaa/. ↗
  9. Fangyi Chen, Syed Mohtashim Abbas Bokhari, Kenrick Cato, Gamze Gürsoy, and Sarah Rossetti. 2024. Examining the generalizability of pretrained de-identification transformer models on narrative nursing notes. Applied Clinical Informatics, 15(02):357– 367. find
  10. Qingyu Chen, Yan Hu, Xueqing Peng, Qianqian Xie, Qiao Jin, Aidan Gilson, Maxwell B Singer, Xuguang Ai, Po-Ting Lai, Zhizheng Wang, and 1 others. 2025. Benchmarking large language models for biomedical natural language processing applications and recommendations. Nature communications, 16(1):3280. find
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  12. Océane Dorémus, Dylan Russon, Benjamin Contrand, Ariel Guerra-Adames, Marta Avalos-Fernandez, Cédric Gil-Jardiné, Emmanuel Lagarde, and 1 others. 2025. Harnessing moderate-sized language models for reliable patient data deidentification in emergency department records: Algorithm development, validat… find
  13. David A Dorr, WF Phillips, Shobha Phansalkar, Shannon A Sims, and John Franklin Hurdle. 2006. Assessing the difficulty and time cost of de-identification in clinical narratives. Methods of information in medicine, 45(03):246–252. find
  14. Margaret Douglass, Gari D Clifford, Andrew Reisner, George B Moody, and Mark Rg. 2004. Computer-assisted de-identification of free text in the mimic ii database. In Computers in Cardiology, 2004, pages 341–344. IEEE. find
  15. Ethnologue. 2025. What are the top 200 most spoken languages? https://www.ethnologue.com/ insights/ethnologue200/. ↗
  16. European Union. 2016. General data protection regulation. https://eur-lex.europa.eu/eli/reg/ 2016/679/oj/eng. ↗
  17. Daniele Faraglia and Other Contributors. 2014. Faker. find
  18. Óscar Ferrández, Brett R South, Shuying Shen, F Jeff Friedlin, Matthew H Samore, and Stéphane M Meystre. 2012. Generalizability and comparison of automatic clinical text de-identification methods and resources. In AMIA Annual Symposium Proceedings, volume 2012, page 199. find
  19. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  20. Paul M Heider, Jihad S Obeid, and Stéphane M Meystre. 2020. A comparative analysis of speed and accuracy for three off-the-shelf de-identification tools. AMIA Summits on Translational Science Proceedings, 2020:241. find
  21. Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342.
  22. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, and 1 others. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. find
  23. Alistair EW Johnson, Lucas Bulgarelli, and Tom J Pollard. 2020. Deidentification of free-text medical records using pre-trained bidirectional transformers. In Proceedings of the ACM conference on health, inference, and learning, pages 214–221. find
  24. Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9.
  25. Woojin Kim, Sungeun Hahm, and Jaejin Lee. 2024. Generalizing clinical de-identification models by privacy-safe data augmentation using gpt-4. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21204– 21218. find
  26. Zengjian Liu, Buzhou Tang, Xiaolong Wang, and Qingcai Chen. 2017. De-identification of clinical notes via recurrent neural network and conditional random field. Journal of biomedical informatics, 75:S34– S42. find
  27. Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. 2022. PEFT: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft. ↗
  28. Stephane M Meystre, Gary Underwood, and Paul Heider. 2023. Clinideid, an open source solution for accurate clinical text de-identification. Studies in Health Technology and Informatics. find
  29. Microsoft. 2018. Presidio - Data Protection and De-identification SDK. find
  30. Mindee. 2021. doctr: Document text recognition. https://github.com/mindee/doctr. ↗
  31. Callandra Moore, Lucas Bulgarelli, Tom Pollard, and Alistair Johnson. 2023. Transformer-deid: Deidentification of free-text clinical notes with transformers. find
  32. Marco Naguib, Xavier Tannier, and Aurelie Neveol. 2024. Few-shot clinical entity recognition in english, french and spanish: masked language models outperform generative model prompting. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6829–6852. find
  33. National Library of Medicine. 2019. Nlm scrubber. https://lhncbc.nlm.nih.gov/scrubber/. ↗
  34. Ishna Neamatullah, Margaret M Douglass, Li-Wei H Lehman, Andrew Reisner, Mauricio Villarroel, William J Long, Peter Szolovits, George B Moody, Roger G Mark, and Gari D Clifford. 2008. Automated de-identification of free-text medical records. BMC medical informatics and decision making, 8(1):32. find
  35. Beau Norgeot, Kathleen Muenzen, Thomas A Peterson, Xuancheng Fan, Benjamin S Glicksberg, Gundolf Schenk, Eugenia Rutenberg, Boris Oskotsky, Marina Sirota, Jinoos Yazdany, and 1 others. 2020. Protected health information filter (philter): accurately and securely de-identifying free-text clinical note… find
  36. OBI Organization. 2022. deid_bert_i2b2. https://github.com/obi-ml-public/ehr_ deidentification. ↗
  37. David Pissarra, Isabel Curioso, João Alveira, Duarte Pereira, Bruno Ribeiro, Tomás Souper, Vasco Gomes, André V Carreiro, and Vitor Rolla. 2024. Unlocking the potential of large language models for clinical text anonymization: A comparative study. arXiv preprint arXiv:2406.00062. find
  38. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, and 1 others. 2018. Improving language understanding by generative pre-training. OpenAI technical report. find
  39. Amber Stubbs and Özlem Uzuner. 2015. Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/uthealth corpus. Journal of biomedical informatics, 58:S20–S29. find
  40. Vaakesan Sundrelingam, Shireen Parimoo, Frances Pogacar, Radha Koppula, Saeha Shin, Chloe Pou-Prom, Surain B Roberts, Amol A Verma, and Fahad Razak. 2025. pydeid: an improved, fast, flexible, and generalizable rule-based approach for deidentification of free-text medical records. JAMIA open, 8(1):oo… find
  41. Latanya Sweeney. 1998. Strategies for de-identifying patient data for research. find
  42. Qwen Team. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2. find
  43. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, and 1 others. 2024. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning an…
  44. Isabella C Wiest, Marie-Elisabeth Leßmann, Fabian Wolf, Dyke Ferber, Marko Van Treeck, Jiefu Zhu, Matthias P Ebert, Christoph Benedikt Westphalen, Martin Wermke, and Jakob Nikolas Kather. 2025. Deidentifying medical documents with local, privacy-preserving large language models: The llm-anonymizer.… find
  45. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transf…
  46. Yuxin Xiao, Shulammite Lim, Tom Joseph Pollard, and Marzyeh Ghassemi. 2023. In the name of fairness: assessing the bias in clinical record de-identification. find
  47. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 123– 137. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zambare-2026-fair-efficient-de-identification-quantifying/