Elamur Research⌕
☰
home / papers / 2026 / Enigma at #SMM4H–HeaRD 2026: Leveraging Multilingual Pre-trained Models for Clinical Named Entity Recognition

Enigma at #SMM4H–HeaRD 2026: Leveraging Multilingual Pre-trained Models for Clinical Named Entity Recognition

Paper · 2026 · Sylvia Vassileva, Plamena Ilieva, Teodor Svetoslavov Kostadinov et al.

Summary

The paper describes the Enigma team's system for MultiClinNER, subtask 1 of the MultiClinAI shared task at SMM4H–HeaRD 2026. The task is to locate mentions of diseases, symptoms and procedures in clinical discharge summaries written in seven languages: Spanish, English, Swedish, Romanian, Italian, Dutch and Czech. The non-Spanish corpora were created by machine-translating a Spanish gold standard, and annotations come as BRAT character spans.

The team trained a separate BIO token-classification encoder for each language and entity type. Long documents were split into sentences or processed with a sliding window. Some models used a CRF layer, and predictions were cleaned up with rule-based post-processing. Language-specific additions included:

  • span-level ensembles combined by majority or weighted voting;
  • noise filtering of the Dutch training data;
  • Czech data augmentation using morphological variants and GPT-4.1-mini paraphrases.

Averaged over the best model per language and category, the system reached an F1 of 0.6502. Ensembles generally beat single models, and monolingual and multilingual encoders proved complementary. The work is a useful reference for clinical NER in languages with little annotated data, especially morphologically rich ones.

Key points

  • Best test results (strict F1):
    • English disease: 0.7452 with XLM-R.
    • Dutch: 0.7119 (disease), 0.7083 (procedure) and 0.6055 (symptom) with a 5-model majority ensemble that includes MedRoBERTa.nl, robbert-v2-dutch-ner and dragon-bert.
    • Zero-shot GliNER_multi-v2.1 used alone performed very poorly, scoring about 0.17 for English disease and 0.11 for Romanian disease.
  • Czech morphological augmentation:
    • The team built a medical dictionary of 1,399 entries grouped into 887 morphological families.
    • Replacing rare entity forms with their variants more than tripled the training data.
    • F1 rose from 0.657 to 0.710–0.712.
  • XLM-R tokenizer fix: SentencePiece byte-fallback produced overlapping tokens. Masking these tokens and post-filtering short spans improved Czech performance by roughly 13–14% depending on entity type.
  • Czech weighted ensemble: Combining XLM-RoBERTa, two RobeCzech checkpoints and zero-shot GPT-4.1-mini reached F1 0.738, 2.4 points above the best single model. Of GPT-4.1-mini's correct predictions, 65.8% were entities that every fine-tuned model missed.
  • Italian ensemble: A 4-model ensemble (bioBIT, medbit-r3-plus, umberto, XLM-R) scored 0.706 strict F1 on disease, compared with 0.348 for medbit-r3-plus alone. The gap came mostly from entity-boundary errors in the single model, which the ensemble avoided. Code is published on GitHub (svassileva/enigma_multiclinai).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sofia University "St. Kliment Ohridski"Sofia · Bulgaria21 works · 2,595 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 worksNamed entity recognition · 200 works

Cite

Cite as Sylvia Vassileva, Plamena Ilieva, Teodor Svetoslavov Kostadinov, Monika Peteva Petkova, Daniel Manchevski, Vitosh Doynov, Ivan Koychev and Svetla Boytcheva. 2026. Enigma at #SMM4H–HeaRD 2026: Leveraging Multilingual Pre-trained Models for Clinical Named Entity Recognition. Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.smm4h-1.19

@inproceedings{vassileva2026enigma,
  title = {{Enigma at \#SMM4H–HeaRD 2026: Leveraging Multilingual Pre-trained Models for Clinical Named Entity Recognition}},
  author = {Sylvia Vassileva, Plamena Ilieva, Teodor Svetoslavov Kostadinov, Monika Peteva Petkova, Daniel Manchevski, Vitosh Doynov, Ivan Koychev and Svetla Boytcheva},
  booktitle = {Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
  year = {2026},
  doi = {10.18653/v1/2026.smm4h-1.19},
  url = {https://doi.org/10.18653/v1/2026.smm4h-1.19}
}

References

22 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Joeran S Bosma, Koen Dercksen, Luc Builtjes, Romain André, Christian Roest, Stefan J Fransen, Constant R Noordman, Mar Navarro-Padilla, Judith Lefkes, Natália Alves, and 1 others. 2025. The dragon benchmark for clinical nlp. NPJ Digital Medicine, 8(1):289. find
  2. Tommaso Mario Buonocore, Claudio Crema, Alberto Redolfi, Riccardo Bellazzi, and Enea Parimbelli. 2023. Localizing in-domain adaptation of transformer-based biomedical language models. Journal of Biomedical Informatics, 144:104431. find
  3. Casimiro Pio Carrino, Jordi Armengol-Estapé, Asier Gutiérrez-Fandiño, Joan Llop-Palao, Marc Pàmies, Aitor Gonzalez-Agirre, and Marta Villegas. 2021. Biomedical and clinical language models for spanish: On the benefits of domain-specific pretraining in a mid-resource scenario. arXiv preprint arXiv:21… — [arXiv:2109.03570](https://arxiv.org/abs/2109.03570) find
  4. Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow. 2022. Clin-x: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain. Bioinformatics, 38(12):3267–3274. find
  5. Salvador Lima-López, Eulàlia Farré-Maduell, Luis Gasco-Sánchez, Jan Rodríguez-Miret, and Martin Krallinger. 2023. Overview of symptemist at biocreative viii: corpus, guidelines and evaluation of systems for the detection and normalization of symptoms, signs and findings from text. In Proceedings of… find
  6. Salvador Lima-López, Eulàlia Farré-Maduell, Jan Rodríguez-Miret, Miguel Rodríguez-Ortega, Livia Lilli, Jacopo Lenkowicz, Giovanna Ceroni, Jonathan Kossof, Anoop Shah, Anastasios Nentidis, and 1 others. 2024. Overview of multicardioner task at bioasq 2024 on medical specialty and language adaptation… find
  7. Salvador Lima López, Judith Rosell, Jan Rodríguez Miret, Fernando Gallego-Donoso, and Martin Krallinger. 2026. MultiClinAI shared task training set + MultiClinNER and MultiClinCorpus test (only texts) and background sets. find
  8. Guillermo Lopez-Garcia, Jose Miguel Acitores Cortina, Jacob Berkowitz, Joey Chan, Ganesh Chandrasekar, Sumon Kanti Dey, Ivan Flores Amaro, Fernando Gallego, Lauren Gryboski, Ari Z Klein, Martin Krallinger, Salvador Lima-López, Tomohiro Nishiyama, Lisa Raithel, Ahmad Rezaie Mianroodi, Roland Roller,… find
  9. Qiuhao Lu, Rui Li, Andrew Wen, Jinlian Wang, Liwei Wang, and Hongfang Liu. 2025. Large language models struggle in token-level clinical named entity recognition. In AMIA Annual Symposium Proceedings, volume 2024, page 748. find
  10. Guillermo López-García, José M. Jerez, Nuria Ribelles, Emilio Alba, and Francisco J. Veredas. 2021. Transformers for clinical coding in spanish. IEEE Access, 9:72387–72397. find
  11. Antonio Miranda-Escalada, Luis Gascó, Salvador Lima-López, Eulàlia Farré-Maduell, Darryl Estrada, Anastasios Nentidis, Anastasia Krithara, Georgios Katsimpras, Georgios Paliouras, and Martin Krallinger. 2022. Overview of distemist at bioasq: Automatic detection and normalization of diseases from cli… find
  12. WikiNEuRal: Combined neural and knowledge-based silver data creation for multilingual NER. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2521–2533, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  13. Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011. 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556. find
  14. Stella Verkijk and Piek Vossen. 2025. Creating, anonymizing and evaluating the first medical language model pre-trained on dutch electronic health records: Medroberta. nl. Artificial Intelligence in Medicine, page 103148. find
  15. Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2024. Gliner: Generalist model for named entity recognition using bidirectional transformer. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol…
  16. A The SentencePiece byte-fallback fix brought substantial precision gains (Table 2), confirming that overlapping token positions were generating a large number of spurious short entity spans. Disease precision improved from 0.565 to 0.708; symptom from 0.564 to 0.695. find
  17. Masoud Monajatipoor, Jiaxin Yang, Joel Stremmel, Melika Emami, Fazlolah Mohaghegh, Mozhdeh Rouhsedaghat, and Kai-Wei Chang. 2024. Llms in biomedicine: A study on clinical named entity recognition. arXiv preprint arXiv:2404.07376. — [arXiv:2404.07376](https://arxiv.org/abs/2404.07376) find
  18. Loreto Parisi, Simone Francia, and Paolo Magnani. 2020. Umberto: an italian language model trained with whole word masking. https://github.com/ musixmatchresearch/umberto. ↗
  19. Milan Straka, Jakub Náplava, Jana Straková, and David Samuel. 2021. Robeczech: Czech roberta, a monolingual contextualized language representation model. In Text, Speech, and Dialogue, pages 197–209, Cham. Springer International Publishing. find
  20. Guillem García Subies, Álvaro Barbero Jiménez, and Paloma Martínez Fernández. 2025. Clintext-sp and rigoberta clinical: a new set of open resources for spanish clinical nlp. arXiv preprint arXiv:2503.18594. — [arXiv:2503.18594](https://arxiv.org/abs/2503.18594) find
  21. Simone Tedeschi, Valentino Maiorca, Niccolò Campolungo, Francesco Cecconi, and Roberto Navigli. 2021. find
  22. Tokenizer Artifact Fix Experiments Entity System find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vassileva-2026-enigma-smm4h-heard-2026-leveraging/