Elamur Research⌕
☰
home / papers / 2026 / MeSHClass-ES and AnatEM-ES: Open Resources for Spanish Biomedical NLP

MeSHClass-ES and AnatEM-ES: Open Resources for Spanish Biomedical NLP

Paper · 2026 · Santiago Martinez Novoa, Lina Gomez Mesa, Juan Prieto et al.

Summary

Spanish biomedical NLP has far fewer open datasets and systematic evaluations than English. This is especially true for anatomy-level entity recognition and for MeSH topic classification at PubMed scale. The authors build two translated, bilingual corpora to fill this gap.

  • MeSHClass-ES is a set of PubMed abstracts rendered into Spanish sentence by sentence with Opus-MT.
  • AnatEM-ES converts the English AnatEM NER corpus using GPT-4o. The model translates text chunks and keeps the BIO tags intact in the same step. Automatic checks and manual review follow.

Both corpora reach a mean COMET score of 0.73. The authors then compare nine encoders (general Spanish, domain-specific and multilingual) and four open-weight 7–9B decoders. The decoders are tested with zero-shot, few-shot, retrieval-augmented and QLoRA setups.

RigoBERTa-2.0 performs best on both tasks. Domain pretraining and model size both help, with a larger effect on NER. XLM-RoBERTa-large is a solid multilingual fallback. Decoders can approach encoders on classification after QLoRA but fall far behind on NER. Fine-tuned encoders therefore remain the practical choice for Spanish biomedical tasks.

Key points

  • MeSHClass-ES: 44,604 translated PubMed articles, of which 29,063 carry major-topic MeSH labels and form the classification benchmark. The label distribution is imbalanced at roughly 9:1.
  • AnatEM-ES: 1,212 documents with 13,849 entity mentions across 12 anatomical types. GPT-4o translated chunks of at most 500 tokens. B-tag counts matched the source in 84.5% of documents. Manual verification covered 600 documents in full, with hybrid checks on the other 612.
  • Encoder results: RigoBERTa-2.0 reaches a classification micro-F1 of 0.69, tied with SciBETO-large, and an NER F1 of 0.66. XLM-RoBERTa-large achieves a macro-F1 of .578 on classification.
  • Decoder results: QLoRA-tuned Gemma-2-9B reaches a micro-F1 of .61, about 88% of the best encoder. On NER, no decoder exceeds F1 .27. Under QLoRA, NER collapses to F1 ≤ .004 because the models output all-O tags or malformed tag sequences.
  • Availability: Both corpora are on the HuggingFace Hub (Flaglab), and the translation and evaluation code is on GitHub. Translation quality (COMET 0.73) probably limits absolute scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Universidad de Los Andes.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

Cite

Cite as Santiago Martinez Novoa, Lina Gomez Mesa, Juan Prieto and Ruben Manrique. 2026. MeSHClass-ES and AnatEM-ES: Open Resources for Spanish Biomedical NLP. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bionlp-1.49

@inproceedings{novoa2026meshclass,
  title = {{MeSHClass-ES and AnatEM-ES: Open Resources for Spanish Biomedical NLP}},
  author = {Santiago Martinez Novoa, Lina Gomez Mesa, Juan Prieto and Ruben Manrique},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.49},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.49}
}

References

37 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Casimiro Pio Carrino, Jordi Armengol-Estapé, Ona de Gibert Bonet, Asier Gutiérrez-Fandiño, Aitor Gonzalez-Agirre, Martin Krallinger, and Marta Villegas. 2021a. Spanish biomedical crawled corpus: A large, diverse dataset for spanish biomedical language models. Preprint, arXiv:2109.07765. — [arXiv:2109.07765](https://arxiv.org/abs/2109.07765) find
  2. Casimiro Pio Carrino, Jordi Armengol-Estapé, Asier Gutiérrez-Fandiño, Joan Llop-Palao, Marc Pàmies, Aitor Gonzalez-Agirre, and Marta Villegas. 2021b. Biomedical and clinical language models for spanish: On the benefits of domain-specific pretraining in a mid-resource scenario. Preprint, arXiv:2109.0… — [arXiv:2109.03570](https://arxiv.org/abs/2109.03570) find
  3. José Cañete, Gabriel Chaperon, Rodrigo Fuentes, Jou-Hui Ho, Hojin Kang, and Jorge Pérez. 2023. Spanish pre-trained bert model and evaluation data. Preprint, arXiv:2308.02976. find
  4. Qingyu Chen, Yan Hu, Xueqing Peng, Qianqian Xie, Qiao Jin, Aidan Gilson, Maxwell B. Singer, Xuguang Ai, Po-Ting Lai, Zhizheng Wang, Vipina K. Keloth, Kalpana Raja, Jimin Huang, Huan He, Fongci Lin, Jingcheng Du, Rui Zhang, W. Jim Zheng, Ron A. Adelman, and 2 others. 2025. Benchmarking large language… find
  5. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. Preprint, arXiv:1911.02116. find
  6. Javier de la Rosa, Eduardo G. Ponferrada, Paulo Villegas, Pablo Gonzalez de Prado Salas, Manu Romero, and Marıa Grandury. 2022. Bertin: Efficient pre-training of a spanish language model using perplexity sampling. Preprint, arXiv:2207.06814. — [arXiv:2207.06814](https://arxiv.org/abs/2207.06814) find
  7. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Preprint, arXiv:2305.14314. \#\# Ethical Considerations Zi-Yi Dou and Graham Neubig. 2021. Word alignment by fine-tuning embeddings on parallel corpora. pages 2112–2128.
  8. All data are derived from publicly available PubMed abstracts and the open AnatEM corpus. Patient data or personally identifiable information Flaglab. 2024. Scibeto collection. https://huggingface.co/collections/Flaglab/ scibeto. Accessed: 2026-04-17. ↗
  9. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André F. T. Martins. 2022. Results of WMT22 metrics shared task: Stop using BLEU – neural metrics are better and more robust. In Proceedings of the Seventh Conferen…
  10. Frederik Gade, Ole Lund, and Marie Lisandra Mendoza. 2025. Benchmarking zero-shot biomedical relation triplet extraction across language model architectures. In Proceedings of the 24th Workshop on Biomedical Language Processing, pages 88–100, Viena, Austria. Association for Computational Linguistics… find
  11. Guillem García Subies, Álvaro Barbero Jiménez, and Paloma Martínez Fernández. 2024. A comparative analysis of spanish clinical encoder-based models on NER and classification tasks. J. Am. Med. Inform. Assoc., 31(9):2137–2146. find
  12. Félix Gaschi, Xavier Fontaine, Parisa Rastin, and Yannick Toussaint. 2023. Multilingual clinical NER: Translation or cross-lingual transfer? In Proceedings of the 5th Clinical Natural Language Processing Workshop, pages 289–311, Toronto, Canada. Association for Computational Linguistics. find
  13. Aitor Gonzalez-Agirre, Montserrat Marimon, Ander Intxaurrondo, Obdulia Rabal, Marta Villegas, and Martin Krallinger. 2019. PharmaCoNER: Pharmacological substances, compounds and proteins named entity recognition track. In Proceedings of the 5th Workshop on BioNLP Open Shared Tasks, pages 1–10, Hong… find
  14. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  15. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare, 3(1):1–23.
  16. Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2020. Clinicalbert: Modeling clinical notes and predicting hospital readmission. Preprint, arXiv:1904.05342.
  17. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  18. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. BioMistral: A collection of open-source pretrained large language models for medical domains. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5848–5864, Bang…
  19. Vojtech Lanz and Pavel Pecina. 2025. When multilingual models compete with monolingual domain-specific models in clinical question answering. In Proceedings of the Second Workshop on Patient-Oriented Language Processing (CL4Health), pages 69–82, Albuquerque, New Mexico. Association for Computational… find
  20. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240.
  21. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6). find
  22. Montserrat Marimon, Aitor Gonzalez-Agirre, Ander Intxaurrondo, Heidy Rodriguez, Jose Antonio Lopez Martin, Marta Villegas, and Martin Krallinger. 2019. Automatic de-identification of medical texts in Spanish: the MEDDOCAN track, corpus, guidelines, methods and evaluation of results. In IberLEF@ SEPL… find
  23. Antonio Miranda-Escalada, Luis Gasco, Salvador Lima-López, Eulàlia Farré-Maduell, da Lisa Jaramillo Estrada, Anastasios Nentidis, Anastasia Krithara, Georgios Katsimpras, Georgios Paliouras, and Martin Krallinger. 2022. Overview of distemist at bioasq: Automatic detection and normalization of diseas… find
  24. Hiroki Nakayama. 2018. seqeval: A python framework for sequence labeling evaluation. Software available from https://github.com/chakki-works/seqeval. ↗
  25. Mariana Neves, Antonio Jimeno Yepes, Aurélie Névéol, Rachel Bawden, Giorgio Maria Di Nunzio, Roland Roller, Philippe Thomas, Federica Vezzani, Maika Vicente Navarro, Lana Yeganova, Dina Wiemann, and Cristian Grozea. 2023. Findings of the WMT 2023 biomedical translation shared task: Evaluation of Cha… find
  26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  27. Shengwen Peng, Ronghui You, Hongning Wang, Chengxiang Zhai, Hiroshi Mamitsuka, and Shanfeng Zhu. 2016. DeepMeSH: deep semantic representation for improving large-scale MeSH indexing. Bioinformatics, 32(12):i70–i79. find
  28. Sampo Pyysalo and Sophia Ananiadou. 2014. Anatomical entity mention recognition at literature scale. Bioinformatics, 30(6):868–875. find
  29. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  30. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
  31. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. Preprint, arXiv:1908.10084.
  32. Guillem García Subies, Álvaro Barbero Jiménez, and Paloma Martínez Fernández. 2025. Clintext-sp and rigoberta clinical: a new set of open resources for spanish clinical nlp. Preprint, arXiv:2503.18594. large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics, 1… — [arXiv:2503.18594](https://arxiv.org/abs/2503.18594) find
  33. Alejandro Vaca Serrano, Guillem Garcia Subies, Helena Montoro Zamorano, Nuria Aldama Garcia, Doaa Samy, David Betancur Sanchez, Antonio Moreno Sandoval, Marta Guerrero Nieto, and Alvaro Barbero Jimenez. 2022. Rigoberta: A state-of-the-art language model for spanish. Preprint, arXiv:2205.10233. — [arXiv:2205.10233](https://arxiv.org/abs/2205.10233) find
  34. Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, and Guoyin Wang. 2023. Gpt-ner: Named entity recognition via large language models.
  35. Chih-Hsuan Wei, Hung-Yu Kao, and Zhiyong Lu. 2015. Gnormplus: an integrative approach for tagging genes, gene families, and protein domains. BioMed Research International, 2015:918710. find
  36. Guangxu Xun, Kishlay Jha, Ye Yuan, Yaqing Wang, and Aidong Zhang. 2019. MeSHProbeNet: a self-attentive probe net for MeSH indexing. In Bioinformatics, volume 35, pages 3794–3802. find
  37. David Yarowsky, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the First International Conference on Human Language Technology Research (HLT). Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/novoa-2026-meshclass-es-anatem-es-open/