Elamur Research⌕
☰
home / papers / 2026 / Training Biomedical Retrievers From Large-Scale Citation Contexts

Training Biomedical Retrievers From Large-Scale Citation Contexts

Paper · 2026 · Xing David Wang, Duy Le Thanh and Ulf Leser

Summary

Strong biomedical dense retrievers such as MedCPT depend on PubMed search and click logs, which are not publicly available. This paper asks whether openly accessible citation sentences ("citances") can replace such logs as weak supervision. The authors processed the OPCitance dataset, which covers PubMed Central full texts up to May 2019. They kept only references with valid PubMed IDs and citances of at most 64 tokens, which left about 62 million citance–abstract pairs.

On these pairs they trained CiteRec, which has two parts: a PubMedBERT-based bi-encoder retriever and a cross-encoder reranker. They evaluated it on three benchmarks:

  • the biomedical portion of BEIR,
  • SciRepEval,
  • CitancePlus, a new citation recommendation test set built from post-2019 articles.

CiteRec roughly matches MedCPT on BEIR and beats it on SciRepEval. For citation recommendation over all of PubMed, CiteRec with its reranker outperforms Qwen3-Embedding-8B, a far larger model, at much lower cost. The work therefore shows that freely available citation data can serve as a privacy-safe and reproducible training source for biomedical retrieval.

Key points

  • Training data: About 62M citance–abstract pairs from OPCitance (PMC, cutoff May 2019). The new CitancePlus test set has about 93K unseen pairs from later articles.

  • Retriever training:

    • Queries and documents each get their own encoder, with both starting from PubMedBERT-base weights and using the [CLS] embedding.
    • An InfoNCE-style loss that allows multiple positives, combining query-to-document and document-to-query terms (α = 0.8).
    • Each pair is weighted by log(1 + Relative Citation Ratio).

  • Hard negatives: Pairs from the same full text are placed in the same batch, so related documents act as hard negatives. Using random in-batch negatives instead lowers NDCG@10 from 0.206 to 0.164 on CitancePlus and from 0.425 to 0.412 on BEIR.

  • CitancePlus results (NDCG@10):

    ModelNDCG@10
    CiteRec retriever0.206
    CiteRec + reranker0.240
    Qwen3-Embedding-8B0.216
    BM250.161
    MedCPT0.097

  • Stricter test: On a filtered subset of about 25K pairs whose cited abstracts never appear in training, the retriever scores 0.205 and the full pipeline 0.248. This suggests little bias toward documents seen during training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Humboldt-Universität zu BerlinBerlin · Germany19 works · 41,895 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Xing David Wang, Duy Le Thanh and Ulf Leser. 2026. Training Biomedical Retrievers From Large-Scale Citation Contexts. BioNLP 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bionlp-1.7

@inproceedings{wang2026training,
  title = {{Training Biomedical Retrievers From Large-Scale Citation Contexts}},
  author = {Xing David Wang, Duy {Le Thanh} and Ulf Leser},
  booktitle = {BioNLP 2026},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-1.7},
  url = {https://doi.org/10.18653/v1/2026.bionlp-1.7}
}

References

28 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, and 1 others. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
  2. Qiao Jin, Won Kim, Qingyu Chen, Donald C Comeau, Lana Yeganova, W John Wilbur, and Zhiyong Lu. 2023. Medcpt: Contrastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval. Bioinformatics, 39(11):btad651. find
  3. Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. A full-text learning to rank dataset for medical information retrieval. In European Conference on Information Retrieval, pages 716–722. Springer. find
  4. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. Preprint, arxiv:2308.03281 [cs].
  5. Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld. 2020. Specter: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 2270–2282.
  6. Yuan Liang, Massimo Poesio, and Roonak Rezvani. 2025. Beyond citations: Integrating finding-based relations for improved biomedical article representations. In Proceedings of the 24th Workshop on Biomedical Language Processing, pages 297–306. Association for Computational Linguistics. find
  7. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solomatin, Ömer Veysel… — MMTEB: Massive Multilingual Text Embedding Benchmark
  8. Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020. S2orc: The semantic scholar open research corpus. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4969–4983. Association for Computational Linguistics.
  9. Zoran Medić and Jan Snajder. 2020. Improved local citation recommendation based on context enhanced with global information. In Proceedings of the First Workshop on Scholarly Document Processing, pages 97–103. Association for Computational Linguistics. find
  10. Tobias J Erb. 2011. Carboxylases in natural and synthetic microbial pathways. Applied and environmental microbiology, 77(24):8466–8477. find
  11. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
  12. Nianlong Gu, Yingqiang Gao, and Richard H. R. Hahnloser. 2022. Local citation recommendation with hierarchical-attention text encoder and SciBERT-based reranking. Preprint, arxiv:2112.01206 [cs]. — [arXiv:2112.01206](https://arxiv.org/abs/2112.01206) find
  13. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23.
  14. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  15. Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein, Bela Gipp, and Georg Rehm. 2022. Neighborhood contrastive learning for scientific document representations with citation embeddings. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11670–11688. As… find
  16. Qi He, Jian Pei, Daniel Kifer, Prasenjit Mitra, and Lee Giles. 2010. Context-aware citation recommendation. In Proceedings of the 19th international conference on World wide web, pages 421–430. find
  17. Navid Rekabsaz, Oleg Lesota, Markus Schedl, Jon Brassey, and Carsten Eickhoff. 2021. Tripclick: the log files of a large health web search engine. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2507–2513. find
  18. Yoshiaki Yamada, Yasusuke Inoue, Kogenta Nakamura, Katsuya Naruse, Shigeyuki Aoki, Tomohiro Taki, Motoi Tobiume, Kenji Zennami, Remi Katsuda, Kouji Hara, and 1 others. 2009. Long-term results and management of ureteral transitional cell carcinoma using the holmium: Yag laser via rigid-ureteroscopy.… find
  19. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond, volume 4. Now Publishers Inc. find
  20. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. Preprint, arxiv:2506.05176 [cs].
  21. M Janina Sarol, Shufan Ming, Shruthan Radhakrishna, Jodi Schneider, and Halil Kilicoglu. 2024. Assessing citation integrity in biomedical publications: Corpus annotation and NLP models. page btae420. find
  22. Amanpreet Singh, Mike D’Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. 2023. SciRepEval: A multi-format benchmark for scientific document representations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5548–5566. Association for Computational Li… find
  23. Aarush Sinha, Pavan Kumar S, Roshan Balaji, and Nirav Pravinbhai Bhatt. 2025. BiCA: Effective biomedical dense retrieval with citation-aware hard negatives. Preprint, arxiv:2511.08029 [cs]. — [arXiv:2511.08029](https://arxiv.org/abs/2511.08029) find
  24. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1. find
  25. George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, and 1 others. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question… find
  26. Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. 2021. Trec-covid: constructing a pandemic information retrieval test collection. In ACM SIGIR Forum, volume 54, pages 1–12. ACM New York, NY, USA. find
  27. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550. Association for Comp…
  28. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-training-biomedical-retrievers-large-scale/