Elamur Research⌕
☰
home / papers / 2026 / Razreshili at ArchEHR-QA 2026: Evidence Alignment via LLM Prompting and Cross-Encoder Fine-tuning

Razreshili at ArchEHR-QA 2026: Evidence Alignment via LLM Prompting and Cross-Encoder Fine-tuning

Paper · 2026 · Arina Zemchyk

Summary

This paper presents the author's submission to the ArchEHR-QA 2026 shared task, targeting Subtask 4, known as Evidence Alignment. The task links each sentence of a clinician-written answer to the sentence(s) in a MIMIC-derived clinical note excerpt that support it. Three things make it hard: links can be many-to-many, answers paraphrase or combine note content, and some answer sentences come from clinical knowledge rather than the note, so they should get no citation.

The author compares two approaches:

  • Cross-encoder: ms-marco-MiniLM-L-12-v2 with a two-stage, threshold-based pipeline. It is used zero-shot and also after LoRA fine-tuning with a mixed InfoNCE + BCE objective, trained on hard negatives taken from human relevance labels.
  • LLM prompting: quantized Qwen2.5-7B-Instruct with chain-of-thought few-shot prompts.

The prompted LLM scored best, at 67.93 micro F1 on the test set. The fine-tuned cross-encoder reached 64.44 but runs far faster. The work suggests that, when clinical training data is scarce, LLM reasoning handles entailment and unsupported sentences better than relevance-trained rankers. It also documents score compression and collapse when fine-tuning on very small contrastive sets.

Key points

  • Data: the development set has 20 cases and 95 answer sentences. Of these, 77 have gold citations and 18 (about 19%) need an empty citation set. The test set has 147 cases, and micro F1 is the main metric.
  • Base model choice: four zero-shot cross-encoders were compared, including S-PubMedBert-MS-MARCO and MedCPT-Cross-Encoder. The general-domain ms-marco-MiniLM-L-12-v2 beat the biomedical ones, reaching 0.723 dev macro F1 with thresholds t_nc=0.2 and t_c=0.4.
  • Fine-tuning setup: LoRA (r=4, α=8) on 138 contrastive groups, with the loss InfoNCE (τ=0.2) + 0.3·BCE. Hard negatives were drawn in priority order: "supplementary" sentences first, then sentences within ±2 positions of the gold citation. On test this gave P 63.27, R 65.66, F1 64.44.
  • Best system: Qwen2.5-7B-Instruct in 4-bit with three CoT examples and JSON output scored P 73.78, R 62.95, F1 67.93. One of the examples shows a case with no supporting sentence; without it, the model over-cited inference-based sentences.
  • Efficiency and calibration: on one T4 GPU, the LLM needed about 30 minutes versus under 1 minute for the cross-encoder. An early cross-encoder run without sigmoid calibration reached only 59.95 F1, with recall of 49.02.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksBiochemistry, Genetics and Molecular Biology · 263 worksMolecular Biology · 260 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksQuestion answering · 487 works

Cite

Cite as Arina Zemchyk. 2026. Razreshili at ArchEHR-QA 2026: Evidence Alignment via LLM Prompting and Cross-Encoder Fine-tuning. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5mop8iu8k9ej

@inproceedings{zemchyk2026razreshili,
  title = {{Razreshili at ArchEHR-QA 2026: Evidence Alignment via LLM Prompting and Cross-Encoder Fine-tuning}},
  author = {Arina Zemchyk},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/5mop8iu8k9ej},
  url = {https://doi.org/10.63317/5mop8iu8k9ej}
}

References

16 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sai Prasanna Teja Reddy Bogireddy, Abrar Ma-jeedi, Viswanath Gajjala, Zhuoyan Xu, Siddhant Rai, and Vaishnav Potlapalli. 2025. Neural at ArchEHR-QA 2025: Agentic prompt optimization for evidence-grounded clinical question answering. In Proceedings of the 24th Workshop on Biomedical Language Processi… find
  2. Bohao Chu, Qianli Wang, Hendrik Damm, Hui Wang, Ula Muhabbek, Elisabeth Livingstone, Christoph M. Friedrich, and Norbert Fuhr. 2026. etracer: Towards traceable text generation via claim-level grounding. arXiv preprint arXiv:2601.03669. — [arXiv:2601.03669](https://arxiv.org/abs/2601.03669) find
  3. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  4. Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. 2022. Understanding dimensional collapse in contrastive self-supervised learning. In International Conference on Learning Representations. find
  5. Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data, 3:160035.
  6. Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. In Advances in Neural Information Processing Systems, volume 35, pages 1950–1965.
  7. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated MAchine reading COmprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches, volume 1773. CEUR-WS.org.
  8. Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. arXiv preprint arXiv:1901.04085.
  9. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language…
  10. Guilherme Rosa, Luiz Bonifacio, Vitor Jeronymo, Hugo Abonizio, Marzieh Fadaee, Roberto Lotufo, and Rodrigo Nogueira. 2022. In defense of cross-encoders for zero-shot retrieval. arXiv preprint arXiv:2212.06121. — [arXiv:2212.06121](https://arxiv.org/abs/2212.06121) find
  11. Nima Sheikholeslami et al. 2025. Optimizing what matters: AUC-driven learning for robust neural retrieval. arXiv preprint arXiv:2510.00137. — [arXiv:2510.00137](https://arxiv.org/abs/2510.00137) find
  12. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, 620:172– 180.
  13. Sarvesh Soni and Dina Demner-Fushman. 2026a. A dataset for addressing patient’s information needs related to clinical course of hospitalization. Scientific Data. find
  14. Sarvesh Soni and Dina Demner-Fushman. 2026b. Overview of the archehr-qa 2026 shared task on grounded question answering from electronic health records. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health), Palma, Mallorca (Spain). ELRA. find
  15. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. find
  16. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: A large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pa…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zemchyk-2026-razreshili-archehr-qa-2026-evidence/