Elamur Research⌕
☰
home / papers / 2026 / tt501 at ArchEHR-QA 2026: Few-Shot Prompting with Retrieval-Augmented Generation for Grounded Clinical EHR Question Answering

tt501 at ArchEHR-QA 2026: Few-Shot Prompting with Retrieval-Augmented Generation for Grounded Clinical EHR Question Answering

Paper · 2026 · Tai Tan Tran

Summary

Team tt501 entered the ArchEHR-QA 2026 shared task, which addresses automatic replies to patient questions sent through EHR portals. Such replies must be grounded in specific sentences of the patient's clinical notes. The author tackled three connected subtasks: locating the supporting note sentences (Subtask 2), writing the replies (Subtask 3) and linking each reply sentence to its evidence (Subtask 4). The data are 167 MIMIC-derived cases.

All systems rely purely on prompting xAI's reasoning models grok-4-fast-reasoning and grok-4-1-fast-reasoning, with no fine-tuning and no extra data. For evidence selection, two approaches were compared:

  • a BM25 filter followed by LLM reranking;
  • the full note with a recall-oriented Chain-of-Thought prompt, followed by a second LLM pass that cleans up the selection.

The full-note approach performed better. Answers were then generated in a RAG fashion from the predicted evidence, with three clinician-written examples in the prompt. For alignment, few-shot exemplars explained why each citation holds, and the patient's own wording was added to the prompt. Both changes improved recall of many-to-many links.

The best runs scored 58.84 Strict Micro-F1, 31.36 overall and 79.11 Micro-F1 on the three subtasks. This suggests careful prompt design on strong reasoning LLMs can compete in grounded clinical QA without task-specific training.

Key points

  • Dataset: ArchEHR-QA 2026 has 167 cases from MIMIC, with sentence-level labels (essential / supplementary / not-relevant) and clinician answers. Subtasks 2–3 are tested on 47 cases (IDs 121–167) and Subtask 4 on 147 cases (IDs 21–167).
  • Subtask 2: The full-context ensemble-refine-v1 run beat hybrid-bm25-llm (top-20 BM25 candidates).
    • Strict F1 rose from 52.9 to 58.8, and Lenient F1 reached 67.8.
    • The gain mixes the prompt change with a model upgrade, so it cannot be credited to prompting alone.
  • Subtask 3: rag-fewshot-v1 improved the overall score (mean of BLEU, ROUGE-L, SARI, BERTScore, AlignScore and MEDCON) from 28.1 to 31.4 over zero-shot full-note generation. MEDCON was the only metric that dipped slightly.
  • Subtask 4: few-shot-reasoning-v2 reached a best Micro-F1 of 79.11. It used four exemplars annotated with citation rationales and an explicit hint that answer sentences often draw on several note sentences.
  • Setup and availability: Temperature was 0, outputs were JSON, and a regex fallback was needed in under 3% of responses. Code and prompts are released at github.com/taitran501/tt501-archehr-qa-2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vietnam National University Ho Chi Minh CityHo Chi Minh City · Vietnam21 works · 14 citations🎓University of Information TechnologyHo Chi Minh City · Vietnam13 works · 7 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksMedicine · 241 worksHealth Informatics · 184 works

🧭 Topics 2Research subjects and languages the work is about.Question answering · 441 worksInformation retrieval & search · 473 works

Cite

Cite as Tai Tan Tran. 2026. tt501 at ArchEHR-QA 2026: Few-Shot Prompting with Retrieval-Augmented Generation for Grounded Clinical EHR Question Answering. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2tjqwa7c7nqf

@inproceedings{tran2026tt501,
  title = {{tt501 at ArchEHR-QA 2026: Few-Shot Prompting with Retrieval-Augmented Generation for Grounded Clinical EHR Question Answering}},
  author = {Tai Tan Tran},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2tjqwa7c7nqf},
  url = {https://doi.org/10.63317/2tjqwa7c7nqf}
}

References

13 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asma Ben Abacha, Wen-wai Yim, George Michalopoulos, and Thomas Lin. 2023. An investigation of evaluation metrics for automated medical note generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 15128–15143. find
  2. Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data, 3:160035.
  3. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81.
  4. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), pages 311–318.
  5. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. find
  6. Christine Sinsky, Lacey Colligan, Ling Li, Mirela Prgomet, Sam Reynolds, Lindsey Goeders, Johanna Westbrook, Michael Tutty, and George Blike. 2016. Allocation of physician time in ambulatory practice: A time and motion study in 4 specialties. Annals of Internal Medicine, 165(11):753–760. find
  7. Sarvesh Soni and Dina Demner-Fushman. 2026a. A dataset for addressing patient’s information needs related to clinical course of hospitalization. Scientific Data. find
  8. Subtask 2, Run 1: hybrid-bm25-llm The BM25 first stage selects the top-20 candidates using the clinician question as the query. The LLM reranker then receives the following prompt, where {clinician_question} is the question text and the note is limited to the BM25-selected sentences. System message.… find
  9. Sarvesh Soni, Soumya Gayen, and Dina Demner-Fushman. 2025. Overview of the ArchEHR-QA 2025 shared task on grounded question answering from electronic health records. In Proceedings of the BioNLP Workshop at ACL 2025. xAI. 2025. Grok-4: Advancing reasoning and scientific discovery. https://x.ai/blog/… ↗
  10. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. In Transactions of the Association for Computational Linguistics, volume 4, pages 401–415.
  11. Subtask 2, Run 2: ensemble-refine-v1 The full note excerpt is provided to the model. The prompt is recall-oriented and uses two annotated development examples as few-shot demonstrations. System message. You are a precise medical evidence classifier. Respond ONLY with valid JSON. User message templat… find
  12. Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. 2023. AlignScore: Evaluating factual consistency with a unified alignment function. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 11328–11348.
  13. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations (ICLR). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tran-2026-tt501-archehr-qa-2026-few/