Elamur Research⌕
☰
home / papers / 2026 / Smart_solutions at MEDIQA-SYNUR 2026: A Multi-Stage LLM Pipeline for Nursing Observation Extraction

Smart_solutions at MEDIQA-SYNUR 2026: A Multi-Stage LLM Pipeline for Nursing Observation Extraction

Paper · 2026 · Prateek Munjal

Summary

The paper describes a system for the MEDIQA-SYNUR 2026 shared task. The task is to turn conversational, fragmented nurse dictations into structured EHR entries covering 192 fields. These fields can be binary, categorical or free-text. The author did not fine-tune a model. Instead, he built a zero-shot pipeline of five prompted agents, all running on the general-purpose Kimi-K2 model:

  1. An extraction step collects every explicit observation with its evidence, aiming for high recall.
  2. A lightweight TF-IDF retriever narrows the large schema to a small set of candidate fields.
  3. An LLM grades how well the transcript supports each candidate, which discards unsupported fields.
  4. A strictly evidence-bound step fills in the values.
  5. A final step formats the output for submission.

The main design idea is to separate deciding which fields are relevant from deciding what their values are. This targets hallucinations and over-generation of defaults. The system placed third among participating teams, with precision and recall both at 0.80. This suggests that well-orchestrated prompting of generalist LLMs can be a practical alternative to costly domain fine-tuning in resource-limited clinical settings.

Key points

  • Result: Third place in MEDIQA-SYNUR 2026, with precision, recall and F1 all around 0.80 on the unseen test set.
  • Agents: The pipeline runs, in order, an Observation Agent, then an Ontology Matching Agent, a Relevance Scoring Agent, an Evidence Assignment Agent, and finally a Formatting Agent. All use Kimi-K2 zero-shot with different system prompts.
  • Retrieval: Each field's name and enumerated values form a TF-IDF corpus (unigrams plus bigrams, ℓ2-normalized). Each observation is used as a query against it with cosine similarity, keeping the top-k = 15 fields with positive scores.
  • Relevance filtering: Candidate fields get a support score from 1 to 5, and only fields scoring ≥ 4 move on to value assignment, which protects precision.
  • Value assignment: This step uses Medprompt-inspired multi-step reasoning. Categorical fields (SINGLE_SELECT/MULTI_SELECT) must take values from the provided enumerations, and STRING fields use short phrases taken from the transcript.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksNursing · 5 worksIssues, ethics and legal aspects · 4 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Prateek Munjal. 2026. Smart_solutions at MEDIQA-SYNUR 2026: A Multi-Stage LLM Pipeline for Nursing Observation Extraction. Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/27y4nwpv3xyg

@inproceedings{munjal2026smart,
  title = {{Smart\_solutions at MEDIQA-SYNUR 2026: A Multi-Stage LLM Pipeline for Nursing Observation Extraction}},
  author = {Prateek Munjal},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/27y4nwpv3xyg},
  url = {https://doi.org/10.63317/27y4nwpv3xyg}
}

References

39 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Huda Adam, Jialing Lin, Jason Lin, Hannah Keenan, Andrew Wilson, and Marzyeh Ghassemi. 2024. Clinical information extraction with large language models: A case study on organ procurement. In AMIA Annual Symposium Proceedings, pages 115–123. find
  3. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  4. Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. 2022. Large language models are few-shot clinical information extractors. arXiv preprint arXiv:2205.12689.
  5. Fernando R Altermatt, Andres Neyem, Nicolás I Sumonte, Ignacio Villagrán, Marcelo Mendoza, Hector J Lacassie, and Alejandro E Delfino. 2025. Evaluating gpt-4o in high-stakes medical assessments: performance and error analysis on a chilean anesthesiology exam. BMC Medical Education, 25(1):1499. find
  6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  7. Ehsaneddin Asgari, Natalia Montaña-Brown, Marie Dubois, Sally Khalil, James Balloch, Jonathan A. Yeung, and Daniela Pimenta. 2025. A framework to assess clinical safety and hallucination rates of llms for medical text summarisation. npj Digital Medicine, 8(1):274. find
  8. Jiacheng Hu, Runyuan Bao, Yang Lin, Hanchao Zhang, and Yanlin Xiang. 2024. Accurate medical named entity recognition through specialized nlp models. In 2024 6th International Conference on Frontier Technologies of Information and Computer (ICFTIC), pages 578–582. IEEE. find
  9. Clément Christophe, Praveen K Kanithi, Prateek Munjal, Tathagata Raha, Nasir Hayat, Ronnie Rajan, Ahmed Al-Mahrooqi, Avani Gupta, Muhammad Umar Salman, Gurpreet Gosal, et al. 2024. Med42–evaluating fine-tuning strategies for medical llms: full-parameter vs. parameter-efficient approaches. arXiv prep… — [arXiv:2404.14779](https://arxiv.org/abs/2404.14779) find
  10. Thelma C Hurd, Fay Cobb Payton, Darryl B Hood, et al. 2024. Targeting machine learning and artificial intelligence algorithms in health care to reduce bias and improve population health. The Milbank Quarterly, 102(3):577. find
  11. AHM Karim and Ozlem Uzuner. 2025. Assessing large language models for structured medical order extraction. arXiv preprint arXiv:2510.10475. — [arXiv:2510.10475](https://arxiv.org/abs/2510.10475) find
  12. Jean-Philippe Corbeil, Asma Ben Abacha, Jérôme Tremblay, Phillip Swazinna, Akila Jeeson Daniel, Miguel Del-Agua, and François Beaulieu. 2025a. Overview of the mediqa-oe 2025 shared task on medical order extraction from doctor-patient consultations. In Proceedings of the 7th Clinical Natural Language… find
  13. Min Soo Kim, Paul Chung, Nima Aghaeepour, and Namkug Kim. 2025. Information extraction from clinical texts with generative pre-trained transformer models. International Journal of Medical Sciences, 22(5):1015–1028. find
  14. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  15. Jean-Philippe Corbeil, Asma Ben Abacha, George Michalopoulos, Phillip Swazinna, Miguel Del-Agua, Jerome Tremblay, Akila Jeeson Daniel, Cari Bader, Kevin Cho, Pooja Krishnan, Nathan Bodenstab, Thomas Lin, Wenxuan Teng, Francois Beaulieu, and Paul Vozila. 2025b. Empowering healthcare practitioners wit… find
  16. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240.
  17. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  18. Adriano Friganović, Polona Selič, Biljana Ilić, and Biserka Sedić. 2019. Stress and burnout syndrome and their associations with coping and job satisfaction in critical care nurses: a literature review. Psychiatria Danubina, 31(Suppl 1):21–31. find
  19. Hui Liu, Ziyi Chen, Peilin Li, Yuan-Zhi Liu, Xiangtao Liu, Ronald X Xu, and Mingzhai Sun. 2025a. Resource-efficient instruction tuning of large language models for biomedical named entity recognition. Journal of Biomedical Informatics, page 104896. find
  20. Cari Bader Nate Bodenstab Asma Ben Abacha George Michalopoulos, Jean-Philippe Corbeil. 2026. Overview of the mediqa-synur 2026 shared task on observation extraction from nurse dictations. In Proceedings of the 8th Clinical Natural Language Processing Workshop, ClinicalNLP@LREC 2026, Palma, Mallorca,…
  21. Longchao Liu, Long Lian, Yiyan Hao, Aidan Pace, Elaine Kim, Nour Homsi, Yash Pershad, Liheng Lai, Thomas Gracie, Ashwin Kishtagari, et al. 2025b. Human level information extraction from clinical reports with finetuned language models. Scientific Reports, 15(1):45239. find
  22. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, 31(3):943–950. find
  23. Ling Luo, Po-Ting Lai, Chih-Hsuan Wei, Cecilia N Arighi, and Zhiyong Lu. 2022. Biored: a rich biomedical relation extraction dataset. Briefings in Bioinformatics, 23(5):bbac282. find
  24. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. 2025. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534.
  25. Zoltan P Majdik, S Scott Graham, Jade C Shiva Edward, Sabrina N Rodriguez, Martha S Karnes, Jared T Jensen, Joshua B Barbour, and Justin F Rousseau. 2024. Sample size considerations for fine-tuning large language models for named entity recognition tasks: methodological study. Jmir ai, 3:e52095. find
  26. Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, et al. 2024. Towards conversational diagnostic ai. arXiv preprint arXiv:2401.05654. — [arXiv:2401.05654](https://arxiv.org/abs/2401.05654) find
  27. Matthew D. McHugh, Ann Kutney-Lee, Jeannie P. Cimiotti, Douglas M. Sloane, and Linda H. Aiken. 2011. Nurses’ widespread job dissatisfaction, burnout, and frustration with health benefits signal problems for patient care. Health Affairs, 30(2):202–210. find
  28. Robin Willard-Grace, Margaret Knox, Bruce Huang, Heather Hammer, Chris Kivlahan, and Kevin Grumbach. 2019. Burnout and health care workforce turnover. Annals of Family Medicine, 17(1):36–41. find
  29. Parth Mehta. 2025. Pnlp at mediqa-oe 2025: A zero-shot prompting strategy with gemini for medical order extraction. In Proceedings of the 7th Clinical Natural Language Processing Workshop, pages 75–83. find
  30. Qianqi Yan, Huy Nguyen, Sumana Srivatsa, Hari Bandi, Xin Eric Wang, and Krishnaram Kenthapadi. 2026. Cite-while-you-generate: Training-free evidence attribution for multimodal clinical summarization. arXiv preprint arXiv:2601.16397. — [arXiv:2601.16397](https://arxiv.org/abs/2601.16397) find
  31. Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, et al. 2023. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. arXiv preprint arXiv:2311.16452. find
  32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  33. Vasileios Ntinopoulos, Hector Rodriguez Cetina Biefer, Igor Tudorache, Nestoras Papadopoulos, Dragan Odavic, Petar Risteski, Achim Haeussler, and Omer Dzemali. 2025. Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performanc… find
  34. Musarrat Zeba, Abdullah Al Mamun, Kishoar Jahan Tithee, Debopom Sutradhar, Mohaimenul Azam Khan Raiaan, Saddam Mukta, Reem E Mohamed, Md Rafiqul Islam, Yakub Sebastian, Mukhtar Hussain, et al. 2025. Mitigating hallucinations in healthcare llms with granular fact-checking and domain-specific adaptati… — [arXiv:2512.16189](https://arxiv.org/abs/2512.16189) find
  35. Cecilia Padilla Fortunatti and Yaritza K. Palmeiro-Silva. 2017. Effort-reward imbalance and burnout among icu nursing staff: A cross-sectional study. Nursing Research, 66(5):410–416. find
  36. Xiaoxuan Zhan, Michael Humbert-Droz, Partha Mukherjee, and Olivier Gevaert. 2021. Structuring clinical text with AI: Old versus new natural language processing techniques evaluated on eight common cardiovascular diseases. Patterns, 2(7):100289. find
  37. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. 2025. Medgemma technical report. arXiv preprint arXiv:2507.05201. find
  38. Zhiyu Zhang and Arbee LP Chen. 2022. Biomedical named entity recognition with the combined feature attention and fully-shared multi-task learning. BMC bioinformatics, 23(1):458. find
  39. Soorya Ram Shimgekar, Shayan Vassef, Abhay Goyal, Navin Kumar, and Koustuv Saha. 2025. Agentic ai framework for end-to-end medical data inference. arXiv preprint arXiv:2507.18115. — [arXiv:2507.18115](https://arxiv.org/abs/2507.18115) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/munjal-2026-smart-solutions-mediqa-synur-2026/