Elamur Research⌕
☰
home / papers / 2026 / Overview of the MEDIQA-SYNUR 2026 Shared Task on Observation Extraction from Nurse Dictations

Overview of the MEDIQA-SYNUR 2026 Shared Task on Observation Extraction from Nurse Dictations

Paper · 2026 · George Michalopoulos, Jean-Philippe Corbeil, Cari Bader et al.

Summary

Hospital nurses record observations in EHR flowsheets, large spreadsheet-like ontologies with hundreds or thousands of rows. Manual entry is slow and adds to burnout. MEDIQA-SYNUR 2026 is presented as the first shared task that turns nurse dictations into normalized observations mapped onto such a flowsheet schema.

The organizers reused SYNUR, a synthetic dataset built by alternating GPT-4o-0806 generation with expert nurse review. The pipeline started from 80 fabricated seed dictations and expanded the concept ontology. Nurses annotated the training/development data, and the test set received an extra disagreement-resolution pass. Systems were scored with F1, with precision and recall reported as secondary metrics. Participants also had to submit code so their runs could be verified.

Thirteen teams reached the final stage. The best ranked system, from Semantica Lab, combined an ensemble of frontier LLMs with LLM-as-a-Judge verification. However, the organizers' unranked GPT-4.1 baseline scored higher. Overall, retrieval-plus-verification pipelines, ensembles and fine-tuned small models were the strongest strategies. Plain prompting performed worse.

Key points

  • Data: the 80 seed dictations yielded 547 observations. These were consolidated into 97 concepts and then expanded to 193 concepts. The final data has 223 train/dev dictations (55:45 split, about 3000 observations) and 200 test dictations.
  • Participation and evaluation: 66 teams registered, 13 finalists submitted code and runs, and each team could submit up to 15 runs. A multi-choice observation counts as correct only if all reference values are present.
  • Baseline: the GPT-4.1 baseline uses segmentation, RAG filtering of schema rows and extraction. It reached F1 0.837, about 2.3 points above the winner.
  • Top systems: Semantica Lab ranked first with F1 0.814. Ranks 2–5 scored between 0.796 and 0.804, mostly using GPT-5, Kimi-K2 MoE or Qwen3 235B.
  • Small and weak systems: NSN fine-tuned a Qwen3 2B model with SFT and GRPO and reached F1 0.740. The bottom three teams relied on prompting Llama 8B/70B models.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States325 works · 142,931 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksNursing · 5 worksIssues, ethics and legal aspects · 4 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as George Michalopoulos, Jean-Philippe Corbeil, Cari Bader, Nathan Bodenstab and Asma Ben Abacha. 2026. Overview of the MEDIQA-SYNUR 2026 Shared Task on Observation Extraction from Nurse Dictations. Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3s6vwtvsw85q

@inproceedings{michalopoulos2026overview,
  title = {{Overview of the MEDIQA-SYNUR 2026 Shared Task on Observation Extraction from Nurse Dictations}},
  author = {George Michalopoulos, Jean-Philippe Corbeil, Cari Bader, Nathan Bodenstab and Asma Ben Abacha},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/3s6vwtvsw85q},
  url = {https://doi.org/10.63317/3s6vwtvsw85q}
}

References

23 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. and Yu Su. 2022. Thinking about gpt-3 in-context learning for biomedical ie? think again. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4497–4512.
  2. Bayu Aryoyudanta, Maria Yuliana, Mikie Rachman, and I Made Agus Setiawan. 2026. Night shift nerds at mediqa-synur 2026: Pushing small large language model capability for clinical observation extraction and normalization from nurse dictation using rlvr. In Proceedings of the 8th Clinical Natural Lang… find
  3. Kyomin Hwang and Nojun Kwak. 2026. Team mkc at mediqa-synur 2026: Retrieval-augmented generation based nurse observation extraction. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain. find
  4. Sy Hwang, Katherine S. Pitcher, Sue Hyon Kim, Yoonjae Lee, Hayoung K. Donelly, Harsh Band-hey, Andrew J. King, Karen O’Connor, Ryan J. Urbanowicz, and Danielle L. Mowery. 2026. Semantica lab at mediqa-synur 2026: Route, extract and verify – an llm-gated ensemble for parsing nurse dictations. In Proc… find
  5. Jean-Philippe Corbeil, Asma Ben Abacha, Jérôme Tremblay, Phillip Swazinna, Akila Jeeson Daniel, Miguel Del-Agua, and François Beaulieu. 2025a. Overview of the mediqa-oe 2025 shared task on medical order extraction from doctor-patient consultations. In Proceedings of the 7th Clinical Natural Language… find
  6. Riham JeebAllah, Adhari AlZaabi, and Abdulrahman Khalifa AAlAbdulsalam. 2026. Squcs at mediqa-synur 2026: A multi-agent open source llm system for nursing observation extraction. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain. find
  7. Jean-Philippe Corbeil, Asma Ben Abacha, George Michalopoulos, Phillip Swazinna, Miguel Del-Agua, Jerome Tremblay, Akila Jeeson Daniel, Cari Bader, Kevin Cho, Pooja Krishnan, Nathan Bodenstab, Thomas Lin, Wenxuan Teng, Francois Beaulieu, and Paul Vozila. 2025b. Empowering healthcare practitioners wit… find
  8. A H M Rezaul Karim and Özlem Uzune. 2026. Masonnlp at mediqa-synur 2026: Retrieval-augmented large language models for schema-constrained clinical information extraction. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain. find
  9. Chen Ling, Xujiang Zhao, Jiaying Lu, Chengyuan Deng, Can Zheng, Junxiang Wang, Tanmoy Chowdhury, Yun Li, Hejie Cui, Xuchao Zhang, et al. 2023. Domain specialization as the key to make large language models disruptive: A comprehensive survey. arXiv preprint arXiv:2305.18703. find
  10. Amin Dada, Osman Alperen Koraş, Marie Bauer, Jean-Philippe Corbeil, Amanda Butler Contreras, Constantin Marc Seibold, Kaleb E Smith, Jens Kleesiek, et al. 2025. Does biomedical training lead to better medical performance? In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (G… find
  11. Chen Ling, Xujiang Zhao, Xuchao Zhang, Yanchi Liu, Wei Cheng, Haoyu Wang, Zhengzhang Chen, Mika Oishi, Takao Osaki, Katsushi Matsuda, et al. Improving open information extraction with large language models: A study on demonstration uncertainty. In ICLR 2024 Workshop on Reliable and Responsible Found… find
  12. John Dagdelen, Alexander Dunn, Sanghoon Lee, Nicholas Walker, Andrew S. Rosen, Gerbrand Ceder, Kristin A. Persson, and Anubhav Jain. 2024. Structured information extraction from scientific text with large language models. Nature Communications, 15(1):1418.
  13. Pasumarthy Deepak, Sushvin Marimuthu, and Parameswari Krishnamurthy. 2026. Ltrc-medicom at mediqa-synur 2026: Schema-guided clinical information extraction with hybrid clustering-sft-verification. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain. find
  14. Shazia Mitha, Jessica Schwartz, Mollie Hoben-sack, Kenrick Cato, Kyungmi Woo, Arlene Smaldone, and Maxim Topaz. 2023. Natural language processing of nursing notes: an integrative review. CIN: Computers, Informatics, Nursing, 41(6):377–384. find
  15. Bernal Jiménez Gutiérrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun, Prateek Munjal. 2026. Smart_solutions at mediqa-synur 2026: A multi-stage llm pipeline for nursing observation extraction. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spai…
  16. Midas_synur at mediqa-synur 2026: A prompting study for clinical observation extraction from nurse dictation transcriptions. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain.
  17. S. Panchal and P. Thakur. 2024. Harnessing the power of natural language processing in nursing services. International Journal of Advanced Research, 12(5):154–156. find
  18. Jerrin John Thomas and Parameswari Krishnamurthy. 2026. Anotherone at mediqa-synur 2026: Detect, extract, normalize - knowledge-grounded llm pipeline for clinical observation extraction. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain.
  19. Michael T. Saban, Arsalan Yaghoubi, Behnaz Eslami, Samie Tootooni, and Dmitriy Dligach. 2026. Lakefront ai ramblers at mediqa-synur 2026: Hybrid retrieval and llm verification for open-source schema-guided clinical information extraction. In Proceedings of the 8th Clinical Natural Language Processin…
  20. Aashwin Vaish and Dipti Misra Sharma. 2026. Ltrc-iiit at mediqa-synur 2026: Benchmarking a fully local, training-free rag pipeline. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain.
  21. Siva Satyanarayana, Raju Pusapati, and Ankit Singh. 2026. Gladiator at mediqa-synur 2026: Contextual clinical extraction: Integrating foundation models with domain-specific validation rules. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain.
  22. Airat A. Valiev. 2026. Hse nlp team at mediqa-synur 2026: Consensus adjudication ensemble (ace): Balancing precision and recall for schema-bystander clinical extraction. In Proceedings of the 8th Clinical Natural Language Processing Workshop, Mallorca Spain.
  23. Swetha Krishna Sriram and Akshitaa Sahoo. 2026. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/michalopoulos-2026-overview-mediqa-synur-2026-shared/