Elamur Research⌕
☰
home / papers / 2026 / AnotherOne at MEDIQA-SYNUR 2026: Detect, Extract, Normalize - Knowledge-Grounded LLM Pipeline for Clinical Observation Extraction

AnotherOne at MEDIQA-SYNUR 2026: Detect, Extract, Normalize - Knowledge-Grounded LLM Pipeline for Clinical Observation Extraction

Paper · 2026 · Jerrin John Thomas and Parameswari Krishnamurthy

Summary

The paper addresses the MEDIQA-SYNUR 2026 shared task: turning noisy, spoken-style nurse dictation transcripts into structured flowsheet observations. Each observation must be mapped to one of 193 ontology concepts with a correctly normalized value, and scoring requires exact matches. The authors built a modular inference system on gpt-oss-120b with JSON-constrained outputs and explicit reasoning fields. Before inference, an offline step uses the LLM to write a clinical definition and a few extraction rules for every concept. The concepts are also grouped into overlapping medical-domain clusters, reflecting the tendency of related findings to appear together in an assessment. The pipeline then runs in four stages: cluster-wise concept detection with evidence quotes, grounded value extraction, schema-constrained normalization, and rule-based cleanup using fuzzy matching and unit pairing. Definitions and prompts were refined repeatedly based on errors in the training data. This raised test F1 from 0.747 to 0.806, mainly through better precision. The work shows that knowledge-grounded decomposition can cut nursing documentation effort with limited training data. Free-text (STRING) values remain the main weakness.

Key points

  • Data: synthetic English SYNUR transcripts (train 122, dev 101, test 199), with roughly 14 gold observations per transcript. The ontology covers SINGLE_SELECT, MULTI_SELECT, NUMERIC (with separate unit concepts), and STRING types.
  • Domain priors: the 193 concepts are split into 19 non-exclusive clusters ranging from 4 to 23 concepts. These priors, along with the LLM-generated definitions and rules, are injected into every downstream prompt.
  • Results: the initial system scored P 0.645 / R 0.886 / F1 0.747 on test. After refinement it reached P 0.755 / R 0.865 / F1 0.806, and dev F1 rose from 0.736 to 0.798.
  • Error analysis guided targeted fixes that sharply reduced false positives:
    • Broset violence checklist errors fell from 102 to 0.
    • Pulse oximetry vs. Oxygen saturation confusion fell from 67 to 4.
    • Patient identification errors fell from 86 to 8.
  • STRING concepts suffer under exact-match scoring because generated text is often lexically different but semantically correct. Proposed remedies include span selection, confidence filtering, fine-tuning, and model ensembles.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓International Institute of Information Technology, HyderabadHyderabad · India38 works · 1,821 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

Cite

Cite as Jerrin John Thomas and Parameswari Krishnamurthy. 2026. AnotherOne at MEDIQA-SYNUR 2026: Detect, Extract, Normalize - Knowledge-Grounded LLM Pipeline for Clinical Observation Extraction. Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5hdskbdu4usi

@inproceedings{thomas2026anotherone,
  title = {{AnotherOne at MEDIQA-SYNUR 2026: Detect, Extract, Normalize - Knowledge-Grounded LLM Pipeline for Clinical Observation Extraction}},
  author = {Jerrin John Thomas and Parameswari Krishnamurthy},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/5hdskbdu4usi},
  url = {https://doi.org/10.63317/5hdskbdu4usi}
}

References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asma Ben Abacha, Jean-Philippe Corbeil, et al. 2025. Overview of the MEDIQA-OE 2025 shared task on medical order extraction from doctor-patient consultations. In Proceedings of the 7th Clinical Natural Language Processing Workshop. Association for Computational Linguistics. find
  2. Asma Ben Abacha, Jean-Philippe Corbeil, et al. 2026. Overview of the MEDIQA-SYNUR 2026 shared task on clinical observation extraction from nurse dictations. In Proceedings of the 8th Clinical Natural Language Processing Workshop. Forthcoming. find
  3. Asma Ben Abacha, Chaitanya Shivade, and Dina Demner-Fushman. 2019. Overview of the MEDIQA 2019 shared task on textual inference, question entailment and question answering. In Proceedings of the BioNLP Workshop at ACL 2019. Association for Computational Linguistics.
  4. Asma Ben Abacha, Wen wai Yim, Yadan Fan, and Thomas Lin. 2023. Overview of the MEDIQA-chat 2023 shared tasks on the summarization & generation of doctor-patient conversations. In Proceedings of the 5th Clinical Natural Language Processing Workshop. Association for Computational Linguistics. find
  5. Asma Ben Abacha, Wen wai Yim, Meliha Yetisgen, and Fei Xia. 2024. Overview of the MEDIQA-CORR 2024 shared task on medical error detection and correction. In Proceedings of the 6th Clinical Natural Language Processing Workshop. Association for Computational Linguistics. find
  6. Jean-Philippe Corbeil, Asma Ben Abacha, George Michalopoulos, Phillip Swazinna, Miguel Del-Agua, Jerome Tremblay, Akila Jeeson Daniel, Cari Bader, Kevin Cho, Pooja Krishnan, Nathan Bodenstab, Thomas Lin, Wenxuan Teng, Francois Beaulieu, and Paul Vozila. 2025. Empowering healthcare practitioners with… find
  7. Henk Harkema, John N. Dowling, Tyler Thornblade, and Wendy W. Chapman. 2009. ConText: An algorithm for determining negation, experiencer, and temporal status from clinical reports. Journal of Biomedical Informatics, 42(5):839–851. find
  8. Yan Hu, Iqra Ameer, Xu Zuo, Xueqing Peng, Yujia Zhou, Zehan Li, Yiming Li, Jianfu Li, Xiaoqian Jiang, and Hua Xu. 2024a. Improving large language models for clinical named entity recognition via prompt engineering. Journal of the American Medical Informatics Association, 31(9):1949– 1961. find
  9. Yan Hu, Qingyu Chen, Jingcheng Du, Xueqing Peng, Vipina Kuttichi Keloth, Qianqian Xie, Yujia Zhou, Zhiyong Lu, Hua Xu, and Qiao Jin. 2024b. On-the-fly definition augmentation of LLMs for biomedical NER. arXiv preprint arXiv:2404.00152. find
  10. Juan C. Quiroz, Liliana Laranjo, Ahmet Baki Kocaballi, Shlomo Berkovsky, Dana Rezazadegan, and Enrico Coiera. 2019. Challenges of developing a digital scribe to reduce clinical documentation burden. npj Digital Medicine, 2(1):114. find
  11. Aaron A. Tierney, Gavin Gayre, Brian Hoberman, Bruce Mattern, Robert Muller, Marie Schmid, et al. 2024. Ambient artificial intelligence scribes to alleviate the burden of clinical documentation. NEJM Catalyst. find
  12. Özlem Uzuner, Brett R. South, Shuying Shen, and Scott L. DuVall. 2011. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556. find
  13. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/thomas-2026-anotherone-mediqa-synur-2026-detect/