Elamur Research⌕
☰
home / papers / 2026 / Gladiator at MEDIQA-SYNUR 2026: Contextual Clinical Extraction: Integrating Foundation Models with Domain-Specific Validation Rules

Gladiator at MEDIQA-SYNUR 2026: Contextual Clinical Extraction: Integrating Foundation Models with Domain-Specific Validation Rules

Paper · 2026 · Siva Satyanarayana Raju Pusapati and Ankit Singh

Summary

The paper addresses the MEDIQA-SYNUR 2026 shared task. The goal is to convert synthetic nurse dictation transcripts into structured EHR observations that fit a 193-concept clinical schema. The schema uses four value types: NUMERIC, STRING, SINGLE_SELECT and MULTI_SELECT. A prediction counts as correct only if both the concept ID and the value match exactly, so raw LLM output is unreliable on its own.

The authors build a hybrid pipeline:

  • LLM extraction: Claude Opus 4 receives the full schema, three training examples and targeted guidance for difficult fields.
  • Post-processing: six filters (for example, dropping speculative diagnoses and physiologically implausible numbers) and five normalisation steps clean the output.
  • Rule-based merging: the cleaned output is combined with matches from over 400 context-checked regex rules and then validated against the schema.

On the development set, the system reaches 78.98% F1, with recall (84.32%) clearly higher than precision (74.27%). Most errors are spurious secondary diagnoses and confusion between near-identical cognitive status options. The work shows how rule-based guardrails can make LLM extraction more dependable for clinical documentation. The authors also note concerns about cost, privacy and reliance on synthetic data.

Key points

  • Data: 122 training and 101 development samples of synthetic nursing dictations, plus a held-out test set. Gold labels were created by expert nurses using an open clinical ontology.
  • Model setup: Claude Opus 4 (ID listed as claude-opus-4-5-20251101), temperature 0.7, max_tokens 4096. Processing the dev set cost roughly $8–10.
  • Dev results: P 74.27%, R 84.32%, F1 78.98%. All reported results come from the dev set, which was also used to tune the system.
  • Error profile: 57% of errors are false positives. These include 32 over-extracted secondary diagnoses (e.g., past conditions such as "history of diabetes"), 12 cognitive status value errors and 10 dyspnea value errors.
  • Stated limitations: no ablation studies, only one LLM tested, synthetic data only, and a proprietary model that hinders reproducibility and deployment in healthcare.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksNursing · 5 worksIssues, ethics and legal aspects · 4 works

Cite

Cite as Siva Satyanarayana Raju Pusapati and Ankit Singh. 2026. Gladiator at MEDIQA-SYNUR 2026: Contextual Clinical Extraction: Integrating Foundation Models with Domain-Specific Validation Rules. Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2tkvpwgzzcn2

@inproceedings{pusapati2026gladiator,
  title = {{Gladiator at MEDIQA-SYNUR 2026: Contextual Clinical Extraction: Integrating Foundation Models with Domain-Specific Validation Rules}},
  author = {Siva Satyanarayana Raju Pusapati and Ankit Singh},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2tkvpwgzzcn2},
  url = {https://doi.org/10.63317/2tkvpwgzzcn2}
}

References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., and Sontag, D. (2022). Large language models are few-shot clinical information extractors. In Proceedings of EMNLP 2022, pages 1998-2022.
  2. Alsentzer, E., Murphy, J., Boag, W., Weng, W.H., Jin, D., Naumann, T., and McDermott, M. (2019). Publicly available clinical BERT embeddings. In Proceedings of the Clinical NLP Workshop, pages 72-78. \#\# 9.
  3. Conclusion Aronson, A.R. and Lang, F.M. (2010). An overview of MetaMap: historical perspective and recent advances. Journal of the American Medical Informatics Association, 17(3):229-236. find
  4. Extraction System (cTAKES): architecture, component evaluation and applications. Journal of the American Medical Informatics Association, 17(5):507-513. find
  5. Baumann, L.A., Baker, J., and Elshaug, A.G. (2018). The impact of electronic health record systems on clinical documentation times: A systematic review. Health Affairs, 37(4):655-663. find
  6. Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620:172-180.
  7. Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, pages 5998-6008.
  8. Brown, T., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
  9. Wang, Y., Wang, L., Rastegar-Mojarad, M., et al. (2018). Clinical information extraction applications: A literature review. Journal of Biomedical Informatics, 77:34-49. find
  10. Chapman, W.W., Bridewell, W., Hanbury, P., Cooper, G.F., and Buchanan, B.G. (2001). A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 34(5):301-310. find
  11. Chen, Z., Chen, W., Smiley, C., Shah, S., Borber, I., and Chang, S. (2021). Schema-guided multi-domain dialogue state tracking with graph attention neural networks. In Proceedings of NAACL 2021, pages 1124-1134. find
  12. Gu, Y., Tinn, R., Cheng, H., et al. (2021). Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare, 3(1):1-23.
  13. Harkema, H., Dowling, J.N., Thornblade, T., and Chapman, W.W. (2009). ConText: An algorithm for determining negation, experiencer, and temporal status from clinical reports. Journal of Biomedical Informatics, 42(5):839-851. find
  14. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., and Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38.
  15. Joukes, E., de Keizer, N.F., de Bruijne, M.C., Abu-Hanna, A., and Cornet, R. (2018). Impact of electronic versus paper-based recording on the quality of clinical documentation in nursing. Studies in Health Technology and Informatics, 247:261-265. find
  16. Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C.H., and Kang, J. (2020). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234-1240.
  17. Michalopoulos, G., Corbeil, J.P., Bader, C., Bodenstab, N., and Ben Abacha, A. (2026). Overview of the MEDIQA-SYNUR 2026 Shared Task on Synthetic Nursing Observation Extraction. In Proceedings of ClinicalNLP @ LREC 2026. find
  18. Rastogi, A., Zang, X., Srivastava, S., Guez, R., and Dilek, Z. (2020). Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In Proceedings of AAAI 2020, pages 8689-8696.
  19. Savova, G.K., Masanz, J.J., Ogren, P.V., et al. (2010). Mayo clinical Text Analysis and Knowledge find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pusapati-2026-gladiator-mediqa-synur-2026-contextual/