Elamur Research⌕
☰
home / papers / 2026 / HSE NLP TEAM at MEDIQA-SYNUR 2026: Consensus Adjudication Ensemble (ACE): Balancing Precision and Recall for Schema-Bystander Clinical Extraction

HSE NLP TEAM at MEDIQA-SYNUR 2026: Consensus Adjudication Ensemble (ACE): Balancing Precision and Recall for Schema-Bystander Clinical Extraction

Paper · 2026 · Airat A. Valiev

Summary

The paper presents an HSE University system for the MEDIQA-SYNUR 2026 shared task. The task is to turn synthetic nurse dictation transcripts, derived from ICU notes, into structured flowsheet entries. Each entry pairs one of 193 ontology concepts with a value of type SINGLE_SELECT, MULTI_SELECT, NUMERIC or STRING, and scoring uses strict exact-match micro-F1.

The proposed Consensus Adjudication Ensemble (ACE) separates the pursuit of recall from the enforcement of precision in three stages:

  1. Two complementary LLM extractors produce candidates, and their pooled outputs are normalized.
  2. An adjudicator acts as a verifier. It keeps only candidates with explicit support in the transcript and treats the schema as a "bystander" rather than as evidence.
  3. A cheap repair step fixes only the outputs that violate schema constraints.

ACE ranked 4th on the official test set. The author also analyses clinically dangerous errors, such as hallucinated interventions and over-confident labels, and shows that retrieval can cause smaller models to copy schema values into their outputs. The work matters because it offers a safety-oriented and token-efficient design for clinical extraction.

Key points

  • Test results: micro-F1 0.7996 (P=0.7812, R=0.8188), 4th place on the leaderboard. The dataset splits contain 122 train, 101 dev and 199 test transcripts.
  • Generators: GPT-5 runs zero-shot, and Qwen3-235B-A22B-Instruct-FP8 runs with retrieval augmentation via the Doubleword batch API. GPT-4o served as a baseline.
  • Adjudication removed 2,219 candidates. Of these, 91.3% were genuine false positives and 8.7% were correct predictions lost.
  • Targeted repair cuts validation context from about 230k tokens to under 2k per document. It handles three kinds of fixes:
    • boolean canonicalization (True→Yes);
    • enum projection;
    • scalar-to-list conversion for MULTI_SELECT concepts.
  • Retrieval augmentation was limited to the large model because of the "contextual copying" effect, in which smaller models copy enumerated values or schema snippets that the transcript does not support.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National Research University Higher School of EconomicsMoscow · Russia23 works · 39,585 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksNursing · 5 worksIssues, ethics and legal aspects · 4 works

Cite

Cite as Airat A. Valiev. 2026. HSE NLP TEAM at MEDIQA-SYNUR 2026: Consensus Adjudication Ensemble (ACE): Balancing Precision and Recall for Schema-Bystander Clinical Extraction. Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4vsjfsrfx246

@inproceedings{valiev2026hse,
  title = {{HSE NLP TEAM at MEDIQA-SYNUR 2026: Consensus Adjudication Ensemble (ACE): Balancing Precision and Recall for Schema-Bystander Clinical Extraction}},
  author = {Airat A. Valiev},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4vsjfsrfx246},
  url = {https://doi.org/10.63317/4vsjfsrfx246}
}

References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Monica Agrawal, Stefan Hegselmann, Hunter Hunter, and David Sontag. 2022. Large language models are few-shot clinical information extractors. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1997–2022.
  2. Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. 2019. Publicly available clinical bert embeddings. In Proceedings of the 1 https://anonymous.4open.science/r/ MEDIQA-SYNUR-HSE_NLP-8C2B/ 2nd Clinical Natural Language Processing Workshop, pages… ↗
  3. Luca Beurer-Kellner, Marc Fischer, and Martin Vechev. 2023. Prompting is programming: A query language for large language models. In Proceedings of the 44th ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 1826–1850. find
  4. Jean-Philippe Corbeil, Asma Ben Abacha, George Michalopoulos, Phillip Swazinna, Miguel Del-Agua, Jerome Tremblay, Akila Jeeson Daniel, Cari Bader, Kevin Cho, Pooja Krishnan, Nathan Bodenstab, Thomas Lin, Wenxuan Teng, Francois Beaulieu, and Paul Vozila. 2025. Empowering healthcare practitioners with… find
  5. Junyou Li, Qi Zhang, and Philip S Yu. 2024. More agents is all you need. arXiv preprint arXiv:2402.05120. find
  6. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  7. Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896. find
  8. George Michalopoulos, Jean-Philippe Corbeil, Cari Bader, Nate Bodenstab, and Asma Ben Abacha. 2026. Overview of the mediqa-synur 2026 shared task on observation extraction from nurse dictations. In Proceedings of the 8th Clinical Natural Language Processing Workshop, ClinicalNLP@LREC 2026, Palma, Ma…
  9. Shazia Mitha, Jessica Schwartz, Mollie Hoben-sack, Kenrick Cato, Kyungmi Woo, Arlene Smaldone, and Maxim Topaz. 2023. Natural language processing of nursing notes: an integrative review. CIN: Computers, Informatics, Nursing, 41(6):377–384. find
  10. AJ Moy, J Adler-Milstein, et al. 2023. Measurement of clinical documentation burden among physicians and nurses using electronic health record audit logs. Journal of the American Medical Informatics Association, 30(4):643–652. find
  11. OpenAI. 2024. Gpt-4o system card.
  12. S Panchal and P Thakur. 2024. Harnessing the power of natural language processing in nursing services. International Journal of Advanced Research, 12(5):154–156. find
  13. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR.
  14. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksandra Spyra, Alex Baker-Whitcomb, Alex Beutel, Alex Karpenko, Alex Makelov, Alex N… — OpenAI GPT-5 System Card
  15. Qwen Team. 2025. Qwen3 technical report. find
  16. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In ICLR 2023.
  17. DRAFT B (Model: {expert_b_name}, Est. F1: {expert_b_f1}): {draft_b_json} Brandon T Willard and Rémi Louf. 2023. Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702. find
  18. YOUR TASK: 1. Synthesize a single high-fidelity extraction. find
  19. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems, volume 36.
  20. A. 3. If an observation appears in only one draft, keep it only when explicitly supported by the transcript. 4. Ensure all IDs map to valid schema IDs. Prompts To ensure reproducibility, we summarize the prompt templates used in each ACE stage. Prompts follow a strict role–task–constraint–output str… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/valiev-2026-hse-nlp-team-mediqa-synur/