Elamur Research⌕
☰
home / papers / 2026 / DocUA at CRF Filling 2026: LLM StructCore –- Schema-Guided Reasoning Condensation and Deterministic Compilation

DocUA at CRF Filling 2026: LLM StructCore –- Schema-Guided Reasoning Condensation and Deterministic Compilation

Paper · 2026 · Serhii Zabolotnii

Summary

This system description addresses the CL4Health 2026 shared task of filling a 134-item Dyspnea Case Report Form (CRF) from English and Italian clinical notes. Most fields in a given note are empty, and the scorer penalizes both missed values and invented ones. Instead of asking one LLM to output all 134 fields at once, the author splits the work into two stages, based on Schema-Guided Reasoning (SGR). In Stage 1, an LLM writes a compact JSON summary organized under nine fixed clinical domains. In Stage 2, a rule-based compiler with no LLM calls turns that summary into the official format. It parses key-value lines, standardizes item names (with an optional UMLS alias map), removes poorly supported positives through evidence checks, and maps values onto the controlled vocabulary. On the dev80 split, a Mistral Large 3 front end reached macro-F1 of 0.6543 (EN) and 0.6905 (IT). The submitted English run scored 0.63 on the hidden test set. The design guarantees valid output, works across inference backends, and needs no language-specific engineering.

Key points

  • Data: the official NLP-FBK/dyspnea-crf-* splits (10 train, 80 dev, 200 hidden test), plus 2,667 unannotated notes (NLP-FBK/dyspnea-clinical-notes) used for teacher generation. The median dev document has only about 12 non-unknown items out of 134.
  • Stage 2 compiler: applies 70+ abbreviation rules and a UMLS alias map covering 134/134 items. It uses regex evidence gates for high-risk items such as active neoplasia, arrhythmia and acute coronary syndrome, and normalizes values into 13 vocabulary categories.
  • Format stability: JSON parsing succeeded on every input without grammar-constrained decoding, even with a quantized 4B model (MedGemma) served through llama.cpp.
  • Teacher comparison: Mistral Large 3 and Qwen 3.5 found the same number of true positives on EN dev (236). Qwen produced more false positives (211 vs. 170).
  • Test result: 0.63 F1 on Codabench, within 0.05 of the top team, Aurum (0.68). The code is released as the llm_structcore package.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Cherkasy State Business College.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksBiochemistry, Genetics and Molecular Biology · 263 worksMolecular Biology · 260 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksReasoning · 1,685 works

Cite

Cite as Serhii Zabolotnii. 2026. DocUA at CRF Filling 2026: LLM StructCore –- Schema-Guided Reasoning Condensation and Deterministic Compilation. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4ar36j4w3vrk

@inproceedings{zabolotnii2026docua,
  title = {{DocUA at CRF Filling 2026: LLM StructCore –- Schema-Guided Reasoning Condensation and Deterministic Compilation}},
  author = {Serhii Zabolotnii},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4ar36j4w3vrk},
  url = {https://doi.org/10.63317/4ar36j4w3vrk}
}

References

19 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rinat Abdullin. 2026. Schema-guided reasoning (SGR): A complete guide. https://abdullin.com/schema-guided-reasoning/. ↗
  2. Alan R. Aronson. 2001. Effective mapping of biomedical text to the UMLS Metathesaurus: The MetaMap program. In Proceedings of the AMIA Symposium, pages 17–21. find
  3. Yixin Dong, Charlie F. Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, and Tianqi Chen. 2024. XGrammar: Flexible and efficient structured generation engine for large language models. Proceedings of Machine Learning and Systems, 7. find
  4. Pietro Ferrazzi, Mattia Franzin, Alberto Lavelli, and Bernardo Magnini. 2026a. Small LLMs for medical NLP: a systematic analysis of few-shot, constraint decoding, fine-tuning and continual pre-training in Italian. arXiv preprint arXiv:2602.17475. find
  5. Pietro Ferrazzi, Soumitra Ghosh, Alberto Lavelli, and Bernardo Magnini. 2026b. Overview of the CRF 2026 shared task on clinical case report forms filling. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health), Palma, Mallorca (Spain). ELRA. find
  6. Pietro Ferrazzi, Alberto Lavelli, and Bernardo Magnini. 2025. Converting annotated clinical cases into structured case report forms. In Proceedings of the 24th Workshop on Biomedical Language Processing, pages 307–318, Viena, Austria. Association for Computational Linguistics. find
  7. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Mober, et al. 2023. DSPy: Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714.
  8. Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. 2019. ScispaCy: Fast and robust models for biomedical natural language processing. In Proceedings of the 18th BioNLP Workshop and Shared Task.
  9. Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, et al. 2024. Capabilities of Gemini models in medicine. arXiv preprint arXiv:2404.18416.
  10. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172–180.
  11. Luca Soldaini and Nazli Goharian. 2016. Quick-UMLS: A fast, unsupervised approach for medical concept extraction. In MedIR Workshop at SIGIR. find
  12. Hanna Suominen, Sanna Salanterä, Sumithra Velupillai, Wendy W. Chapman, Guergana Savova, Noemie Elhadad, Sameer Pradhan, Brett R. South, Danielle L. Mowery, Gareth J.F. Jones, Johannes Leveling, Liadh Kelly, Lorraine Goeuriot, David Martinez, and Guido Zuccon. 2013. Overview of the ShARe/CLEF eHealt… find
  13. Özlem Uzuner, Brett R. South, Shuying Shen, and Scott L. DuVall. 2011. 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. volume 18, pages 552–556. find
  14. Brandon T. Willard and Rémi Louf. 2023. Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702. find
  15. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Jeff Huang, Chuyue Sun, Cody Hao Yu, Shiyi Cao, Christos Kober, Joseph E. Gonzalez, Clark Barrett, Ying Sheng, and Ion Stoica. 2024. SGLang: Efficient execution of structured language model programs. arXiv preprint arXiv:2312.07104. 12.
  16. Language Resource References Anthony McEnery and others. 2004. The EMILLE/CIIL Corpus. EMILLE (Enabling Minority Language Engineering) Project. distributed via ELRA: ELRA-Id W0037, ISLRN 039-846-040-604-0. find
  17. Khalid Choukri and Niklas Paullson. 2004. The OrienTel Moroccan MCA (Modern Colloquial Arabic) database. distributed via ELRA: ELRA-Id ELRA-S0183, ISLRN 613-578-868-832-2. find
  18. Roventini, Adriana and Marinelli, Rita and Bertagna, Francesca. 2016. ItalWordNet v.2. ILC-CNR for CLARIN-IT repository hosted at Institute for Computational Linguistics “A. Zampolli”, National Research Council, in Pisa, ISLRN 532-206-426-067-2. PID http://hdl.handle.net/20.500.11752/ILC-62. Note: Y… ↗
  19. Speecon Consortium. 2011. Catalan Speecon database. SpeeCon. Speecon Project, distributed via ELRA: ELRA-Id S0327, Speecon resources, 1.0, ISLRN 935-211-147-357-5. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zabolotnii-2026-docua-crf-filling-2026-llm/