Elamur Research⌕
☰
home / papers / 2026 / sebis at CRF Filling 2026: A Two-Stage Local LLM Pipeline for Medical CRF Filling

sebis at CRF Filling 2026: A Two-Stage Local LLM Pipeline for Medical CRF Filling

Paper · 2026 · Katharina Sommer, Tristan Till and Florian Matthes

Summary

The paper tackles automatic filling of Case Report Forms (CRFs) from free-text emergency-department notes on dyspnea patients, the setting of the CL4Health 2026 CRF Filling shared task (English track). The authors avoid commercial LLM APIs because of privacy, cost and hallucination concerns. Instead, they run everything on-premise with an open-weight medical model, MedGemma-27B, and use no fine-tuning and no external data. Their pipeline handles each note and CRF item in two steps. First, a gatekeeping prompt asks the model only whether the item is documented at all, answering y or n. Only if it is does a second prompt select a categorical option or pull out a measured value. Both prompts use item-specific few-shot examples built from the small training set. This design targets the heavy dominance of "unknown" labels, which makes false positives the main risk. The system reached a macro-F1 of 0.55 on the hidden test set, ranking second among local open-source English entries. The result suggests that privacy-preserving local pipelines can come reasonably close to proprietary frontier systems.

Key points

  • Model: instruction-tuned MedGemma-27B in GGUF Q8_K_XL quantization, run at temperature 0, fully local.
  • Data: the training set has only 10 notes. Most labels are "unknown" (10,321, 96.3%), followed by y/n (2.0%), categorical (1.2%) and measured (0.5%). The authors built 10-shot examples per item for each stage.
  • Ablations (internal dev split, macro-F1):
    • zero-shot baseline: 0.330
    • two-stage only: 0.423
    • few-shot only: 0.572
    • full pipeline: 0.604
    • The false-positive rate fell from about 92% to about 45%.
  • Leaderboard: 0.55 macro-F1 placed 17th of 32 overall and 11th of 22 in English. It ranked 2nd of 7 local open-source English systems, 0.02 behind the best one (0.57). The overall winner was a closed-source system at 0.68.
  • Reusability: the classification-then-extraction prompting scheme covers binary, categorical and "measured" items. It requires no training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technical University of MunichMunich · Germany55 works · 48,474 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

Cite

Cite as Katharina Sommer, Tristan Till and Florian Matthes. 2026. sebis at CRF Filling 2026: A Two-Stage Local LLM Pipeline for Medical CRF Filling. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2prmobstjfcg

@inproceedings{sommer2026sebis,
  title = {{sebis at CRF Filling 2026: A Two-Stage Local LLM Pipeline for Medical CRF Filling}},
  author = {Katharina Sommer, Tristan Till and Florian Matthes},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2prmobstjfcg},
  url = {https://doi.org/10.63317/2prmobstjfcg}
}

References

8 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. 2022. Large language models are few-shot clinical information extractors. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 1998–2022.
  2. Luc Builtjes, Joeran Bosma, Mathias Prokop, Bram van Ginneken, and Alessa Hering. 2025. Leveraging open-source large language models for clinical information extraction in resource-constrained settings. JAMIA open, 8(5):ooaf109. find
  3. Pietro Ferrazzi, Soumitra Ghosh, Alberto Lavelli, and Bernardo Magnini. 2026. Overview of the crf 2026 shared task on clinical case report forms filling. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health), Palma, Mallorca (Spain). ELRA. find
  4. Pietro Ferrazzi, Alberto Lavelli, and Bernardo Magnini. 2025. Converting annotated clinical cases into structured case report forms. In Proceedings of the 24th Workshop on Biomedical Language Processing, pages 307–318, Viena, Austria. Association for Computational Linguistics. find
  5. Gabriela Anna Kaczmarek, Pietro Ferrazzi, Lorenzo Porta, Vicky Rubini, and Bernardo Magnini. 2026. Toward automatic filling of case report forms: A case study on data from an italian emergency department. find
  6. Phillip Richter-Pechanski, Marvin Seiferling, Christina Kiriakou, Dominic M Schwab, Nicolas A Geis, Christoph Dieterich, and Anette Frank. 2025. Medication information extraction using local large language models. Journal of biomedical informatics, page 104898. find
  7. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. 2025. Medgemma technical report. arXiv preprint arXiv:2507.05201. find
  8. Isabella Catharina Wiest, Fabian Wolf, Marie-Elisabeth Leßmann, Marko van Treeck, Dyke Ferber, Jiefu Zhu, Heiko Boehme, Keno K Bressem, Hannes Ulrich, Matthias P Ebert, et al. 2024. Llm-aix: An open source pipeline for information extraction from unstructured medical text based on privacy preserving… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sommer-2026-sebis-crf-filling-2026-two/