Elamur Research⌕
☰
home / papers / 2026 / NoviceTrio in #SMM4H-HeaRD 2026: Hybrid Clinical Transformer Ensembles for Insomnia Detection and Evidence Extraction from Clinical Notes

NoviceTrio in #SMM4H-HeaRD 2026: Hybrid Clinical Transformer Ensembles for Insomnia Detection and Evidence Extraction from Clinical Notes

Paper · 2026 · Abir Naskar and Mike Conway

Summary

The paper describes the NoviceTrio team's submissions to #SMM4H-HeaRD 2026 Task 2, which involves finding insomnia in MIMIC-III discharge notes. The data comprises 156 training and 23 validation notes. Subtask 1 is note-level yes/no classification. Subtask 2 predicts four criteria (Definition 1, Definition 2, Rule B, Rule C) and extracts character-offset evidence for each.

System 1 combines two independently fine-tuned models, Qwen3-4B-Instruct and Bio_ClinicalBERT, using a weighted average. Long notes are split into overlapping 512-token windows, and the note score is the maximum over its chunks, a design that deliberately favours recall.

System 2 is a single Bio_ClinicalBERT encoder with two heads: one classifies the note, the other assigns BIO tags to tokens to mark evidence spans. Before encoding, sentences are filtered by embedding similarity to an insomnia prototype. Three seed-varied copies vote on the final spans.

System 1 has very high recall but low precision. System 2 beats the shared-task mean and median on every Subtask 2 metric. Both systems show workable approaches for screening long clinical texts, and the code is public on GitHub.

Key points

  • Subtask 1 (System 1):
    • Recall is 0.9474, well above the mean of 0.6935 and the median of 0.6842.
    • F1 is 0.6923, slightly above the mean (0.6805) but below the median (0.7037).
    • Precision is only 0.5455.
  • Subtask 2 (System 2) vs. mean/median:
    • Label F1: 0.6444 vs. 0.5888/0.6000
    • Exact-match span F1: 0.4472 vs. 0.3129/0.3586
    • Partial-match span F1: 0.5093 vs. 0.4584/0.4524
  • System 1 ensemble details:
    • The ensemble weight α is chosen from 0.3–0.7 on the validation set.
    • The minority class is handled with inverse-frequency loss weights.
    • Both models are retrained on train+validation data before final test prediction.
  • System 2 pipeline:
    • Sentences are split with en_core_sci_sm and embedded with all-MiniLM-L6-v2.
    • When a note exceeds 90 sentences, the 30 most relevant ones are kept, each with ±1 neighbouring sentence.
    • Tagging uses 9 BIO classes, and chunks are 500 tokens with a 100-token overlap.
    • A span survives only if at least 2 of the 3 seed models predict it.
  • Stated limitations:
    • Only one train/validation split was used, with no cross-validation.
    • The sentence-filtering heuristics are fixed by hand.
    • No ablation study was run for System 1.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of MelbourneMelbourne · Australia102 works · 22,618 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksSocial Psychology · 330 works

Cite

Cite as Abir Naskar and Mike Conway. 2026. NoviceTrio in #SMM4H-HeaRD 2026: Hybrid Clinical Transformer Ensembles for Insomnia Detection and Evidence Extraction from Clinical Notes. Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.smm4h-1.51

@inproceedings{naskar2026novicetrio,
  title = {{NoviceTrio in \#SMM4H-HeaRD 2026: Hybrid Clinical Transformer Ensembles for Insomnia Detection and Evidence Extraction from Clinical Notes}},
  author = {Abir Naskar and Mike Conway},
  booktitle = {Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
  year = {2026},
  doi = {10.18653/v1/2026.smm4h-1.51},
  url = {https://doi.org/10.18653/v1/2026.smm4h-1.51}
}

References

5 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. Guillermo Lopez-Garcia, Jose Miguel Acitores Cortina, Jacob Berkowitz, Joey Chan, Ganesh Chandrasekar, Sumon Kanti Dey, Ivan Flores Amaro, Fernando Gallego, Lauren Gryboski, Ari Z Klein, Martin Krallinger, Salvador Lima-López, Tomohiro Nishiyama, Lisa Raithel, Ahmad Rezaie Mianroodi, Roland Roller,… find
  3. Lance Ramshaw and Mitch Marcus. 1995. Text chunking using transformation-based learning. In Third Workshop on Very Large Corpora.
  4. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  5. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/naskar-2026-novicetrio-smm4h-heard-2026-hybrid/