Elamur Research⌕
☰
home / papers / 2026 / blue at SMM4H-HeaRD 2026: Class-Weighted Transformer Ensembles with Structured Decoding and Chain-of-Thought Blending across Six Health NLP Shared Tasks

blue at SMM4H-HeaRD 2026: Class-Weighted Transformer Ensembles with Structured Decoding and Chain-of-Thought Blending across Six Health NLP Shared Tasks

Paper · 2026 · Krish Sharma, Rhea Singhal and Jatin Bedi

Summary

Team blue entered six SMM4H-HeaRD 2026 shared tasks on health text from social media, PubMed, pathology reports and clinical documents. The tasks were adverse drug event detection in several languages, flu vaccine effectiveness from tweets, SARS-CoV-2 patient metadata, TNM cancer staging, opioid impact spans in Reddit posts, and multilingual clinical NER with annotation projection. They cover binary, multi-class, multi-label and BIO sequence-labelling setups. Instead of building six unrelated systems, the authors used one toolkit and adjusted it per task. It has three parts: inverse-frequency class weighting against strong label imbalance, ensembles over seeds or backbones to reduce variance, and post-hoc calibration of decision thresholds. Task-specific additions include:

  • Viterbi-constrained BIO decoding,
  • regex overrides for explicit pTNM mentions,
  • blending with chain-of-thought predictions from Qwen2.5-7B-Instruct.

The best results were near-perfect TNM staging and flu vaccination classification above the organisers' 70B CoT baseline. The paper argues that careful fine-tuning of encoders can match or beat LLM prompting. It also warns that thresholds tuned on small dev sets do not transfer reliably to test data.

Key points

  • Task 6 (TNM staging): Clinical-BigBird with a 1536-token context, three classification heads and a regex override reached micro-F1 0.990 and macro-F1 0.938. The M-stage was the weakest component (F1 0.828).
  • Task 3 (flu vaccine effectiveness): a 9-model ensemble (DeBERTa-v3-large, BERTweet-large and RoBERTa-large, each with 3 seeds) used posting dates prepended to the input and Qwen CoT outputs blended at weight 0.2.
    • Vaccination: µF1 0.872, above the baseline of 0.849.
    • Test result: µF1 0.918, below the baseline of 0.950.
  • Task 5 (patient metadata): PubMedBERT with 5-fold CV and an out-of-fold threshold of 0.68 scored F1 0.764. This is far above Llama-3-70B prompting (0.558) and close to the BiomedBERT-Large benchmark (0.776).
  • Task 1 (ADE detection): XLM-RoBERTa-large plus ruRoberta-large and cl-tohoku BERT specialists reached pooled F1 0.580 (baseline 0.525). Per-language thresholds overfit, with Russian dropping 17.0 points and Mandarin 13.1 points from dev to test.
  • Task 7 (opioid impact spans): a ten-model Viterbi-decoded ensemble achieved relaxed F1 0.59 and strict F1 0.50, above the participant mean.
  • Task 8 (multilingual clinical NER): XLM-RoBERTa-base trained jointly on 7 languages reached strict F1 of roughly 0.2–0.41.
  • Cross-task finding: class weighting gave the largest single gains, for example +5.5 F1 on ADE detection and +5.2 AUROC on the M-stage.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Thapar Institute of Engineering & TechnologyPatiāla · India6 works · 11 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksBiochemistry, Genetics and Molecular Biology · 263 worksMolecular Biology · 260 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Krish Sharma, Rhea Singhal and Jatin Bedi. 2026. blue at SMM4H-HeaRD 2026: Class-Weighted Transformer Ensembles with Structured Decoding and Chain-of-Thought Blending across Six Health NLP Shared Tasks. Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.smm4h-1.13

@inproceedings{sharma2026blue,
  title = {{blue at SMM4H-HeaRD 2026: Class-Weighted Transformer Ensembles with Structured Decoding and Chain-of-Thought Blending across Six Health NLP Shared Tasks}},
  author = {Krish Sharma, Rhea Singhal and Jatin Bedi},
  booktitle = {Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
  year = {2026},
  doi = {10.18653/v1/2026.smm4h-1.13},
  url = {https://doi.org/10.18653/v1/2026.smm4h-1.13}
}

References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jun Suzuki, Kentaro Sakuma, and Kentaro Inui. 2020. Pre-training of deep bidirectional transformers for Japanese. In Proc. Annual Conference of JSAI. find
  2. Alexis Conneau, Kartikay Khandelwal, and 1 others. 2020. Unsupervised cross-lingual representation learning at scale. In Proc. ACL, pages 8440–8451.
  3. Dongfang Xu, Guillermo Lopez García, Karen O’Connor, and 1 others. 2025. Mining social media data for influenza vaccine effectiveness using a large language model and chain-of-thought prompting. medRxiv. find
  4. Sumon Kanti Dey, Jeanne M Powell, and 1 others. 2025. Inference gap in domain expertise and machine intelligence in named entity recognition. In Biocomputing 2026: Proc. Pacific Symposium, pages 12–26. find
  5. Manzil Zaheer, Guru Guruganesh, and 1 others. 2020. Big bird: Transformers for longer sequences. In NeurIPS, volume 33, pages 17283–17297.
  6. Yu Gu, Robert Tinn, Hao Cheng, and 1 others. 2022. Domain-specific language model pretraining for biomedical NLP. ACM Trans. Computing for Healthcare, 3(1):1–23. find
  7. Daniil Zmitrovich, Alexander Abramov, and 1 others. 2023. RuRoBERTa: A pre-trained language model for Russian. arXiv:2309.10931. — [arXiv:2309.10931](https://arxiv.org/abs/2309.10931) find
  8. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In Proc. ICLR.
  9. Sarvnaz Karimi, Alejandro Metke-Jimenez, Madonna Kemp, and Chen Wang. 2015. CADEC: A corpus of adverse drug event annotations. J. Biomedical Informatics, 55:73–81. find
  10. Jenna Kefeli, Jacob Berkowitz, and 1 others. 2024. Generalizable and automated classification of TNM stage from pathology reports with external validation. Nature Communications, 15(1):8916. find
  11. Woosuk Kwon, Zhuohan Li, and 1 others. 2023. Efficient memory management for large language model serving with PagedAttention. In Proc. SOSP, pages 611–626.
  12. Yikuan Li, Ramsey M Wehbe, and 1 others. 2023. A comparative study of pretrained language models for long clinical text. JAMIA, 30(2):340–347. find
  13. Yinhan Liu, Myle Ott, Naman Goyal, and 1 others. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692.
  14. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proc. ICLR. find
  15. Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. 2019. When does label smoothing help? In NeurIPS, volume 32. find
  16. Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English tweets. In Proc. EMNLP: System Demonstrations, pages 9–14.
  17. NLLB Team, Marta R Costa-jussà, and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv:2207.04672. find
  18. Abeed Sarker and Graciela Gonzalez. 2015. Portable automatic text classification for adverse drug reaction detection via multi-corpus training. J. Biomedical Informatics, 53:196–207. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sharma-2026-blue-smm4h-heard-2026-class/