Elamur Research⌕
☰
home / papers / 2026 / Gladiators at #SMM4H–HeaRD 2026: Multi-Seed XLM-RoBERTa Ensemble with Focal Loss and Per-Language Threshold Optimization for Multilingual Adverse Drug Event Detection

Gladiators at #SMM4H–HeaRD 2026: Multi-Seed XLM-RoBERTa Ensemble with Focal Loss and Per-Language Threshold Optimization for Multilingual Adverse Drug Event Detection

Paper · 2026 · Ankit Kumar Singh

Summary

This system paper from team Gladiators addresses Task 1 of SMM4H–HeaRD 2026: deciding whether a social media post in one of several languages mentions an adverse drug event. The training data cover six languages (en, ja, ru, zh, de, fr). It has 46,737 posts, only 6.4% of them positive, drawn from tweets, forum posts, reviews and health Q&A. The test set added Farsi, for which no training examples existed. The author fine-tuned three XLM-RoBERTa-large models that differ only in random seed. Focal loss and tripled positive examples countered the class imbalance. The models' probabilities were averaged, and a separate decision threshold was tuned for each language on dev data. Eleven configurations were compared, including zero-shot GLiNER and NLI baselines, NLLB-200 translation augmentation, a Japanese BERT and an mDeBERTa-v3 add-on. None of them beat the seed ensemble. The submission reached a pooled test F1 of 0.6039, above the competition mean and median, and scored 0.435 on zero-shot Farsi. The work offers a practical recipe for multilingual pharmacovigilance. It also documents failures, notably that augmenting one language can shift calibration for all others.

Key points

  • Setup: XLM-RoBERTa-large (560M) with focal loss (α=0.75, γ=2.0) and 3× positive oversampling. Seeds 42, 123 and 456 were trained on a single RTX 5070 Ti GPU. The tuned thresholds differed across languages, with English the lowest at 0.32 and Japanese the highest at 0.70.
  • Scores: the ensemble reached a dev F1 of 0.7505, up from 0.731 for a single seed. Official test F1 was 0.6039, against a competition mean of 0.5465 and median of 0.5798. Farsi made up 35.5% of the 42,736 test samples and scored 0.435, versus a mean of 0.367.
  • Negative results:
    • NLLB-200 translation of tweets into de/fr lowered F1 from 0.704 to 0.687, with fr down 8.3%.
    • GLiNER features added nothing.
    • Adding mDeBERTa-v3-base slightly diluted the ensemble.
    • bert-large-japanese-v2 only matched the multilingual model on ja (0.662).
  • Farsi augmentation pitfalls: translating only positives made the model label 99.9% of Farsi as ADE. A balanced version fixed this but moved the Japanese threshold from 0.70 to 0.27.
  • Post-deadline result: a weighted six-model mix of original and Farsi-augmented models raised dev F1 to 0.7585. The largest dev-to-test drop was in Russian (0.730 → 0.553), which the author attributes to distribution shift.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPharmacology, Toxicology and Pharmaceutics · 7 worksToxicology · 6 works

🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 worksInformation extraction · 167 works

Cite

Cite as Ankit Kumar Singh. 2026. Gladiators at #SMM4H–HeaRD 2026: Multi-Seed XLM-RoBERTa Ensemble with Focal Loss and Per-Language Threshold Optimization for Multilingual Adverse Drug Event Detection. Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.smm4h-1.2

@inproceedings{singh2026gladiators,
  title = {{Gladiators at \#SMM4H–HeaRD 2026: Multi-Seed XLM-RoBERTa Ensemble with Focal Loss and Per-Language Threshold Optimization for Multilingual Adverse Drug Event Detection}},
  author = {Ankit Kumar Singh},
  booktitle = {Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
  year = {2026},
  doi = {10.18653/v1/2026.smm4h-1.2},
  url = {https://doi.org/10.18653/v1/2026.smm4h-1.2}
}

References

5 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Key positive findings. (1) Model scale: XLM-R-large gained +15.7% (de), +9.5% (zh) over base for data-scarce languages. (2) Multi-seed ensembling: Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding.…
  2. Trung Huynh, Yulan He, Alistair Willis, and Stefan Rueger. 2016. Adverse drug reactions discovery from open data: A perspective on social media mining. ACM Computing Surveys, 49(4):1–35. find
  3. Sarvnaz Karimi, Alejandro Metke-Jimenez, Madonna Kemp, and Chen Wang. 2015. CADEC: A corpus of adverse drug event annotations. Journal of Biomedical Informatics, 55:73–81. find
  4. Moritz Laurer, Wouter van Atteveldt, Andreu Casas, and Kasper Welbers. 2024. Less annotating, more classifying: Addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI. Political Analysis, 32(1):84–100. find
  5. Loss LR Dev F1 Best? 1 2 3 4 5 0.0331 0.0145 0.0069 0.0030 0.0016 9.7e-6 7.5e-6 4.1e-6 1.2e-6 0.0 0.702 0.726 0.721 0.729 0.731 ✓ *Table 4: Training progression for XLM-R-large seed 42. Loss is training focal loss; LR is the cosine-decayed learning rate.* Tsung-Yi Lin, Priya Goyal, Ross Girshick, Ka…

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/singh-2026-gladiators-smm4h-heard-2026-multi/