Elamur Research⌕
☰
home / papers / 2026 / DNT at #SMM4H–HeaRD 2026: Leveraging BERT-based Encoders and LLMs for Medical Information Extraction

DNT at #SMM4H–HeaRD 2026: Leveraging BERT-based Encoders and LLMs for Medical Information Extraction

Paper · 2026 · Doan Nhat Tien and Thìn Đặng Văn

Summary

The authors, a team from the University of Information Technology (VNU-HCM), took part in two #SMM4H-HeaRD 2026 shared tasks. Task 1 is binary detection of Adverse Drug Event (ADE) mentions in social media posts in six languages, with Farsi added only at test time to probe zero-shot transfer. For this task they fine-tuned InfoXLM and XLM-RoBERTa, trained Qwen3.5-9B-Base with LoRA instruction tuning, and combined the three through voting ensembles. Training data came from the task data plus translated CADEC, with the positive class upsampled. Task 7 is a token-level NER problem: finding ClinicalImpacts and SocialImpacts spans in first-person opioid narratives. Here they compared a generative Qwen tagger with DeBERTa-Large trained either on BIO tags or on a reduced three-label scheme.

Qwen performed best overall and on seen languages. The soft-voting ensemble, however, transferred better to Farsi. In Task 7 the encoders clearly beat the LLM, and the simplified labels gave a small edge. The study shows that architectural diversity helps with unseen scripts and that a smaller label space can suffice when entity boundaries are rarely ambiguous.

Key points

  • Task 1, development set: the soft-voting ensemble reached 0.8584 macro F1, compared with 0.8543 for Qwen alone.
  • Task 1, test set: Qwen scored 0.6623 overall and 0.7853 on known languages, but only 0.4865 on Farsi. The ensemble reached 0.5304 on Farsi, above the team median of 0.3797.
  • Task 7, test set: DeBERTa-Large with labels O/ClinicalImpacts/SocialImpacts achieved 0.500 strict F1 and 0.583 relaxed F1. BIO tagging achieved 0.484 strict and 0.564 relaxed.
  • Task 7, LLM baseline: the Qwen3.5-9B + LoRA tagger, which output index:token:label triples, reached only 0.2604 strict and 0.2961 relaxed F1 on the development set. This reflects its unreliable structured generation.
  • Setup: training ran on a single NVIDIA RTX Pro 6000 Blackwell GPU (96GB) with the Transformers framework, and the code is released at github.com/TienDoan274/SMM4H.
  • Limitations: apart from 3x upsampling, no data augmentation was used. The simplified labeling scheme cannot separate adjacent spans of the same type.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vietnam National University Ho Chi Minh CityHo Chi Minh City · Vietnam21 works · 14 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 worksPharmacology, Toxicology and Pharmaceutics · 7 worksToxicology · 6 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 167 works

Cite

Cite as Doan Nhat Tien and Thìn Đặng Văn. 2026. DNT at #SMM4H–HeaRD 2026: Leveraging BERT-based Encoders and LLMs for Medical Information Extraction. Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.smm4h-1.7

@inproceedings{tien2026dnt,
  title = {{DNT at \#SMM4H–HeaRD 2026: Leveraging BERT-based Encoders and LLMs for Medical Information Extraction}},
  author = {Doan Nhat Tien and Thìn DJặng Văn},
  booktitle = {Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
  year = {2026},
  doi = {10.18653/v1/2026.smm4h-1.7},
  url = {https://doi.org/10.18653/v1/2026.smm4h-1.7}
}

References

4 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 30 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  2. Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. 2021. Infoxlm: An information-theoretic framework for cross-lingual language model pre-training. In Proceedings of the 2021 Conference of the North American Chapter of the Assoc…
  3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  4. Sumon Kanti Dey, Jeanne M. Powell, Azra Ismail, Jean-marie Perrone, and Abeed Sarker. 2025. Inference gap in domain expertise and machine intelligence in find

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tien-2026-dnt-smm4h-heard-2026-leveraging/