Elamur Research⌕
☰
home / papers / 2026 / BioNLP at #SMM4H-HeaRD 2026 Task 3 Estimating Flu Vaccine Effectiveness: A Temporal-Aware Fine-Tuning and Similarity-Based Few-Shot Prompting Approach

BioNLP at #SMM4H-HeaRD 2026 Task 3 Estimating Flu Vaccine Effectiveness: A Temporal-Aware Fine-Tuning and Similarity-Based Few-Shot Prompting Approach

Paper · 2026 · Irina Patularu

Summary

This work, submitted to SMM4H-HeaRD 2026 Task 3, classifies tweets from the 2020–2021 influenza season along two five-label schemes. Subtask 1 covers the author's flu vaccination status, and Subtask 2 covers flu test outcome. Both schemes separate current-season events from earlier ones.

Two systems were built and compared on each subtask:

  • Fine-tuned BERTweet-large. The model is made time-aware by prepending a readable date prefix to each tweet and by projecting four date-derived features through an MLP that is fused with the [CLS] vector. Training adds class-weighted label-smoothed loss, R-Drop, Multi-Sample Dropout and a 10-checkpoint cross-validation ensemble.
  • Prompted GPT-4o. The prompt uses chain-of-thought and includes, for each label, the most similar training tweet, retrieved from ChromaDB. The model is asked to rank all candidate labels rather than pick one.

Fine-tuning performed best for vaccination status (micro-F1 87.90). Prompting performed best for the smaller, heavily skewed test-outcome data (95.74). Both winning scores exceed the organizers' baselines.

The study suggests that dataset size and label balance should guide whether to fine-tune or prompt, and that temporal signals help with both methods.

Key points

  • Data: 4,216 tweets in total. The vaccination subtask has 2,809 tweets (1,977 train / 270 validation / 562 test), and the test-outcome subtask has 1,407 (990 / 135 / 282). In the test-outcome data, the Other class makes up about 71%.
  • Retrieval setup: Rationales for the examples were produced by Llama-3.3-70B. Tweets were embedded with mixedbread-ai/mxbai-embed-large-v1 and searched by cosine similarity. Inference used gpt-4o-2024-08-06 with temperature 0.
  • Fine-tuning setup:
    • Temporal features (month, flu-season week, season progress, peak-season flag) are expanded to 64 dimensions and concatenated with the 1024-dimensional [CLS] embedding.
    • Training uses 5-fold stratified CV over 2 seeds, layer-wise learning-rate decay (LLRD) and soft-voting.
  • Test results: For vaccination, BERTweet-large reached 87.90 micro-F1, ahead of the best baseline (Few-shot + CoT, 84.88), while prompting reached 83.99. For test outcome, prompting reached 95.74, ahead of the 95.03 baseline, while BERTweet-large reached 88.30.
  • Weakness of fine-tuning on scarce data: On the test-outcome subtask, fine-tuned BERTweet-large scored only 60.00 F1 on Currently-Positive and 68.29 on Currently-Negative.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BucharestBucharest · Romania21 works · 516 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Irina Patularu. 2026. BioNLP at #SMM4H-HeaRD 2026 Task 3 Estimating Flu Vaccine Effectiveness: A Temporal-Aware Fine-Tuning and Similarity-Based Few-Shot Prompting Approach. Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.smm4h-1.41

@inproceedings{patularu2026bionlp,
  title = {{BioNLP at \#SMM4H-HeaRD 2026 Task 3 Estimating Flu Vaccine Effectiveness: A Temporal-Aware Fine-Tuning and Similarity-Based Few-Shot Prompting Approach}},
  author = {Irina Patularu},
  booktitle = {Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
  year = {2026},
  doi = {10.18653/v1/2026.smm4h-1.41},
  url = {https://doi.org/10.18653/v1/2026.smm4h-1.41}
}

References

14 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A Survey on In-context Learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107–1128, Miami…
  3. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, Melbourne, Australia. Association for Computational Linguistics. find
  4. Hiroshi Inoue. 2019. Multi-sample dropout for accelerated training and better generalization. arXiv preprint arXiv:1905.09788. — [arXiv:1905.09788](https://arxiv.org/abs/1905.09788) find
  5. Xuancheng Liang, Lijun Wu, Juntao Li, Yue Wang, Qi Meng, Tao Qin, Wei Chen, Min Zhang, and Tie-Yan Liu. 2021. R-drop: Regularized dropout for neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 10961–10973. Curran Associates, Inc. find
  6. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out: The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures (DeeLIO 2022), pages 84–92.…
  7. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  8. Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 9–14, Online. Association for Computational Linguistics.
  9. Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune BERT for text classification? arXiv preprint arXiv:1905.05583.
  10. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
  11. Davy Weissenbacher, Abeed Sarker, Arjun Magge, Ashlynn Daughton, Karen O’Connor, Michael J. Paul, and Graciela Gonzalez-Hernandez. 2019. Overview of the fourth social media mining for health (SMM4H) shared tasks at ACL 2019. In Proceedings of the Fourth Social Media Mining for Health Applications (\#… find
  12. Dongfang Xu, Guillermo López García, Karen O’Connor, Haily Holston, Ari Z Klein, Ivan Flores Amaro, Matthew Scotch, and Graciela Gonzalez-Hernandez. 2025. Mining Social Media Data for Influenza Vaccine Effectiveness Using a Large Language Model and Chain-of-Thought Prompting. In AMIA Annual Symposiu… find
  13. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  14. Guillermo Lopez-Garcia, Jose Miguel Acitores Cortina, Jacob Berkowitz, Joey Chan, Ganesh Chandrasekar, Sumon Kanti Dey, Ivan Flores Amaro, Fernando Gallego, Lauren Gryboski, Ari Z Klein, Martin Krallinger, Salvador Lima-López, Tomohiro Nishiyama, Lisa Raithel, Ahmad Rezaie Mianroodi, Roland Roller,… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/patularu-2026-bionlp-smm4h-heard-2026-task/