Elamur Research⌕
☰
home / papers / 2026 / _alexcristea@EEUCA 2026: A Robust Early-Fusion ERNIE Pipeline for Multimodal COVID-19 Vaccine Meme Classification

_alexcristea@EEUCA 2026: A Robust Early-Fusion ERNIE Pipeline for Multimodal COVID-19 Vaccine Meme Classification

Paper · 2026 · Cristea Alexandru-Marian and Costin Ionescu

Summary

The paper presents a system built for the multimodal vaccine-critical meme detection shared task held at EEUCA 2026. Each instance is a social media post with an attached image, labeled Vaccine Critical, Neutral, or Pro-Vaccine. The dataset has over 10,000 informal English posts, with imbalanced classes and noisy labels. Many instances lack either the caption or the image text.

Instead of a costly vision-language architecture, the authors reframe the problem as text-pair classification. They extract the text embedded in each image via OCR, join it with the caption using [SEP] tokens, and feed the result to ERNIE-2.0-Large with a 128-token limit. The [CLS] vector goes to a Multi-Sample Dropout head, trained with class-weighted Cross-Entropy.

The system reached 0.834 Macro F1 on the hidden test set, ranking 4th. Ablations suggest Focal Loss offers no benefit on noisy labels, and machine-translating memes into Chinese degrades results sharply. The work shows that a lightweight text-only pipeline can compete with multimodal systems. However, it cannot detect sarcasm expressed only through visuals.

Key points

  • Input and encoder: the input is formatted as [CLS] post [SEP] OCR text [SEP] and encoded by ernie-2.0-large-en (hidden size 1024). The model was trained with AdamW (lr 2×10⁻⁵, weight decay 0.05) for 10 epochs, using FP16 and an effective batch size of 256.
  • Classification head: the Multi-Sample Dropout head uses N = 5 parallel dropout masks (p = 0.3) feeding one shared linear layer whose outputs are averaged. Class imbalance is handled with inverse-frequency class weights.
  • Development-set results (mean over seeds 42, 22, and 100):
    • standard-head baseline: 0.8136;
    • MSD with Focal Loss: 0.8221;
    • MSD with Cross-Entropy (final system): 0.8238 ± 0.0102.
  • Translation ablation: ERNIE 3.0 Base applied to memes translated into Chinese fell to 0.7240 Macro F1. The authors attribute this to the loss of culture-specific slang in translation.
  • Error analysis: the main failures are ironic or debunking posts misread through critical hashtags (e.g., #vaccineinjury) and stances conveyed only by visual reaction faces.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BucharestBucharest · Romania21 works · 516 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Cristea Alexandru-Marian and Costin Ionescu. 2026. _alexcristea@EEUCA 2026: A Robust Early-Fusion ERNIE Pipeline for Multimodal COVID-19 Vaccine Meme Classification. Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eeuca-1.20

@inproceedings{alexandrumarian2026alexcristea,
  title = {{\_alexcristea@EEUCA 2026: A Robust Early-Fusion ERNIE Pipeline for Multimodal COVID-19 Vaccine Meme Classification}},
  author = {Cristea Alexandru-Marian and Costin Ionescu},
  booktitle = {Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.eeuca-1.20},
  url = {https://doi.org/10.18653/v1/2026.eeuca-1.20}
}

References

10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aashish Bhandari, Siddhant B Shah, Surendrabikram Thapa, Usman Naseem, and Mehwish Nasim. 2023. Crisishatemm: Multimodal analysis of directed and undirected hate speech in text-embedded images from russia-ukraine conflict. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco… find
  2. Ali Hürriyetoğlu, Surendrabikram Thapa, Hristo Tanev, Laxmi Thapa, and Surabhi Adhikari. 2026. Overview of the workshop on event extraction and understanding: Challenges and applications. In Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA). find
  3. Hiroshi Inoue. 2019. Multi-sample dropout for accelerated training and better generalization. Neural Networks, 111:66–73. find
  4. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988.
  5. Usman Naseem, Jinman Kim, Matloob Khushi, and Adam G Dunn. 2023. A multimodal framework for the identification of vaccine critical memes on twitter. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 706–714. find
  6. Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020. Ernie 2.0: A continual pre-training framework for language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8963–8970.
  7. Laxmi Thapa, Aryaman Jain, Lakshmojee Koduru, Surabhi Adhikari, Junaid Rashid, Jungeun Kim, Surendrabikram Thapa, and Usman Naseem. 2026a. Concept-grounded detection of vaccine misinformation in multimodal content using interpretable vision-language models. In Companion Proceedings of the ACM on Web… find
  8. Surendrabikram Thapa, Kritesh Rauniyar, Hariram Veeramani, Aditya Shah, Imran Razzak, and Usman Naseem. 2024. Did you tell a deadly lie? evaluating large language models for health misinformation identification. In International Conference on Web Information Systems Engineering, pages 391–405. Sprin… find
  9. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Surabhi Adhikari, Hariram Veeramani, Mehwish Nasim, and Usman Naseem. 2025. Large language models (llm) in computational social science: prospects, current state, and challenges. Social Network Analysis and Mining, 15(1):1–30. find
  10. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Kritesh Rauniyar, Laxmi Thapa, Surabhi Adhikari, Kristina T. Johnson, Ali Hür-riyetoğlu, Hristo Tanev, and Usman Naseem. 2026b. Multimodal identification of vaccine content stance on social media. In Proceedings of the 9th Workshop on Ev…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marian-2026-alexcristea-eeuca-2026-robust-early/