Elamur Research⌕
☰
home / papers / 2026 / CUET_SYNTHETICA@EEUCA 2026: Gated Cross-Modal Attention with Domain-Adapted Text Encoding for Vaccine-Critical Meme Detection

CUET_SYNTHETICA@EEUCA 2026: Gated Cross-Modal Attention with Domain-Adapted Text Encoding for Vaccine-Critical Meme Detection

Paper · 2026 · Sumaiya Zaman, Miftahul Jannat Rishta and Shiti Chowdhury

Summary

The paper addresses three-way stance classification of vaccine-related memes (Vaccine-Critical, Neutral, Pro-Vaccine) for the EEUCA 2026 shared task. It uses the VaxMeme dataset, where each item combines an image, post text and OCR-extracted overlay text. The system joins post and OCR text and encodes them with Twitter-RoBERTa, encodes the image with CLIP ViT-L/14, and fuses the two with 8-head cross-attention, using text as the query. A learned scalar gate then lets the model rely more on text when the image adds little. Training combines weighted focal loss with an upweighted Neutral class and gradual, low-rate unfreezing of CLIP. The final prediction comes from a three-model soft-voting ensemble weighted by validation macro F1, which includes a copy retrained on train plus evaluation data. The system reaches 0.8357 macro F1 on the test set and ranks 3rd. It offers a documented multimodal baseline for monitoring health misinformation. However, the gate's own effect is not ablated, and the "domain over scale" claim is weakened by text-only validation numbers where RoBERTa-large scores slightly higher.

Key points

  • Dataset: VaxMeme has 8,195 train, 1,024 evaluation and 1,025 test memes, with mild imbalance toward Pro-Vaccine. Code is on GitHub (Mif-taa/VaxMemeStance).
  • Validation macro F1 by setup:
    • Twitter-RoBERTa + CLIP ViT-L/14 (multimodal): 0.8079.
    • Ensemble: 0.8090.
    • Text-only Twitter-RoBERTa: 0.8125.
    • Text-only RoBERTa-large: 0.8165.
    • Best image-only model (CLIP ViT-B/32): 0.7479.
  • Training setup:
    • Focal loss with γ=2 and Neutral weight 1.4.
    • AdamW optimizer.
    • CLIP frozen for 4 epochs, then tuned at a learning rate of 1e-7.
    • Shared 512-dimensional fusion space feeding an MLP head (512→256→3).
  • Error analysis (evaluation set):
    • 193 misclassifications in total, mostly between neighbouring classes.
    • Only 20 errors fall between the two opposite stances.
    • 61 Vaccine-Critical memes were labelled Neutral, often sarcastic or ambiguous ones.
    • Pro-Vaccine recall was 86.38%.
  • Explainability: Integrated Gradients highlights keywords such as "antivax" and "side effect". LIME points to symbolic imagery and text overlays as the main visual cues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh28 works · 29 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 worksSocial Sciences · 1,954 worksSociology and Political Science · 626 works

Cite

Cite as Sumaiya Zaman, Miftahul Jannat Rishta and Shiti Chowdhury. 2026. CUET_SYNTHETICA@EEUCA 2026: Gated Cross-Modal Attention with Domain-Adapted Text Encoding for Vaccine-Critical Meme Detection. Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eeuca-1.14

@inproceedings{zaman2026cuet,
  title = {{CUET\_SYNTHETICA@EEUCA 2026: Gated Cross-Modal Attention with Domain-Adapted Text Encoding for Vaccine-Critical Meme Detection}},
  author = {Sumaiya Zaman, Miftahul Jannat Rishta and Shiti Chowdhury},
  booktitle = {Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.eeuca-1.14},
  url = {https://doi.org/10.18653/v1/2026.eeuca-1.14}
}

References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. Tweeteval: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1644–1650. Association for Computational Linguist…
  2. Aashish Bhandari, Siddhant B Shah, Surendrabikram Thapa, Usman Naseem, and Mehwish Nasim. 2023. Crisishatemm: Multimodal analysis of directed and undirected hate speech in text-embedded images from russia-ukraine conflict. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco… find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Ali Hürriyetoğlu, Surendrabikram Thapa, Hristo Tanev, Laxmi Thapa, and Surabhi Adhikari. 2026. Overview of the workshop on event extraction and understanding: Challenges and applications. In Proceedings of… find
  4. Christos Koutlis, Manos Schinas, and Symeon Papadopoulos. 2023. MemeFier: Dual-stage modality fusion for image meme classification. arXiv preprint arXiv:2304.02906. — [arXiv:2304.02906](https://arxiv.org/abs/2304.02906) find
  5. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  6. Usman Naseem, Jinman Kim, Matloob Khushi, and Adam G. Dunn. 2023. A multimodal framework for the identification of vaccine critical memes on Twitter. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 706–714. find
  7. Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, et al. 2023. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193.
  8. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning.
  9. Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. 2022. FLAVA: A foundational language and vision alignment model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15638– 15650. find
  10. Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buitelaar. 2020. Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text. In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, pages 32–41.
  11. Laxmi Thapa, Aryaman Jain, Lakshmojee Koduru, Surabhi Adhikari, Junaid Rashid, Jungeun Kim, Surendrabikram Thapa, and Usman Naseem. 2026a. Concept-grounded detection of vaccine misinformation in multimodal content using interpretable vision-language models. In Companion Proceedings of the ACM on Web… find
  12. Surendrabikram Thapa, Kritesh Rauniyar, Hariram Veeramani, Aditya Shah, Imran Razzak, and Usman Naseem. 2024. Did you tell a deadly lie? evaluating large language models for health misinformation identification. In International Conference on Web Information Systems Engineering, pages 391–405. Sprin… find
  13. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Surabhi Adhikari, Hariram Veeramani, Mehwish Nasim, and Usman Naseem. 2025. Large language models (llm) in computational social science: prospects, current state, and challenges. Social Network Analysis and Mining, 15(1):1–30. find
  14. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Kritesh Rauniyar, Laxmi Thapa, Surabhi Adhikari, Kristina T. Johnson, Ali Hür-riyetoğlu, Hristo Tanev, and Usman Naseem. 2026b. Multimodal identification of vaccine content stance on social media. In Proceedings of the 9th Workshop on Ev…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaman-2026-cuet-synthetica-eeuca-2026-gated/