Elamur Research⌕
☰
home / papers / 2026 / Semantica@DravidianLangTech 2026: Vision-Language Models for Hierarchical Political Meme Classification in Tamil and Malayalam

Semantica@DravidianLangTech 2026: Vision-Language Models for Hierarchical Political Meme Classification in Tamil and Malayalam

Paper · 2026 · Junain Uddin, Rahul Datta, Taha Ibne Abdullah et al.

Summary

The paper describes the Semantica team's entry in the DravidianLangTech@ACL 2026 shared task, which asks systems to label Tamil and Malayalam political memes at two levels of a hierarchy. In the task, a system first decides whether a meme supports or trolls/opposes (Level-1). It then assigns a finer label identifying the target, such as a person, a party or, for Malayalam only, both ("Intersection") (Level-2). The data are small and heavily skewed: 803/201 train/test memes for Tamil and 500/100 for Malayalam, dominated by trolling content. The authors compare three families of approaches:

  • text-only encoders working on OCR output,
  • an early-fusion model combining mBERT and EfficientNet-B3,
  • instruction-tuned Qwen-VL models adapted with 4-bit LoRA and minority-class oversampling.

The official submission, Qwen2.5-VL-7B, placed 3rd for Malayalam and 4th for Tamil by weighted-F1. Later experiments with Qwen3-VL-8B gave the best Tamil scores and the best Level-2 scores. The study suggests that vision-language models handle culturally nuanced memes in low-resource languages better than text-only pipelines. It also shows that accuracy alone can hide failure on rare classes.

Key points

  • Dataset and imbalance: Level-1 training labels are 691 vs 112 (Troll/Oppose vs Support) for Tamil and 477 vs 23 for Malayalam. Tamil has 4 Level-2 classes and Malayalam has 5. The Malayalam test set contains only 4 Support memes.
  • Text-only pipeline: Qwen2-VL-2B-Instruct extracted the meme text via OCR. IndicBERTv2 and XLM-RoBERTa were fine-tuned on this text. The predicted Level-2 label was prepended to the text as input for the Level-1 classifier.
  • VLM setup:
    • Qwen2.5-VL-7B and Qwen3-VL-8B read the memes directly, without OCR.
    • Fine-tuning used 4-bit quantization and LoRA with r=16 and α=16, batch size 2, learning rate 2e-5, and 8-bit AdamW with a cosine schedule.
    • Minority classes were oversampled to about 200 examples each.
  • Best results (Qwen3-VL-8B):
    • Tamil Level-1: macro-F1 82.12 and weighted-F1 91.22.
    • Level-2 macro-F1: 49.70 for Tamil and 36.40 for Malayalam.
  • Malayalam Level-1 caveat: text-only models reach 96% accuracy simply by predicting the majority class, yet their macro-F1 is only about 49. The VLMs score even lower on macro-F1 here (about 31). This contrasts with the paper's conclusion that Qwen3-VL-8B leads across all settings, and it highlights the need for macro-averaged metrics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh25 works · 29 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksVision & language · 657 works

Cite

Cite as Junain Uddin, Rahul Datta, Taha Ibne Abdullah and Hasan Murad. 2026. Semantica@DravidianLangTech 2026: Vision-Language Models for Hierarchical Political Meme Classification in Tamil and Malayalam. Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.dravidianlangtech-1.54

@inproceedings{uddin2026semantica,
  title = {{Semantica@DravidianLangTech 2026: Vision-Language Models for Hierarchical Political Meme Classification in Tamil and Malayalam}},
  author = {Junain Uddin, Rahul Datta, Taha Ibne Abdullah and Hasan Murad},
  booktitle = {Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages},
  year = {2026},
  doi = {10.18653/v1/2026.dravidianlangtech-1.54},
  url = {https://doi.org/10.18653/v1/2026.dravidianlangtech-1.54}
}

References

11 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966.
  2. Mithun Das, Somnath Banerjee, and Animesh Mukherjee. 2022. hate-alert@dravidianlangtech-acl2022: Ensembling multi-modalities for tamil trollmeme classification. Preprint, arXiv:2204.12587. find
  3. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. 8-bit optimizers via block-wise quantization. arXiv preprint arXiv:2110.02861.
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171– 4186.
  5. Siddhanth U Hegde, Adeep Hande, Ruba Priyadharshini, Sajeetha Thavareesan, and Bharathi Raja Chakravarthi. 2021. Uvce-iiitt@dravidianlangtech-eacl2021: Tamil troll meme classification: You need to pay more attention. Preprint, arXiv:2104.09081. find
  6. Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque, and Sarah M. Preum. 2024. Deciphering hate: Identifying hateful memes and their targets. Preprint, arXiv:2403.10829. find
  7. Jiasen Lu et al. 2019. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in Neural Information Processing Systems.
  8. Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. Momenta: A multimodal framework for detecting harmful memes and their targets. Preprint, arXiv:2109.05184.
  9. Saranya Rajiakodi, Shunmuga Priya Muthusamy Chinnan, Premjith B, Subalalitha CN, Rahul Ponnusamy, Anshid K A, Bhuvaneshwari Sivagnanam, Jananaya-gan V, Bharathi Raja Chakravarthi, Ragavan N, and Santhini P. 2026. Overview of the shared task on multilevel political meme classification in tamil and ma…
  10. Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pages 6105–6114. PMLR.
  11. A Md. Refaj Hossan, Nazmus Sakib, Md. Alam Miah, Jawad Hossain, and Mohammed Moshiul Hoque. 2025. CUET-NLP_Big_O@DravidianLangTech 2025: A multimodal fusion-based approach for identifying misogyny memes. In Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/uddin-2026-semantica-dravidianlangtech-2026-vision-language/