📄 MLInitiative at CASE 2025: Multimodal Detection of Hate Speech, Humor,and Stance using Transformers

Summary

The paper addresses the automatic analysis of LGBTQ+-themed memes. In such memes, content, including hateful content, is conveyed jointly by the image and the text embedded in it. The authors took part in the CASE@RANLP 2025 shared task, which has four subtasks. The first is hate speech detection. The second is identifying the target of a hateful statement: an individual, a community, an organization, or no one in particular. The third is stance classification toward the movement: support, neutrality, or opposition. The fourth is humor recognition. The data were provided by the organizers. The team had no OCR of its own, so it used the text the organizers had already extracted from the memes. The authors compared two multimodal architectures. The first combines ResNet-18 for images and BERT for text: the features are concatenated, and label smoothing is applied for the imbalanced classes. The second is SigLIP2 with frozen encoders, on top of which a small classifier is trained. According to the authors, SigLIP2 consistently outperforms the baseline. It gave the best results for hate speech and humor, while stance and target proved more difficult. The work shows that even a lightweight approach without fine-tuning the encoders yields competitive results in moderating multimodal content about vulnerable groups.

Key points

• SigLIP2 achieved an F1 of 79.27 for hate speech (Subtask A), 72.88 for humor (D), 60.59 for stance (C), and 54.86 for target (B).
• Data size: for subtasks A, C, and D — 4,050 examples for training, 506 for validation, and 507 for testing. For B, which includes only hateful memes — 1,985 / 248 / 249.
• The model used is SigLIP2-large-patch16-256 with a sigmoid-based contrastive loss. Its image and text embeddings are concatenated, projected into a smaller space, and passed through ReLU and dropout before classification.
• The alternative system is ResNet-18 (pretrained on ImageNet) plus BERT-base, followed by two fully connected layers.
• Hyperparameters were searched in the following space: batch size 16 or 32, learning rate from 1e-6 to 5e-5, AdamW, 10 epochs. The data come from PrideMM and related corpora from the organizers.