Elamur Research⌕
☰
home / papers / 2026 / SLPGFJWUWarda at SemEval-2026 Task 1: A Multimodal Vision-Language Approach for Humor Generation Using Fine-Tuned BLIP

SLPGFJWUWarda at SemEval-2026 Task 1: A Multimodal Vision-Language Approach for Humor Generation Using Fine-Tuned BLIP

Paper · 2026 · Warda Yousaf

Summary

This system paper describes a submission to SemEval-2026 Task 1 (MWAHAHA), Subtask B1. The subtask asks a model to produce a free-form humorous English caption for a still image or an animated GIF. Standard captioning models describe what is literally visible, so the author tries to steer a vision-language model toward meme-style jokes. The pipeline keeps only the first frame of each GIF, converted to RGB with PIL. It then fine-tunes the pre-trained blip-image-captioning-base model, which pairs a ViT image encoder with a BERT-based causal text decoder. Fine-tuning uses a standard language-modeling loss on a filtered portion of the MemeCap dataset, and captions are decoded with beam search. The fine-tuned model typically writes first-person, exaggerated captions in the familiar "Me trying to…" style. It received a leaderboard rating of 1077 on the 300 test items, reported as a shared first place. However, the official table lists the zero-shot BLIP baseline at a higher rating (1124), also ranked 1. The work illustrates a cheap, single-GPU recipe for adapting captioning models to humor.

Key points

  • Data: MemeCap captions were filtered to 3–20 words, leaving 4,206 samples split 90/10 into train and validation.
  • Training setup: AdamW with a learning rate of 3e-5, 2 epochs, batch size 2, and AMP on a Google Colab T4 GPU. The best checkpoint reached a validation loss of 0.1075.
  • Decoding and GIF handling: Beam search used num_beams = 5 and max_length = 30. Only the first frame (t = 0) of each GIF is used.
  • Official results: The fine-tuned BLIP scored 1077 and the zero-shot BLIP baseline scored 1124, with both listed at Rank 1.
  • Main weakness and future work: The model struggles with images whose humor depends on embedded text, because BLIP has no OCR. Planned extensions are OCR integration and sampling a few frames from GIFs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fatima Jinnah Women UniversityRawalpindi · Pakistan3 works

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksComputer Vision and Pattern Recognition · 3,486 worksPsychology · 1,649 worksSocial Psychology · 330 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 811 works

Cite

Cite as Warda Yousaf. 2026. SLPGFJWUWarda at SemEval-2026 Task 1: A Multimodal Vision-Language Approach for Humor Generation Using Fine-Tuned BLIP. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.156

@inproceedings{yousaf2026slpgfjwuwarda,
  title = {{SLPGFJWUWarda at SemEval-2026 Task 1: A Multimodal Vision-Language Approach for Humor Generation Using Fine-Tuned BLIP}},
  author = {Warda Yousaf},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.156},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.156}
}

References

4 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Santiago Castro, Luis Chiruzzo, Santiago Góngora, Salar Rahili, Naihao Deng, Ignacio Sastre, Victoria Amoroso, Guillermo Rey, Aiala Rosá, Guillermo Moncecchi, J. A. Meaney, Juan José Prada, and Rada Mihalcea. 2026. SemEval-2026 Task 1: MWA-HAHA, Models Write Automatic Humor And Humans Annotate. In P… find
  2. Eu-Jin Hwang, Seong Joon Oh, and Seung-Won Hwang. 2021. Memecap: A dataset for image-based meme captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. find
  3. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the International Conference on Machine Learning (ICML), pages 12888–12900. PMLR.
  4. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, and 1 others. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yousaf-2026-slpgfjwuwarda-semeval-2026-task-1/