Elamur Research⌕
☰
home / papers / 2026 / DANGNT@SGU at SemEval-2026 Task 1: A Two-Stage Mistral Generator with DistilBERT Reranking for English Humor Generation

DANGNT@SGU at SemEval-2026 Task 1: A Two-Stage Mistral Generator with DistilBERT Reranking for English Humor Generation

Paper · 2026 · Tan Loc Nguyen and Dang Tuan Nguyen

Summary

This paper describes the system DANGNT@SGU submitted to Subtask A of SemEval-2026 Task 1 (MWAHAHA), in the English-language category. The task asks systems to write short jokes under two kinds of constraints: including two given words (Word Inclusion) or relating to a news headline (News Headline). Organizers supplied no reference jokes, so the authors built their own training data. For Word Inclusion, they used spaCy to extract keywords from the Kaggle Short Jokes corpus. For News Headline, they paired official headlines with jokes through sentence-embedding similarity.

They fine-tuned Mistral-7B-Instruct-v0.2 with QLoRA in two steps:

  • Stage 1: the model learns general joke style from raw jokes.
  • Stage 2: the model learns the task's prompt formats.

At inference, the system samples five candidates per prompt. A DistilBERT classifier, trained to separate jokes from Wikipedia text, picks the best one.

The submission placed 13th with Elo 962. Local ablations suggest that each training stage contributes a different benefit. The pipeline uses only open models and public data, which makes it a reproducible baseline for controllable humor generation.

Key points

  • Official result: 13th place in the English leaderboard, Elo 962 (95% CI 926–986), based on human pairwise judgments.
  • Resources:
    • Generator: mistralai/Mistral-7B-Instruct-v0.2 with 4-bit NF4 QLoRA (r = 64, α = 16, adapters on q/k/v/o).
    • Reranker: distilbert-base-uncased.
    • Headline–joke matching: all-MiniLM-L6-v2 with a cosine threshold of 0.4.
    • Reranker negatives: wikimedia/wikipedia (20231101.en).
  • Training data balance: Stage 2 caps Word Inclusion to News Headline pairs at roughly 10:1.
  • Local ablation (100 prompts, top-1 output, no reranker):
    • Stage 2 raised word-inclusion success from 68% to 96%.
    • Adding Stage 1 lifted the mean DistilBERT joke score from 0.58 to 0.74, versus 0.44 for zero-shot.
  • Reported weaknesses:
    • Outputs can include the required words yet still be unfunny.
    • Jokes drift away from the headline topic.
    • The reranker favors generic short jokes.
    • The code is released at github.com/tanloc49/dangnt-sgu-mwahaha.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Ton Duc Thang University.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksSocial Psychology · 330 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksInformation retrieval & search · 552 works

Cite

Cite as Tan Loc Nguyen and Dang Tuan Nguyen. 2026. DANGNT@SGU at SemEval-2026 Task 1: A Two-Stage Mistral Generator with DistilBERT Reranking for English Humor Generation. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.375

@inproceedings{nguyen2026dangnt,
  title = {{DANGNT@SGU at SemEval-2026 Task 1: A Two-Stage Mistral Generator with DistilBERT Reranking for English Humor Generation}},
  author = {Tan Loc Nguyen and Dang Tuan Nguyen},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.375},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.375}
}

References

10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Santiago Castro, Luis Chiruzzo, Santiago Góngora, Salar Rahili, Naihao Deng, Ignacio Sastre, Victoria Amoroso, Guillermo Rey, Aiala Rosá, Guillermo Moncecchi, J. A. Meaney, Juan José Prada, and Rada Mihalcea. 2026. SemEval-2026 Task 1: MWA-HAHA, Models Write Automatic Humor And Humans Annotate. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  2. Santiago Castro and Luis Chiruzzo. 2026. Submission list, annotation, and paper submission. Google Groups post to the semeval-2026-task-1-humor-gen mailing list, February 3, 2026. find
  3. Luis Chiruzzo. 2026. Final results. Google Groups post to the semeval-2026-task-1-humor-gen mailing list, February 21, 2026. find
  4. Albert Q. Jiang et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. ↗
  5. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. ↗
  6. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems (NeurIPS).
  7. Peng-Yu Chen and Von-Wun Soo. 2018. Humor Recognition Using Deep Learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 113–117, New Orleans, Louisiana. Association for Computational Linguistics. doi:10.18653/v1/N18-2018. ↗
  8. Abhinav Moudgil. n.d. Short Jokes. Kaggle dataset. https://www.kaggle.com/datasets/ abhinavmoudgil95/short-jokes. Accessed: 2026-02-25. ↗
  9. Hugging Face and Wikimedia Foundation. n.d. wikimedia/wikipedia. Hugging Face Datasets. https://huggingface.co/datasets/ wikimedia/wikipedia. Configuration used: 20231101.en. Accessed: 2026-02-25. ↗
  10. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP-IJCNLP).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2026-dangnt-sgu-semeval-2026-task/