Elamur Research⌕
☰
home / papers / 2026 / DIAG-Sapienza at GSI:detect: Joint Detection and Classification of Gender Stereotypes with Structured Prompting and Fine-Tuning

DIAG-Sapienza at GSI:detect: Joint Detection and Classification of Gender Stereotypes with Structured Prompting and Fine-Tuning

Paper · 2026 · Olga E. Sorokoletova, Emanuele Musumeci and Daniele Nardi

Summary

The paper describes the DIAG-Sapienza team's entry to GSI:detect at EVALITA 2026. The challenge concerns gender stereotypes in short Italian texts taken from social media and informative websites. Systems must score stereotype intensity on a [0, 1] scale, which is the regression main task, and assign a stereotype category, which is the optional sub-task. The authors treat both outputs as one joint problem. Their reasoning is that LLMs handle bare numerical prediction poorly, and category information may help them estimate intensity.

They built one system for each challenge track:

  • Zero-shot: GPT-5 acts as an LLM-as-a-Judge with system, assistant and user prompts. The prompts restate the task's definitions of a stereotype and its categories. The model must also write a short English explanation.
  • Few-shot: The same setup adds in-context examples, either randomly sampled or retrieved.
  • Fine-tuned: An encoder-only RoBERTa-based model takes LLM-generated reasoning sentences as extra input.

On the official test set, the zero-shot and few-shot systems placed first and second on the regression task. The results suggest that joint prediction and requested explanations improve stereotype scoring when training data is scarce.

Key points

  • Label space: six categories (Role, Personality, Competence, Physical, Sexual, Relational) plus None. GPT-5 rounds its scores to 0, 0.25, 0.5, 0.75 or 1 and returns gs_value, gs_category and an explanation in a single JSON call.
  • Context handling: any surrounding context is kept separate from the text. The model is told to use it only for interpreting the text.
  • Retrieval in few-shot: retrieval-based example selection (RAG) gave better classification on the development set but no gain in regression. The team therefore submitted the variant without RAG.
  • Fine-tuning ablation: in the ablation of the ~125M-parameter single-encoder model, validation F1 rose from 0.1257 for the baseline to 0.8293 with 20% synthetic data and 0.8834 with 50% synthetic data (150 epochs). A ~250M dual encoder collapsed to 0.0408.
  • Fine-tuned vs. prompting: the fine-tuned model ranked below both prompting systems. The authors attribute this to the small, imbalanced training set.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGender Studies · 52 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 517 works

Cite

Cite as Olga E. Sorokoletova, Emanuele Musumeci and Daniele Nardi. 2026. DIAG-Sapienza at GSI:detect: Joint Detection and Classification of Gender Stereotypes with Structured Prompting and Fine-Tuning. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{sorokoletova2026diag,
  title = {{DIAG-Sapienza at GSI:detect: Joint Detection and Classification of Gender Stereotypes with Structured Prompting and Fine-Tuning}},
  author = {Olga E. Sorokoletova, Emanuele Musumeci and Daniele Nardi},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.12/}
}

References

12 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. G. Comandini, M. Speranza, S. Brenna, D. Testa, S. Cavagnoli, B. Magnini, Gsi:detect at evalita 2026: Overview of the task on detecting gender stereotypes in italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA… find
  2. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  3. A. Testolin, Can neural networks do arithmetic? a survey on the elementary numerical skills of state-of-the-art deep learning models, Applied Sciences 14 (2024) 744. find
  4. S. Cavagnoli, F. Dragotto, et al., Sessismo, Mondadori Education, 2021. find
  5. M. Sanguinetti, F. Poletto, C. Bosco, V. Patti, M. A. Stranisci, An italian twitter corpus of hate speech against immigrants, in: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), 2018.
  6. Y. Zeng, et al., How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms, 2024. URL: https://arxiv.org/abs/2401.06373. arXiv:2401.06373 . ↗
  7. Y. Xian, B. Schiele, Z. Akata, Zero-shot learning-the good, the bad and the ugly, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4582–4591. find
  8. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in neural information processing systems 33 (2020) 1877–1901.
  9. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al., Chain-of-thought prompting elicits reasoning in large language models, Advances in neural information processing systems 35 (2022) 24824–24837.
  10. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al., Retrieval-augmented generation for knowledge-intensive nlp tasks, Advances in neural information processing systems 33 (2020) 9459–9474.
  11. J. Howard, S. Ruder, Universal language model fine-tuning for text classification, arXiv preprint arXiv:1801.06146 (2018). find
  12. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach, 2019. URL: https://arxiv.org/abs/1907. 11692. arXiv:1907.11692 . ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sorokoletova-2026-diag-sapienza-gsi-detect-joint/