Elamur Research⌕
☰
home / papers / 2026 / Counterspeech Generation using Small Language Models

Counterspeech Generation using Small Language Models

Paper · 2026 · Abubakar Sadiq Muhammad, Simona Frenda and Gavin Abercrombie

Summary

This study asks whether compact language models can write useful counterspeech, meaning replies that challenge hateful posts without censoring them. Large models are costly, energy-hungry and often reachable only through cloud APIs, which raises privacy concerns. The authors pooled 11,651 hate speech–counterspeech pairs from CONAN, DialoConan, MultiConan and Conan-Kn and split them 8:1:1. They tested six SLMs between 135M and 3B parameters under four setups: zero-shot, one-shot and 20-example few-shot prompting, plus LoRA fine-tuning. Outputs were scored with BLEURT, BERTScore, novelty, diversity, Detoxify toxicity and a composite score. One-shot outputs also went through a crowdsourced human evaluation. One-shot prompting gave the best overall automatic results, and fine-tuning brought no benefit. Models in the 1–3B range, especially Gemma-2 2B and Llama-3.2 3B, produced mostly relevant and constructive replies. Models below 1B often produced incoherent text and sometimes agreed with the hateful input. The results suggest that cheap models that can run on a device may be enough for counterspeech, provided humans stay in the loop.

Key points

  • Models tested: Gemma-2 2B, Llama-3.2 (1B, 3B) and SmolLM (135M, 360M, 1.7B), all run quantised through Ollama.
  • Generation strategies: One-shot prompting had the highest composite score (3.293). Fine-tuning had the lowest (2.512) and the highest toxicity (0.098). Few-shot gave the most diverse outputs, and zero-shot the most novel ones.
  • Model ranking: Gemma-2 2B led on the automatic composite score (3.699). The Llama-3.2 models showed low diversity because of safety refusals, with refusal rates of 10.1% for 1B and 4.5% for 3B.
  • Human ratings: 81.1% of Llama-3.2 3B outputs were judged relevant and specific, and Gemma-2 2B had the largest share of empathic replies (43.7%). SmolLM 135M produced about 14% gibberish, and 36.7% of its replies were judged to support the hate speech.
  • Limitations: The models were quantised and fine-tuning used few epochs. Only one randomly chosen one-shot example was used. The human evaluation was a crowdsourced study, not a test in real online settings.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Heriot-Watt UniversityEdinburgh · United Kingdom15 works · 5,365 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Abubakar Sadiq Muhammad, Simona Frenda and Gavin Abercrombie. 2026. Counterspeech Generation using Small Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.52

@inproceedings{muhammad2026counterspeech,
  title = {{Counterspeech Generation using Small Language Models}},
  author = {Abubakar Sadiq Muhammad, Simona Frenda and Gavin Abercrombie},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.52},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.52}
}

References

52 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mana Ashida and Mamoru Komachi. 2022. Towards automatic generation of messages countering online hate speech and microaggressions. In Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), pages 11–23, Seattle, Washington (Hybrid). Association for Computational Linguistics. find
  2. Jaione Bengoetxea, Yi-Ling Chung, Marco Guerini, and Rodrigo Agerri. 2024. Basque and Spanish counter narrative generation: Data creation and evaluation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), page… find
  3. BigCode. 2026. Unsloth. https://unsloth.ai. ↗
  4. Helena Bonaldi, Yi-Ling Chung, Gavin Abercrombie, and Marco Guerini. 2024a. NLP for counterspeech against hate: A survey and how-to guide. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3480–3499, Mexico City, Mexico. Association for Computational Linguistics. find
  5. Helena Bonaldi, Greta Damo, Nicolás Benjamín Ocampo, Elena Cabrio, Serena Villata, and Marco Guerini. 2024b. Is safer better? the impact of guardrails on the argumentative strength of LLMs in hate speech countering. In Proceedings of the 2024 find
  6. Conference on Empirical Methods in Natural Language Processing, pages 3446–3463, Miami, Florida, USA. Association for Computational Linguistics. Helena Bonaldi, Sara Dellantonio, Serra Sinem Tekiroğlu, and Marco Guerini. 2022. Human-machine collaboration approaches to build a dialogue dataset for h… find
  7. Yi-Ling Chung, Gavin Abercrombie, Florence Enock, Jonathan Bright, and Verena Rieser. 2024. Understanding counterspeech for online harm mitigation. Northern European Journal of Language Technology, 10. find
  8. Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. 2019. CONAN -COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2819–…
  9. Yi-Ling Chung, Serra Sinem Tekiroğlu, and Marco Guerini. 2021. Towards knowledge-grounded counter narrative generation for hate speech. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 899–914, Online. Association for Computational Linguistics. find
  10. Brian Dean. 2025. Social network usage & growth statistics. find
  11. Fiona Dennehy. 2023. Almost 90% of young people exposed to harmful content on social media. https://www.turing.ac.uk/news/almost-90-youngpeople-exposed-harmful-content-social-media. [Accessed 22-10-2024]. ↗
  12. Tanvi Dinkar, Aiqi Jiang, Simona Frenda, Poppy Gerrard-Abbott, Nancie Gunson, Gavin Abercrombie, and Ioannis Konstas. 2026. Can NLP tackle hate speech in the real world? Stakeholder-informed feedback and survey on counterspeech. In Proceedings of LREC 2026. find
  13. Margherita Fanton, Helena Bonaldi, Serra Sinem Tekiroğlu, and Marco Guerini. 2021. Human-in-the-loop for data collection: a multi-target counter narrative dataset to fight online hate speech. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int…
  14. Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378–382. find
  15. Muhammad Usman Hadi, Rizwan Qureshi, Abbas Shah, Muhammad Irfan, Anas Zafar, Muhammad Bilal Shaikh, Naveed Akhtar, Jia Wu, Seyedali Mirjalili, and 1 others. 2023. A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints. find
  16. Laura Hanu and Unitary team. 2020. Detoxify. Github. https://github.com/unitaryai/detoxify. ↗
  17. Amey Hengle, Aswini Padhi, Sahajpreet Singh, Anil Bandhakavi, Md Shad Akhtar, and Tanmoy Chakraborty. 2024. Intent-conditioned and non-toxic counterspeech generation using multi-task instruction tuning with RLAIF. In Proceedings of the 2024 Conference of the North American Chapter of the Association… find
  18. Lingzi Hong, Pengcheng Luo, Eduardo Blanco, and Xiaoying Song. 2024. Outcome-constrained large language models for countering hate speech. Preprint, arXiv:2403.17146. find
  19. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.
  20. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, and 6 others. 2024. Minicpm: Unveiling the potential of small lan…
  21. Hugging Face. 2023. Environmental impact of training large language models. https://huggingface.co/ learn/llm-course/chapter1/4. Accessed: Jan 2026. ↗
  22. Hugging Face. 2024. Smollm - blazingly fast and remarkably powerful. https://huggingface.co/ blog/smollm. Accessed: 2024-11-13. ↗
  23. Zhenya Ji and Ming Jiang. 2026. A systematic review of electricity demand for large language models: evaluations, challenges, and solutions. Renewable and Sustainable Energy Reviews, 225:116159. find
  24. Shuyu Jiang, Wenyi Tang, Xingshu Chen, Rui Tanga, Haizhou Wang, and Wenxian Wang. 2023. Raucg: Retrieval-augmented unsupervised counter narrative generation for hate speech. Preprint, arXiv:2310.05650. — [arXiv:2310.05650](https://arxiv.org/abs/2310.05650) find
  25. Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D. Lane, and Mengwei Xu. 2024. Small language models: Survey, measurements, and insights. Preprint, arXiv:2409.15790.
  26. Jacob Mchangama and Natalie Alkiviadou. 2023. Reimagining the current regulatory framework to online hate speech: Why making way for alternative methods is paramount for free speech. In Counterspeech, pages 100–122. Routledge. find
  27. Angelina McMillan-Major, Emily M. Bender, and Batya Friedman. 2024. Data statements: From technical concept to community practice. ACM J. Responsib. Comput., 1(1). find
  28. Meta. 2024. Llama-3.2-1b. https://huggingface.co/meta-llama/Llama-3.2-1B. Accessed: 2024-11-13. ↗
  29. Pauline Möhle, Matthias Orlikowski, and Philipp Cimiano. 2023. Just collect, don’t filter: Noisy labels do not improve counterspeech collection for languages without annotated resources. In Proceedings of the 1st Workshop on CounterSpeech for Online Abuse (CS4OA), pages 44–61, Prague, Czechia. Assoc… find
  30. Jimin Mun, Cathy Buerger, Jenny T Liang, Joshua Garland, and Maarten Sap. 2024. Counterspeakers’ perspectives: Unveiling barriers and ai needs in the fight against online hate. find
  31. Chien Van Nguyen, Xuan Shen, Ryan Aponte, Yu Xia, Samyadeep Basu, Zhengmian Hu, Jian Chen, Mihir Parmar, Sasidhar Kunapuli, Joe Barrow, Junda Wu, Ashish Singh, Yu Wang, Jiuxiang Gu, Franck Dernoncourt, Nesreen K. Ahmed, Nedim Lipka, Ruiyi Zhang, Xiang Chen, and 9 others. 2024. A survey of small lang…
  32. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train…
  33. Kaike Ping, Anisha Kumar, Xiaohan Ding, and Eugenia Rho. 2024. Behind the counter: Exploring the motivations and barriers of online counterspeech writing. Preprint, arXiv:2403.17116. — [arXiv:2403.17116](https://arxiv.org/abs/2403.17116) find
  34. Natalie Pirks, Melissa Sharman, Sarah Dawkins, and Daniel Austin. 2025. Racism, rape and death threats: One weekend of social media abuse in football. Accessed: 2025-12-05. find
  35. Aashima Poudhar, Ioannis Konstas, and Gavin Abercrombie. 2024. A strategy labelled dataset of counterspeech. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 256–265, Mexico City, Mexico. Association for Computational Linguistics. find
  36. Matthew Renze. 2024. The effect of sampling temperature on problem solving in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7346–7356, Miami, Florida, USA. Association for Computational Linguistics.
  37. Gemma Team Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L’eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram’e, Johan Ferret, Peter Liu, Pouya Dehghani Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerom… find
  38. Daniel Russo, Helena Bonaldi, Yi-Ling Chung, Gavin Abercrombie, and Marco Guerini. 2025. NLP for counterspeech against hate and misinformation (CSHAM). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 5: Tutorial Abstracts), pages 9–10, Vienna, Austr… find
  39. Punyajoy Saha, Aalok Agrawal, Abhik Jana, Chris Biemann, and Animesh Mukherjee. 2024. On zero-shot counterspeech generation by LLMs. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12443–12454, Torino… find
  40. Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, Online. Association for Computational Linguistics.
  41. Serra Sinem Tekiroğlu, Helena Bonaldi, Margherita Fanton, and Marco Guerini. 2022a. Using pre-trained language models for producing counter narratives against hate speech: a comparative study. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3099–3114, Dublin, Ireland.… find
  42. Serra Sinem Tekiroğlu, Helena Bonaldi, Margherita Fanton, and Marco Guerini. 2022b. Using pre-trained language models for producing counter narratives against hate speech: a comparative study. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3099–3114, Dublin, Ireland.… find
  43. María Estrella Vallecillo Rodríguez, Maria Victoria Cantero Romero, Isabel Cabrera De Castro, Arturo Montejo Ráez, and María Teresa Martín Valdivia. 2024. CONAN-MT-SP: A Spanish corpus for counternarrative using GPT models. In Proceedings of the 2024 Joint International Conference on Computational L… find
  44. Chien Van Nguyen, Xuan Shen, Ryan Aponte, Yu Xia, Samyadeep Basu, Zhengmian Hu, Jian Chen, Mihir Parmar, Sasidhar Kunapuli, Joe Barrow, and 1 others. 2025. A survey on small language models. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing-Natural… find
  45. Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tzuhao Mo, Qiuhao Lu, Wanjing Wang, Rui Li, Junjie Xu, Xianfeng Tang, and 1 others. 2025. A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trust… find
  46. Haiyang Wang, Zhiliang Tian, Xin Song, Yue Zhang, Yuchen Pan, Hongkui Tu, Minlie Huang, and Bin Zhou. 2024. Intent-aware and hate-mitigating counterspeech generation via dual-discriminator guided LLMs. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language R… find
  47. Ke Wang and Xiaojun Wan. 2018. Sentigan: Generating sentimental texts via mixture adversarial networks. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 4446–4452. International Joint Conferences on Artificial Intelligence Organization. find
  48. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. find
  49. Yi Zheng, Björn Ross, and Walid Magdy. 2023. What makes good counterspeech? a comparison of generation approaches and evaluation metrics. In Proceedings of the 1st Workshop on CounterSpeech for Online Abuse (CS4OA), pages 62–71, Prague, Czechia. Association for Computational Linguistics. find
  50. Wanzheng Zhu and Suma Bhat. 2021. Generate, prune, select: A pipeline for counterspeech generation against online hate speech. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 134–149, Online. Association for Computational Linguistics. find
  51. Irune Zubiaga, Aitor Soroa, and Rodrigo Agerri. 2024a. Ixa at refutes 2024: Leveraging language models for counter narrative generation. In IberLEF (Working Notes). CEUR Workshop Proceedings. find
  52. Irune Zubiaga, Aitor Soroa, and Rodrigo Agerri. 2024b. A llm-based ranking method for the evaluation of automatic counter-narrative generation. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/muhammad-2026-counterspeech-generation-small-language-models/