Elamur Research⌕
☰
home / papers / 2026 / KOCOH: Korean Context-Dependent Hate Speech Dataset

KOCOH: Korean Context-Dependent Hate Speech Dataset

Summary

Hateful intent often lies not in a comment's wording but in what it responds to. A harmless-looking reply such as an approving remark can endorse discrimination when attached to a certain post. Most Korean hate speech corpora label isolated comments, so they cannot test this. The authors present KOCOH v.2, an expanded and revised version of their earlier Korean dataset. It pairs post summaries from Dcinside and FMkorea with comments.

The data follow a contrastive scheme with three pair types:

  • a real context with a hateful comment;
  • an invented context in which that same comment becomes benign;
  • the original context with a different, non-hateful comment.

Three expert annotators labeled every pair, and only unanimous judgments were kept. The authors then compared human judges with 11 recent large language models. Humans scored best overall, and GPT-5 led among the models. Models were much more conservative than humans: they rarely flagged benign comments but missed many hateful ones. Both humans and models were influenced by Honam-related vocabulary and sentiment polarity. The work provides a resource for evaluating context-aware hate detection and AI ethics in Korean.

Key points

  • KOCOH v.2 contains 3,000 context-comment pairs, up from 2,005 in the first release (Park and Song, 2025), so 995 pairs were added. Some existing items were edited or removed, and new labels such as counter speech and profanity were introduced.
  • Sources:
    • Dcinside popular boards, posts from June 2–23, 2024;
    • FMkorea popular boards, posts from January 1–21, 2025, a newly added platform.
  • Annotation used strict three-way consensus: 3,000 of the 3,067 collected pairs were kept, a 97.82% retention rate. The dropped pairs were ambiguous.
  • Eleven LLMs were evaluated, among them GPT-5, Claude Sonnet 4 and Gemini 2.5 Flash. Humans outperformed all of them. Human recall and specificity were balanced, whereas the models' specificity was clearly higher than their recall.
  • The dataset is released at github.com/eparkatgithub/KOCOH.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5pnu2jn6awun

@inproceedings{park2026kocoh,
  title = {{KOCOH: Korean Context-Dependent Hate Speech Dataset}},
  author = {Park, Eunah and Song, Sanghoun},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5pnu2jn6awun},
  url = {https://doi.org/10.63317/5pnu2jn6awun}
}

References

34 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ashutosh Baheti, Maarten Sap, Alan Ritter, and Mark Riedl. 2021. Just say no: Analyzing the stance of neural dialogue generation in offensive contexts. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4846–4862, Online and Punta Cana, Dominican Republi… find
  2. Alexandra D’Arcy and Emily M Bender. 2023. Ethics in linguistics. Annual Review of Linguistics, 9(1):49–69. find
  3. Jeevithashree Divya Venkatesh, Aparajita Jaiswal, and Gaurav Nanda. 2024. Comparing human text classification performance and explainability with large language and machine learning models using eye-tracking. Scientific reports, 14(1):14295. find
  4. Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. 2021. Latent hatred: A benchmark for understanding implicit hate speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345–363, Onl… find our card
  5. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Comput. Surv., 51(4). find
  6. Lei Gao and Ruihong Huang. 2017. Detecting online hate speech using context aware models. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 260–266, Varna, Bulgaria. INCOMA Ltd. find
  7. Katharine Gelber. 2002. Speaking Back: The Free Speech Versus Hate Speech Debate. Discourse approaches to politics, society, and culture. J. Benjamins Publishing Company. find
  8. Sung Soo Hong. 2019. Law and policy on hate. Journal of Law, 30(2):191–228. find
  9. Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598, Berlin, Germany. Association for Computational Linguistics. find our card
  10. Duyoung Jeon, Junho Lee, and Cheongtag Kim. 2024. User guide for KOTE: Korean online that-gul emotions dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17254–17270, Torino, Italia. ELRA and IC… find
  11. Younghoon Jeong, Juhyun Oh, Jongwon Lee, Jaimeen Ahn, Jihyung Moon, Sungjoon Park, and Alice Oh. 2022. KOLD: Korean offensive language dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10818–10833, Abu Dhabi, United Arab Emirates. Association f… find
  12. Minju Kim, Heuiyeen Yeen, and Myoung-Wan Koo. 2024. Towards context-based violence detection: A Korean crime dialogue dataset. In Findings of the Association for Computational Linguistics: EACL 2024, pages 603–623, St. Julian’s, Malta. Association for Computational Linguistics. find
  13. Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Meeyoung Cha, Yejin Choi, Byoungpil Kim, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park, and Jung-Woo Ha. 2023a. SQuARe: A large-scale dataset of sensitive questions and acceptable responses created through human-machine collaboration.… find
  14. Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Gunhee Kim, and Jung-woo Ha. 2023b. KoSBI: A dataset for mitigating social bias risks towards safer large language model applications. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry… find
  15. Jean Lee, Taejun Lim, Heejun Lee, Bogeun Jo, Yangsok Kim, Heegeun Yoon, and Soyeon Caren Han. 2022. K-MHaS: A multi-label hate speech detection dataset in Korean online news comment. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3530–3538, Gyeongju, Republic… find
  16. Jihyung Moon, Won Ik Cho, and Junbum Lee. 2020. BEEP! Korean corpus of online news comments for toxic speech detection. In Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, pages 25–31, Online. Association for Computational Linguistics. find
  17. Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, page 145–153, Republic and Canton of Geneva, CHE. International World Wide Web Confere… find
  18. John T Nockleby. 2000. Hate speech. Encyclopedia of the American constitution, 3(2):1277–1279. find
  19. Eunah Park and Sanghoun Song. 2025. Kocoh: A dataset for detecting context-dependent hate speech. Korean Linguistics, 106:251–277. find
  20. In-seoung Park. 2022. Tribalism in the internet community reading memes and neologisms. Journal of Popular Narrative, 28(2):59–93. find
  21. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799–2804, Brussels, Belgium. Association for Computational Linguistics. find our card
  22. Vinodkumar Prabhakaran, Zeerak Waseem, Seyi Akiwowo, and Bertie Vidgen. 2020. Online abuse and human rights: WOAH satellite session at RightsCon 2020. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 1–6, Online. Association for Computational Linguistics. find
  23. Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference… find our card
  24. Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, and Edward Grefenstette. 2023. The goldilocks of pragmatic understanding: fine-tuning strategy matters for implicature resolution by llms. In Proceedings of the 37th International Conference on Neural Information Processing Syste… find
  25. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10, Valencia, Spain. Association for Computational Linguistics. find our card
  26. Jisu Shin, Hoyun Song, Huije Lee, Fitsum Gaim, and Jong Park. 2023. Generation of Korean offensive language by leveraging large language models via prompt design. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapt… find
  27. Mingi Shin, Hyojin Chin, Hyeonho Song, Yubin Choi, Junghoi Choi, and Meeyoung Cha. 2024. Context-aware offensive language detection in human-chatbot conversations. In 2024 IEEE International Conference on Big Data and Smart Computing (BigComp), pages 270–277. find
  28. Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, and Minlie Huang. 2022. On the safety of conversational models: Taxonomy, dataset, and benchmark. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3906–3923, Dublin, Irelan… find
  29. Jasmina Tacheva and Srividya Ramasubramanian. 2023. Ai empire: Unraveling the interlocking systems of oppression in generative ai’s global order. Big Data & Society, 10(2):20539517231219241. find
  30. Joshua Tint. 2025. Guardrails, not guidance: Understanding responses to LGBTQ+ language in large language models. In Proceedings of the Queer in AI Workshop, pages 6–16, Hybrid format (in-person and virtual). Association for Computational Linguistics. find
  31. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Language Online, pages 78–84, Vancouver, BC, Canada. Association for Computational Linguistics. find our card
  32. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, California. Association for Computational Linguistics. find our card
  33. Hye-seung Yang. 2022. Others who are called ‘hongeo’: A text mining analysis of hate comments towards jeolla province in the context of naver crime news. Locality & Communication, 26(2):36–70. find
  34. Xinchen Yu, Eduardo Blanco, and Lingzi Hong. 2022. Hate speech and counter speech detection: Conversational context does matter. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5918–5930, Seattle… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/park-2026-kocoh-korean-context-dependent-hate/