Elamur Research⌕
☰
home / papers / 2026 / Anatomy of Data Repositories for the Analysis and Detection of Toxicity in Portuguese

Anatomy of Data Repositories for the Analysis and Detection of Toxicity in Portuguese

Paper · 2026 · Lorena Souza Moreira, Paula Teresa M. Gibrim, Leonardo Rocha et al.

Summary

This paper asks why toxicity classifiers trained on one Brazilian Portuguese corpus often fail on others. The authors gathered six public datasets: HateBRXplain, OffComBR, OLID-BR, Fortuna, ToLD-BR and TuPy. They come from Twitter, Instagram, YouTube and news portals, and their labels for hate speech, offensiveness or toxicity were mapped to a single binary "toxic" label. Each corpus was profiled with the multilingual HurtLex lexicon, comparing category distributions and vocabulary overlap between toxic and non-toxic messages. Every train/test pairing was then evaluated using BERTimbau embeddings as features for an XGBoost classifier.

At a coarse level the corpora look alike, with toxicity clustering around moral defects and negative stereotypes. They diverge, however, in how individual words are used and annotated. Corpora with wide, varied toxic vocabularies transfer best. Corpora whose labels hinge on profanity transfer poorly, even when their vocabulary overlaps heavily with others.

This matters because collection and annotation choices clearly drive how robust a model is. Dataset selection and curation for Portuguese should therefore look beyond word lists.

Key points

  • The models trained on ToLD-BR generalized best across datasets, reaching a Macro-F1 of about 0.78 on HateBR(Xplain). The authors attribute this to ToLD-BR acting like a lexical superset of the other corpora.
  • OLID-BR is lexically close to ToLD-BR, with Jaccard ≈55% and HurtLex-category cosine ≈0.92. Yet it transfers badly, scoring a Macro-F1 of about 0.26 on TuPy, because it labels almost every profanity as toxic and so over-flags colloquial slang.
  • The toxic vocabulary differs by corpus:
    • In ToLD-BR and OLID-BR it is dominated by explicit swear words.
    • In HateBR it consists mostly of neutral political entities (e.g., "Lula", "Brasil"), so toxicity there depends on context and world knowledge.
  • The highest in-domain score was for HateBRXplain, at a Macro-F1 of about 0.87. Most cross-domain scores were roughly 0.24–0.69.
  • The authors note that lexicon-based profiling handles polysemy, irony and sarcasm poorly. Their planned future work includes:
    • comparing labeling schemes across corpora,
    • extending lexicons with MOL,
    • building collections and models that are more aware of domain and culture.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Bias, fairness & safety · 587 works

Cite

Cite as Lorena Souza Moreira, Paula Teresa M. Gibrim, Leonardo Rocha and Julio C. S. Reis. 2026. Anatomy of Data Repositories for the Analysis and Detection of Toxicity in Portuguese. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{moreira2026anatomy,
  title = {{Anatomy of Data Repositories for the Analysis and Detection of Toxicity in Portuguese}},
  author = {Lorena Souza Moreira, Paula Teresa M. Gibrim, Leonardo Rocha and Julio C. S. Reis},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.45/}
}

References

41 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lora Aroyo, Lucas Dixon, Nithum Thain, Olivia Redfield, and Rachel Rosen. 2019. Crowdsourcing subjective tasks: The case study of understanding toxicity in online discussions. In Companion Proceedings of the 2019 World Wide Web Conference, pages 1100–1105, Stroudsburg, PA, USA. Association for Compu… find
  2. Gabriel Assis, Annie Amorim, Jonnathan Carvalho, Daniel de Oliveira, Daniela Vianna, and Aline Paes. 2024. Exploring Portuguese hate speech detection in low-resource settings: Lightly tuning encoder models or in-context learning of large models? In Proceedings of the 16th International Conference on… find
  3. Elisa Bassignana, Valerio Basile, and Viviana Patti. 2018. HurtLex: A multilingual lexicon of words to hurt. In CEUR Workshop Proceedings, volume 2253, pages 1–6, Turin, Italy.
  4. Danah M. Boyd and Nicole B. Ellison. 2007. Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1):210–230. find
  5. Raluca Buturoiu and Nicoleta Corbu. 2026. Online hate speech. In Zizi Papacharissi, editor, The Routledge Companion to Digital Media and Democracy. Routledge, London, UK. find
  6. Sergio Andrés Castaño-Pulgarín, Natalia Suárez-Betancur, Luz Magnolia Tilano Vega, and Harvey Mauricio Herrera López. 2021. Internet, social media and online hate speech: Systematic review. Aggression and Violent Behavior, 58:101608. find
  7. Dave Chaffey. 2025. Global social media statistics research summary 2025 [Feb 2025 update]. Accessed: July 28, 2025. find
  8. Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You can’t stay here: The efficacy of Reddit’s 2015 ban examined through hate speech. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW):1–22. find
  9. Paula Fortuna, Juan Soler, and Leo Wanner. 2020. Toxic, hateful, offensive or abusive? What are we really classifying? An empirical analysis of hate speech datasets. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 6786–6794, Marseille, France. European Language Resourc…
  10. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, San Francisco, CA, USA. Association for Computing Machinery. find
  11. Iginio Gagliardone, Danit Gal, Thiago Alves, and Gabriela Martinez. 2015. Countering online hate speech. UNESCO Publishing, Paris. find
  12. Lu Cheng, Ahmadreza Mosallanezhad, Yasin N. Silva, Deborah L. Hall, and Huan Liu. 2022. Bias mitigation for toxicity detection via sequential decisions. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1750–1760, Madrid, Spain.… find
  13. Patricia Chiril, Endang Wahyu Pamungkas, Farah Benamara, Véronique Moriceau, and Viviana Patti. 2022. Emotionally informed hate speech detection: A multi-target perspective. Cognitive Computation, 14(1):322–352. find
  14. Gabriela Nunes Pinto Da Silva, Thiago Henrique Costa Silva, and João Da Cruz Gonçalves Neto. 2021. Liberdade de expressão e seus limites: Uma análise dos discursos de ódio na era das fake news. Revista Argumenta, 34:415–437. find
  15. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the International AAAI Conference on Web and Social Media, volume 11, pages 512–515, Montreal, Canada. AAAI Press.
  16. Felipe Ramos de Oliveira, Victoria Dias Reis, and Nelson Francisco Favilla Ebecken. 2024. Detecting hate speech on Brazilian social media: New dataset and analysis. In Proceedings of the XLV Ibero-Latin American Congress on Computational Methods in Engineering, Maceió, Brazil. ABMEC. find
  17. Rogers P. de Pelle and Viviane P. Moreira. 2017. Offensive comments in the Brazilian web: A dataset and baseline results. In Proceedings of the 6th Brazilian Workshop on Social Network Analysis and Mining, São Paulo, Brazil. SBC. find
  18. Paula Fortuna, Joao Rocha da Silva, Leo Wanner, and Sérgio Nunes. 2019. A hierarchically-labeled Portuguese hate speech dataset. In Proceedings of the Third Workshop on Abusive Language Online, pages 94–104, Florence, Italy. Association for Computational Linguistics.
  19. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Computing Surveys, 51(4):85:1–85:30. find
  20. Samuel Guimarães, Gabriel Kakizaki, Philipe Melo, Márcio Silva, Fabricio Murai, Julio C. S. Reis, and Fabrício Benevenuto. 2023. Anatomy of hate speech datasets: Composition analysis and cross-dataset classification. In Proceedings of the 34th ACM Conference on Hypertext and Social Media, pages 33:1… find
  21. Bing He, Caleb Ziems, Sandeep Soni, Naren Ramakrishnan, Diyi Yang, and Srijan Kumar. 2021. Racism is a virus: Anti-Asian hate and counterspeech in social media during the COVID-19 crisis. In Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining,… find
  22. Fatih Karabiber. 2025. Jaccard similarity. Accessed: August 11, 2025. find
  23. Kepios. 2025. Global social media statistics. Accessed: July 28, 2025. find
  24. Anna Koufakou, Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2020. HurtBERT: Incorporating lexical features with BERT for the detection of abusive language. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 34–43, Online. Association for Computational Linguistics. find
  25. Alfirna Rizqi Lahitani, Adhistya Erna Permanasari, and Noor Akhmad Setiawan. 2016. Cosine similarity to determine similarity measure: Study case in online essay assessment. In Proceedings of the 4th International Conference on Cyber and IT Service Management, pages 1–6, Banda Aceh, Indonesia. IEEE. find
  26. João Augusto Leite, Diego Silva, Kalina Bontcheva, and Carolina Scarton. 2020. Toxic language detection in social media for Brazilian Portuguese: New dataset and multilingual analysis. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics a… find
  27. Lucas Lima, Julio C. S. Reis, Philipe Melo, Fabricio Murai, Leandro Araujo, Pantelis Vikatos, and Fabricio Benevenuto. 2018. Inside the right-leaning echo chambers: Characterizing Gab, an unmoderated social system. In Proceedings of the IEEE/ACM International Conference on Advances in Social Network… find
  28. Lucas Lima, Julio C. S. Reis, Philipe Melo, Fabrício Murai, and Fabrício Benevenuto. 2020. Characterizing (un)moderated textual data in social systems. In Proceedings of the 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pages 430–434, Virtual Event. IEEE… find
  29. Luiz Henrique Quevedo Lima, Adriana Silvina Pagano, and Ana Paula Couto da Silva. 2024. Toxic content detection in online social networks: A new dataset from brazilian reddit communities. In Proceedings of the 16th International Conference on Computational Processing of Portuguese, pages 472–482, Sa… find
  30. Nishant Vishwamitra, Ruijia Roger Hu, Feng Luo, Long Cheng, Matthew Costello, and Yin Yang. 2020. On analyzing COVID-19-related hate speech using BERT attention. In Proceedings of the 2020 19th IEEE International Conference on Machine Learning and Applications, pages 669–676, Virtual Event. IEEE. find
  31. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, CA, USA. Association for Computational Linguistics.
  32. Ariadna Matamoros-Fernández. 2017. Platformed racism: The mediation and circulation of an Australian race-based controversy on Twitter, Facebook and YouTube. Information, Communication & Society, 20(6):930–946. find
  33. Endang Wahyu Pamungkas and Viviana Patti. 2019. Cross-domain and cross-lingual abusive language detection: A hybrid approach with deep learning and a multilingual lexicon. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 363… find
  34. Savvas Zannettou, Barry Bradlyn, Emiliano De Cristofaro, Haewoon Kwak, Michael Sirivianos, Gianluca Stringini, and Jeremy Blackburn. 2018. What is Gab: A bastion of free speech or an alt-right echo chamber. In Companion Proceedings of the The Web Conference 2018, pages 1007–1014, Lyon, France. Assoc… find
  35. Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: A systematic review. Language Resources and Evaluation, 55(3):477–523. find
  36. Yael Ravin and Claudia Leacock. 2000. Polysemy: Theoretical and computational approaches. Oxford University Press, Oxford, UK. find
  37. Isadora Salles, Francielle Vargas, and Fabrício Benevenuto. 2025. HateBRXplain: A benchmark dataset with human-annotated rationales for explainable hate speech detection in Brazilian Portuguese. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6659–6669, Abu Dh… find
  38. Fábio Souza, Rodrigo Nogueira, and Roberto Lotufo. 2020. BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Proceedings of the 9th Brazilian Conference on Intelligent Systems, pages 403–417, Rio Grande, Brazil. Springer. find
  39. Douglas Trajano, Rafael H. Bordini, and Renata Vieira. 2024. OLID-BR: Offensive language identification dataset for Brazilian Portuguese. Language Resources and Evaluation, 58(4):1263–1289. find
  40. United Nations. What is hate speech? Accessed: July 28, 2025. find
  41. Francielle Vargas, Isabelle Carvalho, Thiago A. S. Pardo, and Fabrício Benevenuto. 2025. Context-aware and find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/moreira-2026-anatomy-data-repositories-analysis-detection/