Elamur Research⌕
☰
home / papers / 2026 / A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR

A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR

Paper · 2026 · Guilherme Silva, Pedro Silva, Matheus Peixoto et al.

Summary

The paper argues that treating offensive language detection in Brazilian Portuguese as a single yes/no decision ignores the layered structure of toxicity. It tests whether one jointly trained model can handle all three annotation layers of the HateBR corpus. HateBR contains 7,000 Instagram comments on Brazilian politicians, labeled for offensiveness, ordinal severity, and nine hate-target categories.

The authors use a hard-parameter-sharing design. A single BERTimbau Base encoder feeds its [CLS] vector into three linear heads: binary, 4-class severity, and 9-label targets. Training minimizes an unweighted sum of the three losses. This MTL-Unified model is compared against three separately fine-tuned single-task models.

Joint training improves the main offensiveness task and slightly helps severity prediction. It also produces no logically contradictory outputs, meaning no hate target is assigned to a comment judged non-offensive. However, target identification degrades noticeably, which the authors attribute to severe class imbalance and to binary-task gradients dominating the shared encoder.

The work matters for content moderation because one model is cheaper to deploy than several and gives coherent outputs. It also clarifies when a unified model is not the right choice.

Key points

  • Data split: the fixed, binary-stratified split is 4,480 / 1,120 / 1,400 (train/val/test). Results are averaged over two seeds (88, 89).
  • Binary task (STL vs. MTL): MCC rose from 0.8012 to 0.8232, accuracy from 89.96% to 91.11%, and positive-class F1 from 0.8965 to 0.9088. Run-to-run spread was also smaller (±0.0045 vs. ±0.0096 accuracy).
  • Severity task: gains were small, with Macro-F1 0.4634 → 0.4717 and Balanced Accuracy 0.4641 → 0.4720.
  • Hierarchical consistency: target inconsistency was 0%, and level inconsistency was only 0.14%, without any rule-based post-processing.
  • Negative transfer on targets:
    • Micro-F1 fell from 0.59 to 0.42, and subset accuracy from 0.94 to 0.90.
    • F1 for Partyism, the most frequent target, dropped from 0.72 to 0.51.
    • The authors propose GradNorm-style loss weighting and data augmentation as future fixes.
    • Code is released at github.com/silvagal/hatebr_multitask_transformer.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 972 works

Cite

Cite as Guilherme Silva, Pedro Silva, Matheus Peixoto, Gladston Moreira and Eduardo Luz. 2026. A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{silva2026multitask,
  title = {{A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR}},
  author = {Guilherme Silva, Pedro Silva, Matheus Peixoto, Gladston Moreira and Eduardo Luz},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.109/}
}

References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rich Caruana. 1997. Multitask learning. Machine learning, 28(1):41–75. find
  2. Wenliang Dai, Tiezheng Yu, Zihan Liu, and Pascale Fung. 2020. Kungfupanda at SemEval-2020 task 12: BERT-based multi-TaskLearning for offensive language detection. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2060–2066, Barcelona (online). International Committee for Comput… find
  3. Jose A Diaz-Garcia and Julio Amador Diaz Lopez. 2025. A survey on cutting-edge relation extraction techniques based on language models. Artificial Intelligence Review, 58(9):287. find
  4. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. Acm Computing Surveys (CSUR), 51(4):1–30. find
  5. Ankita Gandhi, Param Ahir, Kinjal Adhvaryu, Pooja Shah, Ritika Lohiya, Erik Cambria, Soujanya Poria, and Amir Hussain. 2024. Hate speech detection: A comprehensive review of recent works. Expert Systems, 41(8):e13562. find
  6. Thomas Mandl, Sandip Modha, Anand Kumar M, and Bharathi Raja Chakravarthi. 2020. Overview of the hasoc track at fire 2020: Hate speech and offensive language identification in tamil, malayalam, hindi, english and german. In Proceedings of the 12th annual meeting of the forum for information retrieva… find
  7. Amanda Oliveira, Thiago Cecote, Pedro Silva, Jadson Gertrudes, Vander Freitas, and Eduardo Luz. 2023. How good is chatgpt for detecting hate speech in portuguese? In Proceedings of the 14th Brazilian Symposium in Information and Human Language Technology, pages 103–112. find
  8. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 75–86, Minneapolis,…
  9. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 task 12: Multilingual offensive language identification in social media (OffensEval 2020). In Proceedings of the Fourteenth Worksho… find
  10. Yu Zhang and Qiang Yang. 2022. A survey on multi-task learning. IEEE transactions on knowledge and data engineering, 34(12):5586–5609. find
  11. Ji Ho Park and Pascale Fung. 2017. One-step and two-step classification for abusive language detection on Twitter. In Proceedings of the First Workshop on Abusive Language Online, pages 41–45, Vancouver, BC, Canada. Association for Computational Linguistics.
  12. Flor Miriam Plaza-del Arco, Sercan Halat, Sebastian Padó, and Roman Klinger. 2021. Multi-task learning with sentiment, emotion, and target detection to recognize hate speech and offensive language. arXiv preprint arXiv:2109.10255. — [arXiv:2109.10255](https://arxiv.org/abs/2109.10255) find
  13. Sebastian Ruder. 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098.
  14. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10.
  15. Fábio Souza, Rodrigo Nogueira, and Roberto Lotufo. 2020. Bertimbau: pretrained bert models for brazilian portuguese. In Brazilian conference on intelligent systems, pages 403–417. find
  16. Asa Cooper Stickland and Iain Murray. 2019. BERT and pals: Projected attention layers for efficient adaptation in multi-task learning. In Proceedings of the 36th International Conference on Machine Learning, pages 5986–5995. find
  17. Francielle Vargas, Isabelle Carvalho, Fabiana Rodrigues de Góes, Thiago Pardo, and Fabrício Benevenuto. 2022. HateBR: A large expert annotated corpus of Brazilian Instagram comments for offensive language and hate speech detection. In Proceedings of the Thirteenth Language Resources and Evaluation C… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/silva-2026-multitask-transformer-offensive-language-detection/