Elamur Research⌕
☰
home / papers / 2025 / A Hybrid DeBERTa and Gated Broad Learning System for Cyberbullying Detection in English Text

A Hybrid DeBERTa and Gated Broad Learning System for Cyberbullying Detection in English Text

Paper · 2025 · Devesh Kumar

Summary

The paper addresses automatic detection of cyberbullying in English online text, where harmful intent is often hidden behind sarcasm, neutral-sounding claims or context. The author builds a two-part hybrid. First, a DeBERTa encoder is extended with Squeeze-and-Excitation blocks and enriched with VADER sentiment signals to produce features. A feature-selection step then filters these features, and a Gated Broad Learning System (GBLS) makes the final decision. GBLS is a wide, shallow learner with GRU/LSTM-like gates. The model is evaluated on four English benchmarks (HateXplain, SOSNet, Mendeley-I, Mendeley-II) and is reported to outperform earlier approaches on each. The work also adds an explanation layer with token attributions, LIME and confidence calibration. It runs ablations for every component and sorts the remaining errors into recurring types. The study matters for two reasons. It shows that a transformer used as a feature producer, paired with a cheap gated classifier, can be both strong and inspectable. It also gives an honest map of where lexicon-driven toxicity judgments still break down.

Key points

  • Reported accuracy for ModifiedDeBERTa + GBLS is 79.3% on HateXplain, 95.41% on SOSNet, 91.37% on Mendeley-I and 94.67% on Mendeley-II.
  • In the ablation, the Squeeze-and-Excitation block adds 0.11 percentage points of accuracy, VADER sentiment adds 0.23 and feature selection adds 0.25.
  • Mutual Information feature selection slightly outperformed L1 regularization (91.37% vs 91.24% accuracy).
  • Explainability combines token-level attribution, LIME local explanations and confidence calibration. Future work proposes temperature or Platt scaling to address overconfident wrong predictions.
  • Failure analysis identifies four error types: implicit bias phrased as objective fact, harsh but legitimate criticism, sarcasm built from positive words, and counter-speech misread as abuse.

Interesting ideas

  • I1. The four-way failure taxonomy shows that aggressive vocabulary alone misleads classifiers, both when a reply condemns bad behaviour and when someone strongly criticises an argument rather than a person.
  • I2. Sarcastic abuse often pairs a positive act (thanking, praising) with a harmful event, such as thanking someone for wrecking one's day, so surface sentiment points the wrong way.
  • I3. The pretrained encoder serves only as a feature source, while a lightweight gated classifier makes the decision. This keeps the final layer cheap to retrain and easier to inspect.
  • I4. Adding an explicit sentiment lexicon on top of contextual embeddings gave a measurable gain, which suggests that a symbolic affect prior still helps a strong encoder.
  • I5. The paper reports per-component accuracy deltas, compares alternative feature-selection methods, and plans calibration for confident errors, making each design choice auditable.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could check whether the negative predicate targets a person or a behaviour/argument, and whether the speech act is condemnation or attack. These four patterns would serve as ready-made hard test cases.
  • I2 (frames): the irony module could flag a clash between a gratitude or praise speech act and a harm-denoting verb frame in the same utterance, giving an explainable irony cue.
  • I3 (tree): gparse's contextual states could feed small gated heads for auxiliary tasks (e.g., clause-type or attachment-confidence), without retraining the full transformer parser.
  • I4 (world): valence values from compiled knowledge could be attached to entity states and events, so the world model can track who is harmed or pleased and explain why.
  • I5 (learn): evaluations could report the gain of each repair rule or witness parser separately and calibrate witness confidence (e.g., temperature scaling) before agreeing on silver data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Devesh Kumar. 2025. A Hybrid DeBERTa and Gated Broad Learning System for Cyberbullying Detection in English Text. arXiv:2506.16052.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2506.16052 · arXiv 2506.16052

@misc{kumar2025hybrid,
  title = {{A Hybrid DeBERTa and Gated Broad Learning System for Cyberbullying Detection in English Text}},
  author = {Devesh Kumar},
  year = {2025},
  doi = {10.48550/arxiv.2506.16052},
  eprint = {2506.16052},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2506.16052}
}

References

27 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. W. Patchin and S. Hinduja, ”Cyberbullying trends among teens: A nationally representative sample,” Cyberbullying Research Center, 2023. find
  2. R. M. Kowalski, G. W. Giumetti, A. N. Schroeder, and M. R. Lattanner, ”Bullying in the digital age: A critical review and meta-analysis of cyberbullying research among youth,” Psychological Bulletin, vol. 140, no. 4, pp. 1073–1137, 2014. find
  3. A. Schmidt and M. Wiegand, ”A survey on hate speech detection using natural language processing,” in Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, 2017, pp. 1–10.
  4. T. Davidson, D. Warmsley, M. Macy, and I. Weber, ”Automated hate speech detection and the problem of offensive language,” in Proceedings of the 11th International AAAI Conference on Web and Social Media, 2017, pp. 512–515.
  5. P. Badjatiya, S. Gupta, M. Gupta, and V. Varma, ”Deep learning for hate speech detection in tweets,” in Proceedings of the 26th International Conference on World Wide Web Companion, 2017, pp. 759–760. find
  6. S. Hochreiter and J. Schmidhuber, ”Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. find
  7. K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, ”Learning phrase representations using RNN encoder-decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 2014, pp. 1724–1734.
  8. T. Caselli, V. Basile, J. Mitrović, and M. Granitzer, ”HateBERT: Retraining BERT for abusive language detection in English,” in Proceedings of the 5th Workshop on Online Abuse and Harms, 2021, pp. 17–25.
  9. F. Barbieri, J. Camacho-Collados, L. Espinosa Anke, and L. Neves, ”TweetEval: Unified benchmark and comparative evaluation for tweet classification,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 1644–1650.
  10. Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, and Q. V. Le, ”XLNet: Generalized autoregressive pretraining for language understanding,” in Advances in Neural Information Processing Systems, 2019, pp. 5753–5763.
  11. P. He, X. Liu, J. Gao, and W. Chen, ”DeBERTa: Decoding-enhanced BERT with disentangled attention,” in Proceedings of the 9th International Conference on Learning Representations, 2021.
  12. X. Jiao, Y. Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu, ”TinyBERT: Distilling BERT for natural language understanding,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 4163–4174.
  13. V. Sanh, L. Debut, J. Chaumond, and T. Wolf, ”DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” in Proceedings of the 7th Workshop on Energy Efficient Machine Learning and Cognitive Computing, 2019. find
  14. C. J. Hutto and E. Gilbert, ”VADER: A parsimonious rule-based model for sentiment analysis of social media text,” in Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media, 2014, pp. 216–225. find
  15. J. Du, C.-M. Vong, and C. L. P. Chen, ”Novel efficient RNN and LSTM-like architectures: Recurrent and gated broad learning systems and their applications for text classification,” IEEE Transactions on Cybernetics, vol. 51, no. 3, pp. 1586–1597, 2020. find
  16. C. Van Hee, G. Jacobs, C. Emmery, B. Desmet, E. Lefever, B. Verhoeven, G. De Pauw, W. Daelemans, and V. Hoste, ”Automatic detection of cyberbullying in social media text,” PLoS One, vol. 13, no. 10, pp. e0203794, 2018. find
  17. H. Wang and J. Leskovec, ”Combining graph convolutional neural networks and label propagation,” ACM Transactions on Information Systems, vol. 40, no. 4, pp. 1–27, 2021. find
  18. O. Zaidan, J. Eisner, and C. Piatko, ”Using ’annotator rationales’ to improve machine learning for text categorization,” in Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, 2007, pp. 260–267.
  19. B. Mathew, P. Saha, S. M. Yimam, C. Biemann, P. Goyal, and A. Mukherjee, ”HateXplain: A benchmark dataset for explainable hate speech detection,” in Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021, pp. 14867–14875. find
  20. J. Wang, K. Fu, and C. T. Lu, ”SOSNet: A graph convolutional network approach to fine-grained cyberbullying detection,” in 2020 IEEE International Conference on Big Data, 2020, pp. 1699–1708. find
  21. N. Ejaz, S. Choudhury, and F. Razi, ”A comprehensive dataset for automated cyberbullying detection,” Mendeley Data V2, doi: 10.17632/wmx9jj2htd.2, 2024. ↗
  22. M. T. Ribeiro, S. Singh, and C. Guestrin, ”“Why should I trust you?”: Explaining the predictions of any classifier,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135–1144. — "Why Should I Trust You?": Explaining the Predictions of Any Classifier
  23. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, ”On calibration of modern neural networks,” in International Conference on Machine Learning, 2017, pp. 1321–1330. find
  24. M. Sundararajan, A. Taly, and Q. Yan, ”Axiomatic attribution for deep networks,” in International Conference on Machine Learning, 2017, pp. 3319–3328. find
  25. F. Barbieri, L. E. Anke, and J. Camacho-Collados, ”XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond,” in Proceedings of the Language Resources and Evaluation Conference, 2022, pp. 258–266.
  26. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, ”Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998. find
  27. D. Bahdanau, K. Cho, and Y. Bengio, ”Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kumar-2025-hybrid-deberta-gated-broad-learning/