Elamur Research⌕
☰
home / papers / 2026 / HNK@DravidianLangTech 2026: Investigating Grapheme-Level Normalization for Abusive Tamil Text Classification

HNK@DravidianLangTech 2026: Investigating Grapheme-Level Normalization for Abusive Tamil Text Classification

Paper · 2026 · Hanish Vigneshwar R, Nahul Alaguraj, Karthikeyan Manimaran et al.

Summary

This paper addresses automatic detection of abusive Tamil social media comments aimed at women. It treats the problem as a binary Abusive vs. Non-Abusive task within the DravidianLangTech 2026 shared task.

The authors fine-tune three pretrained transformers, IndicBERT, MuRIL and TamilBERT, with identical hyperparameters. They compare two preprocessing pipelines:

  • Baseline: URL removal plus whitespace and Unicode normalization.
  • Grapheme-aware: text is segmented into graphemes before subword tokenization, so Tamil characters built from several Unicode code points are not split inconsistently.

Across the six resulting configurations, TamilBERT with grapheme-aware normalization performed best on both validation and test data. The two multilingual models barely changed between pipelines. The authors attribute TamilBERT's gain to its Tamil-only pretraining and vocabulary. The selected system ranked third in the shared task.

The study matters because a cheap, linguistically motivated preprocessing step noticeably improved a monolingual model without any architectural change. Error analysis shows that implicit abuse and Tamil–English code-mixing remain the main difficulties.

Key points

  • Data: shared-task corpus (Sivagnanam et al., 2026).
    • Training split: 3,652 comments (1,883 Non-Abusive / 1,768 Abusive; these per-class figures add up to 3,651).
    • Test set: 913 labelled comments (472 / 441).
    • The text is informal and contains slang, spelling variants and English code-mixing.
  • Training setup: learning rate 2×10⁻⁵, batch size 16, 5 epochs, AdamW, cross-entropy loss. No augmentation, class weighting or ensembling was used. Models were selected by validation F1.
  • Validation F1 (baseline → grapheme-aware):
    • TamilBERT: 0.7941 → 0.8476
    • IndicBERT: 0.8201 → 0.8156
    • MuRIL: 0.7958 → 0.7956
  • Test accuracy (baseline → grapheme-aware):
    • TamilBERT: 0.8018 → 0.8105 (best overall)
    • IndicBERT: 0.7919 → 0.8039
    • MuRIL: 0.7766 → 0.7777
  • Code and limitations: the code is released at github.com/The-Silly-Glitch/GraphemeAware-Tamil-Abuse-Detection. The authors note several limitations: a small dataset, binary labels only, and just three models and two preprocessing strategies tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vellore Institute of Technology UniversityVellore · India7 works · 268 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Hanish Vigneshwar R, Nahul Alaguraj, Karthikeyan Manimaran and Ratnavel Rajalakshmi. 2026. HNK@DravidianLangTech 2026: Investigating Grapheme-Level Normalization for Abusive Tamil Text Classification. Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.dravidianlangtech-1.38

@inproceedings{r2026hnk,
  title = {{HNK@DravidianLangTech 2026: Investigating Grapheme-Level Normalization for Abusive Tamil Text Classification}},
  author = {Hanish Vigneshwar R, Nahul Alaguraj, Karthikeyan Manimaran and Ratnavel Rajalakshmi},
  booktitle = {Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages},
  year = {2026},
  doi = {10.18653/v1/2026.dravidianlangtech-1.38},
  url = {https://doi.org/10.18653/v1/2026.dravidianlangtech-1.38}
}

References

10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Vaswani Ashish. 2017. Attention is all you need. Advances in neural information processing systems, 30:I. the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages. Association for Computational Linguistics.
  2. Menan Velayuthan and Kengatharaiyer Sarveswaran. 2023. Egalitarian language representation in language models: It all begins with tokenizers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics.
  3. Tareque Md Hanif and Md Rashadur Rahman. 2025. CUET_Agile@DravidianLangTech 2025: Fine-tuning transformers for detecting abusive text targeting women from Tamil and Malayalam texts. In Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages, pages 315–3… find
  4. Khouloud Mnassri, Reza Farahbakhsh, Razieh Chalehchaleh, Praboda Rajapaksha, Amir Reza Jafari, Guanlin Li, and Noel Crespi. 2024. A survey on multi-lingual offensive language detection. PeerJ Computer Science, 10:e1934. find
  5. Marzieh Mozafari, Khouloud Mnassri, Reza Farahbakhsh, and Noel Crespi. 2024. Offensive language detection in low resource languages: A use case of persian language. PLoS one, 19(6):e0304166. find
  6. Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr, and Adel Bibi. 2023. Language model tokenizers introduce unfairness between languages. ArXiv, abs/2305.15425.
  7. Ratnavel Rajalakshmi, Ankita Duraphe, and Antonette Shibani. 2022. Dlrg@ dravidianlangtech-acl2022: Abusive comment detection in tamil using multilingual transformer models. In Proceedings of the second workshop on speech and language technologies for Dravidian languages, pages 207–213. find
  8. Ratnavel Rajalakshmi, Yashwant Reddy, and Lokesh Kumar. 2021. Dlrg@ dravidianlangtech-eacl2021: Transformer based approachfor offensive language identification on code-mixed tamil. In Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, pages 357–362. find
  9. Ratnavel Rajalakshmi, Srivarshan Selvaraj, Faerie Mat-tins, Pavitra Vasudevan, and Anand Kumar. 2023. Hottest: Hate and offensive content identification in tamil using transformers and enhanced stemming. Computer Speech and Language, 78:101464. find
  10. Bhuvaneswari Sivagnanam, Kathiravan Pannerselvam, Jananayagan V, Charmathi Rajkumar, Ramesh Kannan R, Ratnavel Rajalakshmi, Shunmuga Priya Muthusamy Chinnan, Saranya Rajiakodi, and Bharathi Raja Chakravarthi. 2026. From comments to harm: A findings report on abusive tamil text targeting women on soc… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/r-2026-hnk-dravidianlangtech-2026-investigating-grapheme/