Elamur Research⌕
☰
home / papers / 2026 / ByteBuilders@DravidianLangTech 2026: Transformer-Based Weighted Ensemble for Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments

ByteBuilders@DravidianLangTech 2026: Transformer-Based Weighted Ensemble for Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments

Paper · 2026 · Mitharshana T V, Shanthi S, Lavana V et al.

Summary

The authors, from Kongu Engineering College, address the DravidianLangTech 2026 shared task: labelling Tamil political posts from X (Twitter) with one of seven sentiment categories. The label set covers Positive, Negative and Neutral, along with Opinionated, Sarcastic, Substantiated and a catch-all None of the Above class. The posts are noisy, with informal spelling, code-mixing, dialect and sarcasm, and most examples fall into a few categories.

The system fine-tunes XLM-RoBERTa-Base and IndicBERT separately. Each feeds mean-pooled token embeddings into a linear classifier. Both are trained with stratified 10-fold cross-validation, which yields twenty models. Imbalance is handled in two ways: random oversampling of rare labels, and balanced class weights combined with focal loss. Fold logits are averaged, then the two model families are merged by a weighted sum tuned on development accuracy.

The ensemble reaches 36.8% accuracy and 34% weighted F1 on the development set, a small gain over either single model. The modest scores show that fine-grained political sentiment in low-resource Tamil remains hard. The work provides a reproducible baseline, with code released on GitHub.

Key points

  • Data: the training label counts are highly skewed. Opinionated has 1361 examples, Sarcastic 790, Neutral 637, Positive 575, Substantiated 412, Negative 406 and None of the Above 171. The development set has 544 items.
  • Development accuracy: XLM-RoBERTa scores 0.3438, IndicBERT 0.3456 and the weighted ensemble 0.3676. The ensemble weights are nearly equal (0.499 for XLM-RoBERTa, 0.501 for IndicBERT).
  • Training setup:
    • focal loss with γ = 2.0;
    • AdamW optimizer, learning rate 1.5×10⁻⁵, batch size 16;
    • maximum length of 256 tokens and dropout of 0.3;
    • up to 15 epochs, with early stopping after 3 epochs without improvement;
    • gradient clipping at 1.0 and linear warmup.
  • Per-class F1:
    • Highest for None of the Above at 0.76, but this rests on only 20 development examples.
    • Opinionated (0.47) and Sarcastic (0.45) are moderate.
    • Negative and Neutral are lowest at 0.14 each.
  • Errors: the confusion matrix shows Negative and Neutral posts are often predicted as Opinionated. This bias toward dominant classes persists despite the imbalance countermeasures. The system classifies single sentences and does not model discourse context.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 1,035 works

Cite

Cite as Mitharshana T V, Shanthi S, Lavana V and Kaviya Varma R. 2026. ByteBuilders@DravidianLangTech 2026: Transformer-Based Weighted Ensemble for Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments. Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.dravidianlangtech-1.20

@inproceedings{v2026bytebuilders,
  title = {{ByteBuilders@DravidianLangTech 2026: Transformer-Based Weighted Ensemble for Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments}},
  author = {Mitharshana T V, Shanthi S, Lavana V and Kaviya Varma R},
  booktitle = {Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages},
  year = {2026},
  doi = {10.18653/v1/2026.dravidianlangtech-1.20},
  url = {https://doi.org/10.18653/v1/2026.dravidianlangtech-1.20}
}

References

14 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. B. K. Behera, A. Kumar, and S. Pradhan. 2021. Sentiment analysis in indian languages: A survey. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(3). find
  2. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13:281–305. find
  3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, and et al. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8440–8451.
  4. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. In Advances in Neural Information Processing Systems (NeurIPS). find
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171– 4186.
  6. Thomas G. Dietterich. 2000. Ensemble methods in machine learning. In Multiple Classifier Systems, volume 1857 of Lecture Notes in Computer Science, pages 1–15. Springer. find
  7. Haibo He and Edwardo A. Garcia. 2009. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9):1263–1284. find
  8. Daniel Jurafsky and James H. Martin. 2023. Speech and Language Processing, 3 edition. Prentice Hall. Draft. find
  9. Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, and et al. 2020. Indicnlp suite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual models for indian languages. In Findings of EMNLP, pages 4948–4961. find
  10. Ron Kohavi. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI), pages 1137–1145. find
  11. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980– 2988.
  12. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30.
  13. M. Vegupatti, K. K. Ponnusamy, B. R. Chakravarthi, S. Rajiakodi, D. Thenmozhi, P. K. Kumaresan, and S. Thangasamy. 2026. find
  14. Tamilpolisent 2026: A shared task report on multiclass political sentiment analysis in tamil. In Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/v-2026-bytebuilders-dravidianlangtech-2026-transformer-based/