Elamur Research⌕
☰
home / papers / 2026 / Wave2Word@DravidianLangTech 2026: WhisTam: A unified framework for dialect based Tamil speech recognition and classification

Wave2Word@DravidianLangTech 2026: WhisTam: A unified framework for dialect based Tamil speech recognition and classification

Paper · 2026 · Ruwad Naswan and Shadab Tanjeed Ahmad

Summary

Speech recognition for Tamil is hindered by strong regional differences in pronunciation, vocabulary and syntax, and by the small amount of dialect-annotated audio. As an entry to the DravidianLangTech@ACL 2026 shared task, the authors present WhisTam, one system that both transcribes dialectal Tamil and assigns each recording to one of four regions: Northern, Southern, Western or Central. It starts from vasista22/whisper-tamil-medium and adds a Regional Adapter. During training, a learned dialect embedding is projected, passed through LayerNorm and added to every encoder frame. At inference this embedding is removed. A separate head pools the encoder states with attention and predicts the dialect. The transcription and classification losses are optimized together. Training used the 9.22-hour Tamil Dialect Speech Dataset with extensive augmentation. On the hidden test set the system reached macro F1 0.53 and WER 0.55. These scores are far below the validation results, which shows how hard it is to generalize across Tamil dialects. The work also shows that multi-task adaptation can be added to a pretrained model at low cost for low-resource speech.

Key points

  • Base model and parameters: WhisTam builds on vasista22/whisper-tamil-medium (Whisper medium) and adds only a small number of parameters. The training loss is L_ASR + α·L_dialect.
  • Data: The Tamil Dialect Speech Dataset contains 9.22 h of training audio and 2.05 h of test audio at 16 kHz, with both spontaneous and read speech. The training data was split 90/10 into train and validation sets using stratified sampling.
  • Validation results: WER averaged 37.17% across dialects, ranging from 30.23% for Central to 41.68% for Western. Dialect macro F1 was 0.97.
  • Hidden test results: WER was 55.0%, ranking 3rd in the ASR subtask. Macro F1 was 0.53, ranking 2nd in the dialect classification subtask.
  • Embeddings and code: t-SNE plots show clear dialect clusters after training, whereas the pretrained encoder shows none. The code is published at github.com/rwd51/DravidianLangTech-Wave2Word.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Islamic University of TechnologyDhaka · Bangladesh10 works · 555 citations🎓Bangladesh University of Engineering and TechnologyDhaka · Bangladesh7 works · 308 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSignal Processing · 493 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 519 works

Cite

Cite as Ruwad Naswan and Shadab Tanjeed Ahmad. 2026. Wave2Word@DravidianLangTech 2026: WhisTam: A unified framework for dialect based Tamil speech recognition and classification. Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.dravidianlangtech-1.70

@inproceedings{naswan2026wave2word,
  title = {{Wave2Word@DravidianLangTech 2026: WhisTam: A unified framework for dialect based Tamil speech recognition and classification}},
  author = {Ruwad Naswan and Shadab Tanjeed Ahmad},
  booktitle = {Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages},
  year = {2026},
  doi = {10.18653/v1/2026.dravidianlangtech-1.70},
  url = {https://doi.org/10.18653/v1/2026.dravidianlangtech-1.70}
}

References

17 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zrar Khald Abdul and Abdulbasit K. Al-Talabani. 2022. Mel frequency cepstral coefficient and its applications: A review. IEEE Access, 10:122136–122158. find
  2. Muzaffar Ahmad Dar and Jagalingam Pushparaj. 2025. Machine learning and deep learning approaches for accent recognition: A review. IEEE Access, 13:51527–51550. find
  3. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, pages 12449–12460. Curran Associates, Inc.
  4. Laurent Besacier, Etienne Barnard, Alexey Karpov, and Tanja Schultz. 2014. Automatic speech recognition for under-resourced languages: A survey. Speech Communication, 56:85–100. find
  5. B. Bharathi, Bharathi Raja Chakravarthi, Shunmuga Priya Muthusamy Chinnan, S. Saranya, and S. Suhasini. 2026. Findings in tamil dialect speech recognition and classification. In Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages. Association for Co… find
  6. B. Bharathi, S. Saranya, P. Vijayalakshmi, and T. Nagarajan. 2025. Multi-dialect speech corpus creation for enhancing tamil automatic speech recognition. Circuits, Systems, and Signal Processing, pages 1– 19. find
  7. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2020. Unsupervised cross-lingual representation learning for speech recognition. Preprint, arXiv:2006.13979. find
  8. Shashidhar G. Koolagudi, Deepika Rastogi, and K. S. Sekhara Rao. 2012. Identification of language using mel-frequency cepstral coefficients (mfcc). Procedia Engineering, 38:3391–3398. find
  9. A. Madhavaraj and Ramakrishnan Angarai Ganesan. 2022. Data and knowledge-driven approaches for multilingual training to improve the performance of speech recognition systems of indian languages. Preprint, arXiv:2201.09494. — [arXiv:2201.09494](https://arxiv.org/abs/2201.09494) find
  10. M. Nanmalar, S. Johanan Joysingh, P. Vijayalakshmi, and T. Nagarajan. 2024a. A feature engineering approach for literary and colloquial tamil speech classification using 1d-cnn. Speech Commun., 173:103254. find
  11. M. Nanmalar, S. Johanan Joysingh, P. Vijayalakshmi, and T. Nagarajan. 2024b. A feature engineering approach for literary and colloquial tamil speech classification using 1d-cnn. Speech Commun., 173:103254. find
  12. M. Nanmalar, P. Vijayalakshmi, and T. Nagarajan. 2019. Literary and colloquial dialect identification for tamil using acoustic features. find
  13. TENCON 2019 - 2019 IEEE Region 10 Conference (TENCON), pages 1303–1306. find
  14. M. Nanmalar, P. Vijayalakshmi, and T. Nagarajan. 2022. Literary and colloquial tamil dialect identification. Circuits, Systems, and Signal Processing, 41:4004 – 4027. find
  15. Vineel Pratap, Anuroop Sriram, Paden Tomasello, Awni Hannun, Vitaliy Liptchinsky, Gabriel Synnaeve, and Ronan Collobert. 2020. Massively multilingual asr: 50 languages, 1 model, 1 billion parameters. Preprint, arXiv:2007.03001. — [arXiv:2007.03001](https://arxiv.org/abs/2007.03001) find
  16. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. Preprint, arXiv:2212.04356.
  17. Hem Das and Utpal Bhattacharjee. 2024. Assamese dialect identification using static and dynamic features from vowel. Journal of Advances in Information Technology, 15:306–321. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/naswan-2026-wave2word-dravidianlangtech-2026-whistam-unified/