Elamur Research⌕
☰
home / papers / 2026 / IIITK_SpeechScape@DravidianLangTech 2026: Dialect based speech recognition and classification using Speech Foundation Models and Deep Learning Techniques

IIITK_SpeechScape@DravidianLangTech 2026: Dialect based speech recognition and classification using Speech Foundation Models and Deep Learning Techniques

Paper · 2026 · G Srishtik Sekar, Harissh Ragav Dhamodaran, Kishore Shankar S et al.

Summary

This shared-task paper from IIIT Kottayam addresses two problems in Tamil speech processing: identifying which regional dialect an utterance belongs to, and transcribing dialectal speech into text. Tamil's dialects differ in pronunciation, vocabulary and intonation, which makes both tasks hard for a low-resource language. The authors used a dataset of about 9.4 hours of speech covering four dialects (Central, Southern, Northern, Western). They resampled the audio to 16 kHz mono and discarded recordings below 10 dB SNR.

For dialect classification, they compared a CLDNN baseline (CNN plus BiLSTM over log-Mel features) with three pretrained encoders: Whisper, WavLM and wav2vec. Each encoder used a learnable weighted pooling of its layers, followed by a projection and a feed-forward classifier. For ASR, they fine-tuned Whisper-v3-large with LoRA, optionally adding IndicVoices data, and ran IndicConformer zero-shot as a comparison. Whisper performed best on both tasks. This suggests that large supervised multilingual pretraining transfers well to Tamil dialects, although overall scores remain modest.

Key points

  • Dialect classification (Macro F1): Whisper scored 0.46 (accuracy 0.64), ahead of CLDNN (0.33), WavLM (0.30) and Wav2Vec2 (0.26).
  • ASR results:
    • Whisper fine-tuned on IndicVoices plus dialect data was best (WER 0.57, CER 0.17).
    • Zero-shot IndicConformer came second (WER 0.61, CER 0.20).
    • Whisper fine-tuned on dialect data alone had a worse WER than the untuned base model (0.79 vs 0.76), though its CER was lower (0.25 vs 0.31).
  • Dataset composition: Northern 3:29 h, Southern 2:44 h, Western 2:00 h, Central 1:08 h, for 9:22 h in total.
  • Per-dialect behavior: Whisper classified Southern speech best, Northern moderately, and Central and Western poorly. The authors attribute the weak classes to limited training data, although Northern is the largest class yet is not the best recognized.
  • Training setup and availability: all experiments ran on two NVIDIA T4 GPUs. The code is published on GitHub (GL3MON/IIITK-SpeechScape-DravidianLangTech2026).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Information Technology KottayamPalai · India2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSpeech & spoken language · 480 works

Cite

Cite as G Srishtik Sekar, Harissh Ragav Dhamodaran, Kishore Shankar S, Balasubramanian Palani and R Tharaniya Sairaj. 2026. IIITK_SpeechScape@DravidianLangTech 2026: Dialect based speech recognition and classification using Speech Foundation Models and Deep Learning Techniques. Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.dravidianlangtech-1.40

@inproceedings{sekar2026iiitk,
  title = {{IIITK\_SpeechScape@DravidianLangTech 2026: Dialect based speech recognition and classification using Speech Foundation Models and Deep Learning Techniques}},
  author = {G Srishtik Sekar, Harissh Ragav Dhamodaran, Kishore Shankar S, Balasubramanian Palani and R Tharaniya Sairaj},
  booktitle = {Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages},
  year = {2026},
  doi = {10.18653/v1/2026.dravidianlangtech-1.40},
  url = {https://doi.org/10.18653/v1/2026.dravidianlangtech-1.40}
}

References

13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. B. Bharathi, Bharathi Raja Chakravarthi, Shunmuga Priya Muthusamy Chinnan, S. Saranya, and S. Suhasini. 2026. Findings in Tamil Dialect Speech Recognition and Classification. In Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages. Association for Co… find
  2. B Bharathi, S Saranya, P Vijayalakshmi, and T Nagarajan. 2025. Multi-dialect speech corpus creation for enhancing tamil automatic speech recognition. Circuits, Systems, and Signal Processing, pages 1–19. find
  3. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Long Zhou, Yanmin Qian, and 1 others. 2021. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. arXiv preprint arXiv:2110.13900. find
  4. Bo Li, Tara N. Sainath, Khe Chai Sim, Michiel Bacchiani, Eugene Weinstein, Patrick Nguyen, Zhifeng Chen, Yanghui Wu, and Kanishka Rao. 2018. Multi-dialect speech recognition with a single sequence-to-sequence model. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal… find
  5. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 28492–2…
  6. Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre�training for speech recognition. arXiv preprint arXiv:1904.05862. find
  7. Pourdamghani Ghahremani and 1 others. 2015. Investigation of cnn�dnn and cnn�rnn architectures for speech recognition. In Proceedings of Interspeech 2015, pages 2019–2023. find
  8. Shweta Sinha, Aruna Jain, and S. S. Agrawal. 2015. Fusion of multi-stream speech features for dialect classification. CSI Transactions on ICT, 2(4):243–252. find
  9. Anmol Gulati, James Qin, Chung�Cheng Chiu, and 1 others. 2020. Conformer: Convolution�augmented transformer for speech recognition. In Proceedings of Interspeech 2020, pages 5036–5040.
  10. Camille Harris, Chijioke Mgbahurike, Neha Kumar, and Diyi Yang. 2024. Modeling gender and dialect bias in automatic speech recognition. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15166–15184, Miami, Florida, USA. Association for Computational Linguistics.
  11. Nikhil Jakhar, Sudhanshu Srivastava, and Arun Baby. 2024. A Unified Approach to Multilingual Automatic Speech Recognition with Improved Language Identification for Indic Languages. In Interspeech 2024, pages 3949–3953. find
  12. Tahir Javed, Janki Nawale, Eldho George, Sakshi Joshi, Kaushal Bhogale, Deovrat Mehendale, Ishvinder Sethi, Aparna Ananthanarayanan, Hafsah Faquih, Pratiti Palit, Sneha Ravishankar, Saranya Sukumaran, Tripura Panchagnula, Sunjay Murali, Kunal Gandhi, Ambujavalli R, Manickam M, C Vaijayanthi, Krishna… find
  13. Shashidhar G. Koolagudi, Akash Bharadwaj, Y. V. Srinivasa Murthy, Nishaanth Reddy, and Priya Rao. 2017. Dravidian language classification from speech signal using spectral and prosodic features. International Journal of Speech Technology, 20(4):1005–1016. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sekar-2026-iiitk-speechscape-dravidianlangtech-2026-dialect/