Elamur Research⌕
☰
home / papers / 2026 / Cross-Domain Evaluation of Transformer-Based Models for Punjabi Speech Emotion Recognition

Cross-Domain Evaluation of Transformer-Based Models for Punjabi Speech Emotion Recognition

Paper · 2026 · Fatima Tu Zahra, Kulsoom Asim, Sandesh Kumar et al.

Summary

The paper addresses speech emotion recognition (SER) for Punjabi, a widely spoken but low-resource language. Existing Punjabi resources are mostly private or studio-recorded. The authors build a new 893-utterance dataset covering four emotions: angry, happy, sad, and neutral. It mixes three sources: scripted recordings from ten Pakistani Punjabi volunteers in quiet and noisy settings, clips from Indian Punjabi films and TV, and neutral speech taken from the Shrutilipi corpus. They compare CNN-BiLSTM, ResNet-34, and Wav2Vec 2.0 under four setups:

  • E1: in-domain training and testing on the new data
  • E2: training on RASA, testing on the new data
  • E3: the reverse of E2
  • E4: training on both corpora combined

Results:

  • In-domain: accuracy is high. Wav2Vec 2.0 reaches about 92.5% in E1.
  • Combined training: ResNet-34 performs best in E4, at 96%.
  • Cross-corpus transfer: accuracy drops sharply, and neutral speech is especially hard to recognise.

The work matters because it provides a public benchmark for Punjabi SER. It also shows that models trained on clean studio data handle naturalistic speech poorly.

Key points

  • Curated dataset: 893 labelled utterances (sad 327, angry 241, neutral 165, happy 160), released on Kaggle. Audio is 16 kHz mono WAV, and neutral samples come exclusively from Shrutilipi (All India Radio news).
  • RASA Punjabi subset: 8,672 training and 962 test samples, studio-recorded TTS speech. Neutral is heavily overrepresented (6,039 training samples). It was capped at 2,000 for ResNet-34, while the other two models used class-weighted loss.
  • Best results: ResNet-34 reaches 96% test accuracy in E4, with perfect happy and sad scores. Wav2Vec 2.0 reaches 92.48% macro accuracy in E1.
  • Cross-domain baselines: E2 reaches only 37.7–41.3% and E3 only 25.0–34.0%, showing the models are brittle under domain shift.
  • Limitations: the E1 80/20 split is not speaker-independent. Dialect coverage is uneven, since all volunteers speak Pakistani Punjabi and all media clips are Indian Punjabi.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Habib UniversityKarachi · Pakistan5 works · 4 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksEvaluation & benchmarks · 1,220 worksSpeech & spoken language · 480 works

Cite

Cite as Fatima Tu Zahra, Kulsoom Asim, Sandesh Kumar and Abdul Samad. 2026. Cross-Domain Evaluation of Transformer-Based Models for Punjabi Speech Emotion Recognition. Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3ao5m9p6ni9x

@inproceedings{zahra2026cross,
  title = {{Cross-Domain Evaluation of Transformer-Based Models for Punjabi Speech Emotion Recognition}},
  author = {Fatima Tu Zahra, Kulsoom Asim, Sandesh Kumar and Abdul Samad},
  booktitle = {Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026)},
  year = {2026},
  doi = {10.63317/3ao5m9p6ni9x},
  url = {https://doi.org/10.63317/3ao5m9p6ni9x}
}

References

7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aggarwal, S., Singh, P. & Sharma, K. (2023). A survey on speech emotion recognition: Datasets, features, and classification methods. Journal of Intelligent Systems, 33(1), pp. 203– 220. [12] Varadhan, P. S., Sankar, A., Raju, G. & Khapra, M. M. (2024). Rasa: Building Expressive Speech Synthesis Syst… find
  2. Baevski, A., Zhou, H., Mohamed, A. & Auli, M. (2020). wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. In Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS 2020). [13] Ververidis, D. & Kotropoulos, C. (2006). Emotional…
  3. Barhoumi, C. & Ben Ayed, Y. (2023). Real-Time Speech Emotion Recognition Using Deep Learning and Data Augmentation. Research Square [Preprint]. doi: 10.21203/rs.3.rs-2874039/v1.12. — [doi:10.21203/rs.3.rs-2874039/v1.12](https://doi.org/10.21203/rs.3.rs-2874039/v1.12) find
  4. Bhogale, K. S., Jain, N., Raman, A., Agarwal, S., Chhimwal, A., Bansal, D. & Jyothi, P. (2022). Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages. arXiv preprint arXiv:2208.12666. [1] Singla, C. & Singh, S. (2022). PEMO: A New Validate… — [arXiv:2208.12666](https://arxiv.org/abs/2208.12666) find
  5. El Ayadi, M., Kamel, M. S. & Karray, F. (2011). Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition, 44(3), pp. 572–587. find
  6. Gourisaria, M. K., Agrawal, R., Sahni, M. & Singh, P. K. (2024). Comparative analysis of audio classification with MFCC and STFT features using machine learning techniques. Discover Internet of Things, 4(1). find
  7. Kaur, K. & Singh, P. (2022). Impact of Feature Extraction and Feature Selection Algorithms on Punjabi Speech Emotion Recognition Using Convolutional Neural Network. ACM Transactions on Asian and Low-Resource Language Information Processing, 21(5), Article 100. Language Resource References find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zahra-2026-cross-domain-evaluation-transformer-based/