Elamur Research⌕
☰
home / papers / 2026 / Speech-Based Emotion Recognition and Classification Integrating a CNN and BiLSTM Network

Speech-Based Emotion Recognition and Classification Integrating a CNN and BiLSTM Network

Paper · 2026 · Fatima Uroosa, Asim Abbas, Muhammad Tayyab Zamir et al.

Summary

This paper addresses speech emotion recognition (SER). The task is hard because speech varies across speakers, and some emotions sound acoustically alike, such as neutral vs. calm or happy vs. surprise. The authors propose a hybrid architecture. Convolutional layers extract local spectral patterns from log Mel-frequency spectral coefficients (MFSC), and a BiLSTM layer then captures long-range temporal dependencies across the utterance. They train several deep baselines under one shared experimental setup for comparison. All experiments use the Toronto Emotional Speech Set (TESS), a public corpus of acted English speech labelled with seven emotion categories. The CNN–BiLSTM achieves 96.36% accuracy and beats every baseline. The authors present it as a lighter and less data-hungry alternative to transformer models, suitable for real-time or resource-constrained settings. The evaluation limits how far these claims reach, however. Only a single clean, acted dataset is used, so performance on spontaneous, noisy, or other-language speech has not been demonstrated.

Key points

  • Accuracy on TESS: CNN–BiLSTM reaches 96.36%. Baselines score lower: GRU 91.84%, BiLSTM 93.12% and CNN–GRU 94.67%. The paper also reports precision and F1, with CNN–BiLSTM ahead.
  • Recurrent vs. convolutional models: Recurrent models (LSTM, BiLSTM, GRU) outperform the CNN-only model. The authors take this as evidence that temporal dynamics are central to emotional speech.
  • Input features: The abstract names log MFSC as input. Other sections mention MFCC and log-Mel spectrograms, so the exact feature set is described inconsistently.
  • Comparison with prior work: A table places the result above earlier systems such as Dual-Sequence LSTM (73.3%), Deep C-RNN (75%) and other hybrids (about 79–92%). These systems were evaluated on different datasets, so the comparison is only indicative.
  • Stated limitations:
    • The data are acted and recorded in clean conditions.
    • Some classes are imbalanced, and acoustically overlapping emotions get confused.
    • The model is audio-only.
    • Future work targets spontaneous speech and multimodal input combining text and visual cues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BirminghamBirmingham · United Kingdom25 works · 62,564 citations🎓Instituto Politécnico NacionalMexico City · Mexico18 works · 4,307 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 972 works

Cite

Cite as Fatima Uroosa, Asim Abbas, Muhammad Tayyab Zamir and Grigori Sidorov. 2026. Speech-Based Emotion Recognition and Classification Integrating a CNN and BiLSTM Network. Proceedings of Computational Affective Science (CAS) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4tq9wz92ymg3

@inproceedings{uroosa2026speech,
  title = {{Speech-Based Emotion Recognition and Classification Integrating a CNN and BiLSTM Network}},
  author = {Fatima Uroosa, Asim Abbas, Muhammad Tayyab Zamir and Grigori Sidorov},
  booktitle = {Proceedings of Computational Affective Science (CAS) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4tq9wz92ymg3},
  url = {https://doi.org/10.63317/4tq9wz92ymg3}
}

References

27 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Omar Abdelfattah, George Gal, Gordon W. Roberts, Ishiang Shih, and Yi-Chi Shih. 2016. A top-down design methodology encompassing component variations due to wide-range operation in frequency synthesizer plls. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 24(6):2050–2061. find
  2. Mrunal Prakash Gavali and Abhishek Verma. 2025. Ensemble of large self-supervised transformers for improving speech emotion recognition. International Journal of Data Mining, Modelling and Management, 17(2):217–244. find
  3. Tolulope Olalekan Abiola, Tewodros Achamaleh Bizuneh, Fatima Uroosa, Nida Hafeez, Grigori Sidorov, Olga Kolesnikova, and Olumide Ebenezer Ojo. 2025. Cic-nlp at genai detection task 1: Advancing multilingual machine-generated text detection. In In Proceedings of the CIC-NLP Workshop on GenAI Detectio… find
  4. James J. Gross. 1998. The emerging field of emotion regulation: An integrative review. Review of General Psychology, 2(3):271–299. find
  5. Rashid Jahangir, Ying Wah Teh, Faiqa Hanif, and Ghulam Mujtaba. 2021. Deep learning approaches for speech emotion recognition: State of the art and research challenges. Multimedia Tools and Applications, 80(16):23745–23812. find
  6. Md Rayhan Ahmed, Salekul Islam, AKM Muza-hidul Islam, and Swakkhar Shatabda. 2023. An ensemble 1d-cnn-lstm-gru model with data augmentation for speech emotion recognition. Expert Systems with Applications, 218:119633. find
  7. Eric Jordan, Raphaël Terrisse, Valeria Lucarini, Motasem Alrahabi, Marie-Odile Krebs, Julien Desclés, and Christophe Lemey. 2025. Speech emotion recognition in mental health: Systematic review of voice-based applications. JMIR Mental Health, 12(1):e74260. find
  8. Waleed Alsabhan. 2023. Human–computer interaction with real-time speech emotion recognition using ensemble techniques. Sensors, 23(3):1386. find
  9. Bagus Tris Atmaja and Akira Sasou. 2022. Sentiment analysis and emotion recognition from speech using universal speech representations. Sensors, 22(17):6369. find
  10. Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark D. Plumbley. 2020. Panns: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28:2880–2894. find
  11. Borealis (TESS) Dataset U. Kumaran, S. Radha Rammohan, S. M. Nagarajan, and A. Prathik. 2021. Fusion of mel and gammatone frequency cepstral coefficients for speech emotion recognition using deep crnn. International Journal of Speech Technology, 24(2):303–314. find
  12. Maria Vaida and Ziyuan Huang. 2025. Multimodal graph neural networks in healthcare: A review of fusion strategies across biomedical domains. Frontiers in Artificial Intelligence, 8:1716706. find
  13. Jianyou Wang, Michael Xue, Ryan Culhane, Enmao Diao, Jie Ding, and Vahid Tarokh. 2020. Speech emotion recognition with dual-sequence lstm architecture. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6474– 6478. find
  14. Harshawardhan S. Kumbhar and Sheetal U. Bhandari. 2019. Speech emotion recognition using mfcc features and lstm network. In In Proceedings of ICCUBEA, pages 1–3. find
  15. Jennifer S. Lerner, Ye Li, Piercarlo Valdesolo, and Karim S. Kassam. 2015. Emotion and decision making. Annual Review of Psychology, 66:799– 823. find
  16. World Health Organization. 2025. Over a billion people living with mental health conditions – services require urgent scale-up. Accessed: 2026-03-25. find
  17. Yanlin Liu, Aibin Chen, Guoxiong Zhou, Jizheng Yi, Jin Xiang, and Yaru Wang. 2024. Combined cnn-lstm with attention for speech emotion recognition based on feature-level fusion. Multimedia Tools and Applications, 83(21):59839–59859. find
  18. Cheng Xiefeng, Yue Wang, Shicheng Dai, Pengjun Zhao, and Qifa Liu. 2019. Heart sound signals can be used for emotion recognition. Scientific Reports, 9(1):6486. find
  19. Cristina Luna-Jiménez, David Griol, Zoraida Callejas, Ricardo Kleinlein, Juan M. Montero, and Fernando Fernández-Martínez. 2021. Multimodal emotion recognition on ravdess dataset using transfer learning. Sensors, 21(22):7665. find
  20. Wisha Zehra, Abdul Rehman Javed, Zunera Jalil, Habib Ullah Khan, and Thippa Reddy Gadekallu. 2021. Cross corpus multi-lingual speech emotion recognition using ensemble learning. Complex & Intelligent Systems, 7(4):1845–1854. find
  21. Syeda Tamanna Alam Monisha and Sadia Sultana. 2022. A review of the advancement in speech emotion recognition for indo-aryan and dravidian languages. Advances in Human-Computer Interaction, 2022:9602429. find
  22. H. Zhang, H. Huang, and H. Han. 2021. A novel heterogeneous parallel convolution bi-lstm for speech emotion recognition. Applied Sciences, 11(21):9897. find
  23. A. Muppidi and M. Radfar. 2021. Speech emotion recognition using quaternion convolutional neural networks. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6309–6313. find
  24. M. Kathleen Pichora-Fuller and Kate Dupuis. 2020. Toronto emotional speech set (tess). find
  25. Y. Lakshmi Prasanna, Y. Tarakaram, Y. Mounika, Suja Palaniswamy, and Susmitha Vekkot. 2022. Comparative deep network analysis of speech emotion recognition models using data augmentation. In Proceedings of the International Conference on Disruptive Technologies for Multi-Disciplinary Research and Ap… find
  26. N. Senthilkumar, S. Karpakam, M. G. Devi, R. Bal-akumaresan, and P. Dhilipkumar. 2022. Speech emotion recognition based on bidirectional lstm architecture and deep belief networks. Materials Today: Proceedings, 57:2180–2184. find
  27. Youddha Beer Singh and Shivani Goel. 2022. A systematic literature review of speech emotion recognition approaches. Neurocomputing, 492:245– 263. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/uroosa-2026-speech-based-emotion-recognition-classification/