Elamur Research⌕
☰
home / papers / 2026 / Towards Robust Arabic Speech Emotion Recognition with Deep Learning

Towards Robust Arabic Speech Emotion Recognition with Deep Learning

Summary

Arabic speech emotion recognition (SER) lags behind work on English and other Indo-European languages. Labelled emotional speech is scarce, dialects differ widely, and a model has to catch both brief spectral cues and patterns that span a whole utterance. Earlier Arabic studies mostly tested a single model on a single corpus, which makes their results hard to compare. The authors build one shared pipeline, with the same preprocessing, features and metrics for every model, and use it to compare three paradigms. Two hybrids work on MFCC and spectrogram inputs: a convolutional-recurrent model (CNN-LSTM, called CNN-BiLSTM-Attention in the conclusion) and a CNN-Transformer. The third is wav2vec 2.0, fine-tuned on raw waveforms. Experiments use the EYASE and BAVED corpora. The CNN-Transformer performs best on both, reaching 98.1% accuracy on BAVED, and ablations show that removing either its convolutional or its attention part hurts performance. It also costs much less compute than wav2vec 2.0. This suggests that a mid-sized hybrid can be a more practical choice than a large self-supervised model for a low-resource, dialect-rich language.

Key points

  • CNN-Transformer results: 97.1% accuracy (96.9% F1) on EYASE and 98.1% accuracy (97.9% F1) on BAVED, with 11.2M parameters and 5.8 GB VRAM.
  • Prior Arabic results cited for context:
    • SVM/KNN on EYASE: about 64%.
    • CNN on log-Mel spectrograms (EYASE): 88.6%.
    • Transformer on BAVED: 95.2%.
    • wav2vec 2.0 on BAVED: about 80% F1.
  • BAVED is easier than EYASE. The authors attribute this to BAVED's controlled, cleaner recordings, while EYASE is spontaneous speech with fuzzier emotion boundaries.
  • Most errors fall between acoustically similar classes, especially neutral and happy.
  • Stated limitations: only two corpora, little cross-corpus testing, and no evaluation on noisy conversational audio.

Interesting ideas

  • I1. Comparing rival architectures under one fixed protocol (same preprocessing, features and metrics) makes their differences attributable to the models rather than the setup. — For Mova (learn): Run the perceptron parser, gparse and LLM-teacher outputs through one frozen evaluation harness so that score gaps reflect model quality rather than tokenization or scoring differences.
  • I2. Ablating each half of a hybrid (dropping the local encoder, then the global one) shows that both parts contribute instead of just assuming it. — For Mova (tree): Ablate gparse's local features and its long-range attention separately to check which part actually fixes long-distance attachments.
  • I3. Pairing a local feature extractor with a global context model handles cues at both short and long scales. — For Mova (frames): Combine clause-local cues with document-wide context when classifying speech acts or irony, since tone often depends on material far from the target sentence.
  • I4. Reporting results per dataset and explaining the gap through data traits (controlled versus spontaneous) keeps one easy corpus from inflating the headline number. — For Mova (answer): Report scores separately for clean school tests and for messier fables or tales, so that performance on easy sets does not hide weaknesses on harder ones.
  • I5. Error analysis focused on which close classes get confused points to where labels are inherently ambiguous rather than where the model is simply broken. — For Mova (frames): Build confusion tables for neighbouring VerbNet roles or speech-act types to separate genuine annotation ambiguity from fixable parser mistakes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Sentiment & emotion · Speech & spoken language · Semitic languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2606.10278 · arXiv 2606.10278

@misc{gheffari2026robust,
  title = {{Towards Robust Arabic Speech Emotion Recognition with Deep Learning}},
  author = {Gheffari, Youcef Soufiane and Silarbi, Samiya},
  year = {2026},
  doi = {10.48550/arxiv.2606.10278},
  eprint = {2606.10278},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2606.10278}
}

References

25 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. El Ayadi, M.S. Kamel, and F. Karray, “Survey on speech emotion recognition: Features, classification schemes, and databases,” Pattern Recognit., vol. 44, no. 3, pp. 572–587, 2011. https://doi.org/10.1016/j.patcog.2010.09.020 ↗
  2. M. Tajalsir, S.M. Hernandez, and F.A. Mohammed, “ASERS-CNN: Arabic speech emotion recognition system based on CNN model,” Signal Image Process.: Int. J., 2023. https://www.signalimageprocessingjournal.com/ ↗
  3. L. Abdel-Hamid, “Egyptian Arabic speech emotion recognition using prosodic, spectral and wavelet features,” Speech Commun., vol. 122, pp. 19–30, 2020. https://doi.org/10.1016/j.specom.2020.04.005 ↗
  4. S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. ICML, 2015, pp. 448–456. https://proceedings.mlr.press/v37/ioffe15.html ↗
  5. M. B. Akçay and K. Oğuz, “Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classifiers,” Speech Communication, vol. 116, pp. 56–76, 2020. find
  6. A. Baevski, Y. Zhou, A. Mohamed, and A. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in Adv. Neural Inf. Process. Syst., vol. 33, pp. 12449–12460, 2020. find our card
  7. A.H. Meftah, Y.A. Alotaibi, and S.-A. Selouani, ‘King Saud University Emotions corpus: Construction, analysis, evaluation, and comparison,” IEEE Access, vol. 9, pp. 54201–54219, 2021. https://doi.org/10.1109/ACCESS.2021.3070882 ↗
  8. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, pp. 1929–1958, 2014. https://jmlr.org/papers/v15/srivastava14a.html ↗
  9. R.W. Picard, Affective Computing. MIT Press, Cambridge, MA, 2000. https://mitpress.mit.edu/9780262161701/affective-computing/ ↗
  10. S.K. D’Mello and A. Graesser, “Dynamics of affective states during complex learning,” Learn. Instr., vol. 22, no. 2, pp. 145–157, 2012. https://doi.org/10.1016/j.learninstruc.2011.10.001 ↗
  11. O. Mahmoudi and M.F. Bouami, “Arabic speech emotion recognition using deep neural network,” in Int. Conf. on Digital Technologies and Applications, Springer, 2023, pp. 124–133. https://doi.org/10.1007/978-3-031-29860-8_13 ↗
  12. I. Shahin, A.B. Nassif, N. Hamsa, et al., “An efficient feature selection method for Arabic and English speech emotion recognition using Grey Wolf Optimizer,” Appl. Acoust., vol. 205, p. 109279, 2023. https://doi.org/10.1016/j.apacoust.2023.109279 ↗
  13. M. El Seknedy and S. Fawzi, “Arabic English speech emotion recognition system,” in Proc. 20th Learn. Technol. Conf. (L&T), IEEE, 2023, pp. 167–170. https://doi.org/10.1109/LT58023.2023.10092316 ↗
  14. D. Djeridi and R. Kedidi, “Emotion recognition in Arabic speech signal,” M.Sc. Thesis, University of Kasdi Merbah Ouargla, Algeria, 2020. find
  15. R. Rakan, S. Safwat, and M.A.-M. Salem, “Advancing Egyptian Arabic speech emotion recognition: Insights from 2D representations and model evaluations,” in Proc. 11th ICICIS, IEEE, 2023, pp. 154– 159. https://doi.org/10.1109/ICICIS58388.2023.10391592 ↗
  16. O. Atila and A. Şengür, “Attention guided 3D CNN-LSTM model for accurate speech based emotion recognition,” Appl. Acoust., vol. 182, p. 108260, 2021. https://doi.org/10.1016/j. apacoust.2021.108260 ↗
  17. B. Al-onazi, S. Alyahya, M. Alahmadi, et al., “Transformer-based Arabic speech emotion recognition with feature fusion and data augmentation,” 2022. find
  18. O. Mohamed and S. A. Aly, “Arabic speech emotion recognition employing wav2vec2.0 and HuBERT based on BAVED dataset,” arXiv preprint arXiv:2110.04425, 2021. https://arxiv.org/abs/2110.04425 ↗
  19. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems (NeurIPS), 2012. https://papers.nips.cc/paper_files/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html ↗
  20. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735 ↗
  21. A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM and other neural network architectures,” Neural Networks, vol. 18, no. 5–6, pp. 602–610, 2005. https://doi.org/10.1016/j.neunet.2005.03.001 ↗
  22. A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017. https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html ↗
  23. M. Tellai, L. Gao, and Q. Mao, “An efficient speech emotion recognition based on a dual-stream CNN-Transformer fusion network,” Int. J. Speech Technol., vol. 26, no. 2, pp. 541–557, 2023. https://doi.org/10.1007/s10772-023-10020-7 ↗
  24. A. Abdalla, N. Sharaf, and C. Sabty, “An enhanced Compact Convolution Transformer for age, gender and emotion detection in Egyptian Arabic speech,” in Proc. Int. Conf. on Speech and Computer, Springer, 2024, pp. 30–42. https://doi.org/10.1007/978-3-031-72775-6_3 ↗
  25. S. Safwat, M.A.-M. Salem, and N. Sharaf, “Building an Egyptian-Arabic speech corpus for emotion analysis using deep learning,” in Pacific Rim Int. Conf. on Artificial Intelligence, Springer, 2023, pp. 320–332. https://doi.org/10.1007/978-981-99-7022-3_28 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gheffari-2026-towards-robust-arabic-speech-emotion/