Elamur Research⌕
☰
home / papers / 2026 / Dynamic Layer Selection for Efficient Tone Recognition in Self-Supervised Speech Models

Dynamic Layer Selection for Efficient Tone Recognition in Self-Supervised Speech Models

Summary

Tonal languages use pitch to distinguish word meanings. Many African tonal languages have very little labelled speech, which makes tone recognition hard. Earlier work by the same authors fed only the final layer of Wav2vec 2.0 into a tone recogniser for Yoruba. This study asks whether the intermediate layers of a self-supervised model hold tonal cues that a final-layer-only setup misses.

The authors use SSA-HuBERT, a HuBERT-base model pre-trained on African speech. They attach a SUPERB-style module that learns a softmax-normalised weight for every layer and mixes the layers accordingly. The mixed representation goes to a bidirectional GRU decoder trained with CTC, so no syllable alignment is required. The system is tested on a Yoruba corpus (about 4 h) and on YembaTones (47 min).

Middle layers performed best overall. The learned weighting beat the final layer clearly on both languages, and it cost far less than training one model per layer. The learned weights also show which layers each language relies on. This gives a cheap and interpretable recipe for low-resource tonal speech processing.

Key points

  • Backbone: SSA-HuBERT has 12 Transformer layers and about 90 M parameters. It was pre-trained on roughly 60 000 hours of unlabelled audio covering 21 African languages and dialects.
  • Yoruba data: The corpus of Gutkin et al. (2020) was originally built for TTS/ASR and has 36 speakers. It was re-annotated for three tones via automatic syllabification. After keeping utterances of 6 s or less, it contains 3 223 utterances (≈3.3 h).
  • Yemba data: The YembaTones corpus has 3 420 recordings (47 min) from 11 native speakers, covering 344 words.
  • Accuracy gains: Compared with using only the last layer, weighted layer combination lowers tone error rate (TER) by 20.4% relative for Yoruba and 15.8% relative for Yemba.
  • Efficiency and layer patterns: Training time and memory drop by more than 90% compared with training a separate model per layer. The learned weights favour middle layers for Yoruba, while Yemba leans more on early layers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3hi6ut7agp4y

@inproceedings{obiang2026dynamic,
  title = {{Dynamic Layer Selection for Efficient Tone Recognition in Self-Supervised Speech Models}},
  author = {OBIANG, Saint Germes B. BENGONO and TSOPZE, Norbert and YONTA, Paulin MELATAGIA},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3hi6ut7agp4y},
  url = {https://doi.org/10.63317/3hi6ut7agp4y}
}

References

24 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Oliver Adams, Trevor Cohn, Graham Neubig, and Alexis Michaud. 2017. Phonemic transcription of low-resource tonal languages. In Proceedings of the Australasian Language Technology Association Workshop 2017, pages 53–60, Brisbane, Australia. find
  2. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, pages 12449–12460. Curran Associates, Inc. find our card
  3. S.G.B. Bengono OBIANG, Paulin Melatagia Yonta, Norbert Tsopze, Jean-Francois Bonastre, and Tania Jimenez. 2024. Improving tone recognition performance using wav2vec 2.0-based learned representation in yoruba, a low-resourced language. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 1(1). find
  4. Antoine Caubrière and Elodie Gauthier. 2024. Africa-centric self-supervised pre-training for multilingual speech representation in a sub-saharan context. find
  5. Charles Chen, Razvan C. Bunescu, Li Xu, and Chang Liu. 2016. Tone classification in mandarin chinese using convolutional neural networks. In Interspeech. find
  6. Mingming Chen, Zhanlei Yang, and Wen-Ju Liu. 2014. Deep neural networks for mandarin tone recognition. pages 1154–1158. find
  7. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei. 2022. Wavlm: Large-scale self-supervised pre-training for full stack… find
  8. Wojciech Chmiel, Joanna Kwiecień, and Kacper Motyka. 2023. Saliency map and deep learning in binary classification of brain tumours. Sensors, 23(9). find
  9. Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference on Machine Learning, ICML ’06, page 369–376, New York, NY, USA… find
  10. Alexander Gutkin, Işın Demirşahin, Oddur Kjartansson, Clara Rivera, and Kọ́lá Túbọ̀sún. 2020. Developing an Open-Source Corpus of Yoruba Speech. In Proceedings of Interspeech 2020, pages 404–408, Shanghai, China. International Speech and Communication Association (ISCA). find
  11. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdel rahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451–3460. find our card
  12. Marc Sturm Kenfack Jeuguim, Paulin Melatagia Yonta, and Etienne Sandembouo. 2024. Yembatones: A syllable-tone annotated dataset for speech recognition and prosodic analysis of the yemba language. Data in Brief, 52:109860. find
  13. Tom Ko, Vijayaditya Peddinti, Daniel Povey, Michael L. Seltzer, and Sanjeev Khudanpur. 2017. A study on data augmentation of reverberant speech for robust speech recognition. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5220–5224. find
  14. Tuan Anh Le, Maximilian Igl, Tom Rainforth, Tom Jin, and Frank Wood. 2018. Auto-encoding sequential monte carlo. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. find
  15. Xu Li, Zhang Wenle, Zhou Ning, Lee Chaoyang, Li Yongxin, Chen Xiuwu, and Zhao Xiaoyan. 2006. Mandarin chinese tone recognition with an artificial neural network. Journal of Otology, 1(1):30–34. find
  16. Loren Lugosch and Vikrant Singh Tomar. 2018. Tone recognition using lifters and ctc. In Interspeech. find
  17. Odetunji Ajadi Odejobi. 2008. Recognition of tones in yorùbá speech: Experiments with artificial neural networks. In Speech, Audio, Image and Biomedical Signal Processing using Neural Networks. find
  18. Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Specaugment: A sim-Quoc V. Le. 2019. ple data augmentation method for automatic speech recognition. In Interspeech 2019. ISCA. find
  19. Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong, Ju-Chieh Chou, Sung-Lin Yeh, Szu-Wei Fu, Chien-Feng Liao, Elena Rastorgueva, François Grondin, William Aris, Hwidong Na, Yan Gao, Renato D… — [arXiv:2106.04624](https://arxiv.org/abs/2106.04624) find
  20. Neville Ryant, Jiahong Yuan, and Mark Liberman. 2014. Mandarin tone classification without pitch tracking. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4868–4872. find
  21. Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. In Interspeech 2019, pages 3465–3469. find
  22. Daniel R. van Niekerk and Etienne Barnard. 2012. Tone realisation in a yorùbá speech recognition corpus. In Proc. 3rd Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU 2012), pages 54–59. find
  23. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11:3371–3408. find
  24. Shu-wen Yang, Heng-Jui Chang, Zili Huang, Andy T. Liu, Cheng-I Lai, Haibin Wu, Jiatong Shi, Xuankai Chang, Hsiang-Sheng Tsai, Wen-Chin Huang, Tzu-hsun Feng, Po-Han Chi, Yist Y. Lin, Yung-Sung Chuang, Tzu-Hsien Huang, Wei-Cheng Tseng, Kushal Lakhotia, Abdelrahman Mohamed, Shang-Wen Li, Shinji Watanab… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/obiang-2026-dynamic-layer-selection-efficient-tone/