Elamur Research⌕
☰
home / papers / 2026 / MelTrim: Coarse-to-Fine Data Pruning for Speech Classification

MelTrim: Coarse-to-Fine Data Pruning for Speech Classification

Paper · 2026 · Shaobo Wang, Tianle Niu, Xuan Ouyang et al.

Summary

Data pruning, which means selecting a small coreset that trains models nearly as well as the full dataset, has been studied mostly for images and text. It has received little attention for speech classification. The authors used Sylber syllable features on M3ED to show that speech is redundant in two ways: across utterances and between neighbouring frames within an utterance. They propose MelTrim, a two-stage coarse-to-fine pruning method:

  1. Coarse stage (utterance level): MFCCs are padded or truncated, flattened, and projected to 2D with UMAP. DBSCAN then removes noise points, and from each cluster the utterances nearest the centroid are kept in proportion to cluster size.
  2. Fine stage (frame level): A judge model pre-trained on the full data scores utility, and uninformative segments are dropped.

The method was tested on Speech Emotion Recognition (MEAD, MELD, M3ED) and Speaker Identification (VoxCeleb1). It beats general-purpose pruning baselines even with coresets far below 3% of the data, while reducing total training time. The authors say this is the first pruning method designed specifically for speech classification. The code is public.

Key points

  • The reported best gains are up to 49.5% higher Weighted Accuracy for SER and a 41.9% lower EER on VoxCeleb1 for SID. The paper is inconsistent about the 49.5% figure: the abstract attributes it to MEAD, while the conclusion names M3ED.
  • Wav2Vec and Whisper-tiny serve as judge models, and GradNorm-based scoring is used in Step 2. For example, on MEAD with a 1.813% coreset, WA rises from 50.18 (best baseline) to 64.67 with the Whisper-tiny judge.
  • Coreset sizes range from roughly 0.026% to 2.4% of each dataset.
  • On a single NVIDIA H20 GPU, end-to-end time (selection plus training) falls by about 31–63% compared with 40-epoch full-data training.
  • The paper also reports:
    • a small ASR test on MELD with lower WER;
    • weak-to-strong transfer across Whisper model sizes;
    • the limitation that only acoustic features are used, with no transcripts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🎓Northeastern UniversityBoston · United States68 works · 7,316 citations🎓Northeastern UniversityShenyang · China53 works · 2,205 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSignal Processing · 493 works

🧭 Topics 1Research subjects and languages the work is about.Efficient models & compression · 297 works

Cite

Cite as Shaobo Wang, Tianle Niu, Xuan Ouyang, Xintong Li, Zhengkun Ge, Yue Min, Xiaoqian Liu, Hankun Wang and Linfeng Zhang. 2026. MelTrim: Coarse-to-Fine Data Pruning for Speech Classification. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.672

@inproceedings{wang2026meltrim,
  title = {{MelTrim: Coarse-to-Fine Data Pruning for Speech Classification}},
  author = {Shaobo Wang, Tianle Niu, Xuan Ouyang, Xintong Li, Zhengkun Ge, Yue Min, Xiaoqian Liu, Hankun Wang and Linfeng Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.672},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.672}
}

References

41 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sharat Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. 2020. Contextual diversity for active learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pages 137–153. Springer. find
  2. Keyu An, Qian Chen, Chong Deng, Zhihao Du, Changfeng Gao, Zhifu Gao, Yue Gu, Ting He, Hangrui Hu, Kai Hu, Shengpeng Ji, Yabin Li, Zerui Li, Heng Lu, Haoneng Luo, Xiang Lv, Bin Ma, Ziyang Ma, Chongjia Ni, and 14 others. 2024. Funaudiollm: Voice understanding and generation foundation models for natur… — [arXiv:2407.04051](https://arxiv.org/abs/2407.04051) find
  3. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460.
  4. Seung bin Kim, Chan yeong Lim, Jungwoo Heo, Ju ho Kim, Hyun seo Shin, Kyo-Won Koo, and Ha-Jin Yu. 2024. Mr-rawnet: Speaker verification system with multiple temporal resolutions for variable duration utterances using raw waveforms. In Interspeech 2024, pages 2125–2129. find
  5. Cheol Jun Cho, Nicholas Lee, Akshat Gupta, Dhruv Agarwal, Ethan Chen, Alan W Black, and Gopala K Anumanchipalli. 2024. Sylber: Syllabic embedding representation of speech from raw audio. arXiv preprint arXiv:2410.07168. — [arXiv:2410.07168](https://arxiv.org/abs/2410.07168) find
  6. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. Qwen2-audio technical report. CoRR, abs/2407.10759.
  7. Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. 2019. Selection via proxy: Efficient data selection for deep learning. In Int. Conf. Learn. Represent. find
  8. Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. Arcface: Additive angular margin loss for deep face recognition. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4685–4694. find
  9. Muqing Deng, Tingting Meng, Jiuwen Cao, Shimin Wang, Jing Zhang, and Huijie Fan. 2020. Heart sound classification based on improved mfcc features and convolutional recurrent neural networks. Neural Networks, 130:22–32. find
  10. Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification. In Interspeech 2020, pages 3830–3834. find
  11. Moataz El Ayadi, Mohamed S Kamel, and Fakhri Karray. 2011. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern recognition, 44(3):572–587. find
  12. Moataz MH El Ayadi, Mohamed S Kamel, and Fakhri Karray. 2007. Speech emotion recognition using gaussian mixture vector autoregressive models. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, volume 4, pages IV–957. IEEE. find
  13. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE ACM Trans. Audio Speech Lang. Process., 29:3451–3460.
  14. Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, and Furu Wei. 2024. Wavllm: Towards robust and adaptive speech large language model. In Findings of the Association for Computational Linguistics: EMNLP 2024,… find
  15. Dias Issa, M Fatih Demirci, and Adnan Yazici. 2020. Speech emotion recognition with deep convolutional neural networks. Biomedical Signal Processing and Control, 59:101894. find
  16. Jee-weon Jung, You Jin Kim, Hee-Soo Heo, Bong-Jin Lee, Youngki Kwon, and Joon Son Chung. 2022a. Pushing the limits of raw waveform speaker recognition. Preprint, arXiv:2203.08488. — [arXiv:2203.08488](https://arxiv.org/abs/2203.08488) find
  17. Jee-weon Jung, You Jin Kim, Hee-Soo Heo, Bong-Jin Lee, Youngki Kwon, and Joon Son Chung. 2022b. Pushing the limits of raw waveform speaker recognition. Proc. Interspeech. find
  18. Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. 2021. Grad-match: Gradient matching based data subset selection for efficient deep model training. In Proc. Int. Conf. Mach. Learn., pages 5464–5474. find
  19. Siddique Latif, Rajib Rana, Shahzad Younis, Junaid Qadir, and Julien Epps. 2018. Transfer learning for improving speech emotion classification accuracy. arXiv preprint arXiv:1801.06353. — [arXiv:1801.06353](https://arxiv.org/abs/1801.06353) find
  20. Aristidis Likas, Nikos Vlassis, and Jakob J Verbeek. 2003. The global k-means clustering algorithm. Pattern recognition, 36(2):451–461. find
  21. Xiaoqian Liu, Guoqiang Hu, Yangfan Du, Erfeng He, Yingfeng Luo, Chen Xu, Tong Xiao, and Jingbo Zhu. 2024. Recent advances in end-to-end simultaneous speech translation. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pages 8142–8150. Internatio… find
  22. Ziyang Ma, Mingjie Chen, Hezhao Zhang, Zhisheng Zheng, Wenxi Chen, Xiquan Li, Jiaxin Ye, Xie Chen, and Thomas Hain. 2024a. Emobox: Multilingual multi-corpus speech emotion recognition toolkit and benchmark. Preprint, arXiv:2406.07162. — [arXiv:2406.07162](https://arxiv.org/abs/2406.07162) find
  23. Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, ShiLiang Zhang, and Xie Chen. 2024b. emotion2vec: Self-supervised pre-training for speech emotion representation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15747–15760, Bangkok, Thailand. Association for… find
  24. Katerina Margatina, Giorgos Vernikos, Loïc Barrault, and Nikolaos Aletras. 2021. Active learning by acquiring contrastive examples. arXiv preprint arXiv:2109.03764. find
  25. Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. 2018. Umap: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29):861. find
  26. Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. 2020. Coresets for data-efficient training of machine learning models. In Proc. Int. Conf. Mach. Learn., pages 6950–6960. find
  27. Edmilson Morais, Ron Hoory, Weizhong Zhu, Itai Gat, Matheus Damasceno, and Hagai Aronowitz. 2022. Speech emotion recognition using self-supervised features. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6922–6926. IEEE. find
  28. A. Nagrani, J. S. Chung, and A. Zisserman. 2017. Voxceleb: a large-scale speaker identification dataset. In INTERSPEECH. find
  29. Muhammet Pakyurek, Mahir Atmis, Selman Kulac, and Umut Uludag. 2020. Extraction of novel features based on histograms of mfccs used in emotion classification from generated original speech dataset. Elektronika ir Elektrotechnika, 26(1):46–51. find
  30. Mansheej Paul, Brett Larsen, Surya Ganguli, Jonathan Frankle, and Gintare Karolina Dziugaite. 2022. Lottery tickets on a data diet: Finding initializations with sparse trainable networks. In Adv. Neural Inform. Process. Syst., volume 35, pages 18916–18928. find
  31. Karl Pearson. 1901. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559–572. find
  32. Alexander Platzer. 2013. Visualization of snps with t-sne. PloS one, 8(2):e56883. find
  33. Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger. 2020. Identifying mislabeled data using the area under the margin ranking. In Adv. Neural Inform. Process. Syst., volume 33, pages 17044– 17056. find
  34. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR.
  35. Ozan Sener and Silvio Savarese. 2018. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations. find
  36. Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. 2019. An empirical study of example forgetting during deep neural network learning. In Int. Conf. Learn. Represent. find
  37. Daren Wang, Xinyang Lu, and Alessandro Rinaldo. 2017. Dbscan: Optimal rates for density based clustering. arXiv preprint arXiv:1706.03113. — [arXiv:1706.03113](https://arxiv.org/abs/1706.03113) find
  38. Max Welling. 2009. Herding dynamical weights to learn. In Proc. Int. Conf. Mach. Learn., pages 1121– 1128. find
  39. Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. 2022. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In Int. Conf. Learn. Represent. find
  40. Chen Xu, Yuhao Zhang, Chengbo Jiao, Xiaoqian Liu, Chi Hu, Xin Zeng, Tong Xiao, Anxiang Ma, Huizhen Wang, and Jingbo Zhu. 2023. Bridging the granularity gap for acoustic modeling. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 10816– 10… find
  41. Min Xu, Ling-Yu Duan, Jianfei Cai, Liang-Tien Chia, Changsheng Xu, and Qi Tian. 2004. Hmm-based audio keyword generation. In Pacific-Rim Conference on Multimedia, pages 566–574. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-meltrim-coarse-fine-data-pruning/