Elamur Research⌕
☰
home / papers / 2026 / A Unified Feature Mixture Framework for Joint Speech and Singing Deepfake Detection

A Unified Feature Mixture Framework for Joint Speech and Singing Deepfake Detection

Paper · 2026 · Aastha Sharma and Guangjing Wang

Summary

Detectors trained on conversational speech handle speech deepfakes well but break down on synthetic singing. Singing has sustained harmonics, vibrato and wide pitch ranges that speech lacks. Running separate speech and singing detectors would also need a routing step, so the authors build one model for both domains. GenuVoice is a mixture-of-experts detector with three feature branches: log-mel spectrograms and MFCC, each processed by a ResNet-18, and a frozen Wav2Vec 2.0 encoder with a pooling head. Each branch gets its own auxiliary classification loss so it remains useful alone. An MLP gating network weights the branch embeddings before a linear spoof classifier. Training has three stages: experts on speech, then the gate and classifier with frozen experts, then joint fine-tuning on mixed speech and singing to limit catastrophic forgetting. GenuVoice reaches 1.82% EER on CtrSVDD, keeps strong speech results on ASVspoof 2019 and generalizes to held-out ASVspoof 2021. This is useful for media that mixes talking and singing, and for spotting cloned artist voices.

Key points

  • Four speech-trained baselines (RawNet2, AASIST, Wav2Vec2-AASIST, Wav2Vec2-DF) score 37–62% EER on the CtrSVDD singing benchmark.
  • GenuVoice gets 1.82% EER on CtrSVDD, about 95% better relative to the best baseline. It also scores 0.38% EER on ASVspoof 2019 and 8.89% on held-out ASVspoof 2021 (unseen generators and codecs).
  • Adversarial domain alignment (DANN) causes negative transfer, reaching 26.45% EER even at 50/50 speech/singing mixing. The authors attribute this to spoof cues that partly depend on the domain.
  • In ablations, removing auxiliary expert supervision or diversity regularization raises EER to 3.45–12.34%.
  • Stated limitations are an opaque gate, coverage limited to benchmark generators, and higher inference cost from multiple branches. Code is released at github.com/aastha-sharma/genuvoice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of South FloridaTampa · United States7 works · 33,127 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

Cite

Cite as Aastha Sharma and Guangjing Wang. 2026. A Unified Feature Mixture Framework for Joint Speech and Singing Deepfake Detection. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1245

@inproceedings{sharma2026unified,
  title = {{A Unified Feature Mixture Framework for Joint Speech and Singing Deepfake Detection}},
  author = {Aastha Sharma and Guangjing Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1245},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1245}
}

References

22 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460.
  2. Alexandre Défossez, Nicolas Usunier, Léon Bottou, and Francis Bach. 2019. Music source separation in the waveform domain. arXiv preprint arXiv:1911.13254. — [arXiv:1911.13254](https://arxiv.org/abs/1911.13254) find
  3. Joel Frank and Lea Schönherr. 2021. Wavefake: A data set to facilitate audio deepfake detection. arXiv preprint arXiv:2111.02813. — [arXiv:2111.02813](https://arxiv.org/abs/2111.02813) find
  4. Robert M French. 1999. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128–135. find
  5. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, and 1 others. 2017. Overcom… find
  6. Kavita Kumari, Maryam Abbasihafshejani, Alessandro Pegoraro, Phillip Rieger, Kamyar Arshi, Murtuza Jadliwala, and Ahmad-Reza Sadeghi. 2025. Voiceradar: Voice deepfake detection using micro-frequency and compositional analysis. In NDSS. find
  7. Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, and Zhou Zhao. 2022. Diffsinger: Singing voice synthesis via shallow diffusion mechanism. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 11020–11028. find
  8. Xuechen Liu, Xin Wang, Md Sahidullah, Jose Patino, Héctor Delgado, Tomi Kinnunen, Massimiliano Todisco, Junichi Yamagishi, Nicholas Evans, Andreas Nautsch, and 1 others. 2023. find
  9. Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2507–2522. find
  10. Nicolas M Müller, Pavel Czempin, Franziska Dieckmann, Adam Froghyar, and Konstantin Böttinger. 2022. Does audio deepfake detection generalize? arXiv preprint arXiv:2203.16263. find
  11. Viola Negroni, Davide Salvi, Alessandro Ilic Mezza, Paolo Bestagini, and Stefano Tubaro. 2025. Leveraging mixture of experts for improved speech deepfake detection. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  12. Hemlata Tak, Jose Patino, Massimiliano Todisco, Andreas Nautsch, Nicholas Evans, and Anthony Larcher. 2021. End-to-end anti-spoofing with rawnet2. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6369–6373. IEEE. find
  13. Yongyi Zang, Jiatong Shi, You Zhang, Ryuichi Yamamoto, Jionghao Han, Yuxun Tang, Shengyuan Xu, Wenxiao Zhao, Jing Guo, Tomoki Toda, and 1 others. 2024a. Ctrsvdd: A benchmark dataset and baseline analysis for controlled singing voice deepfake detection. arXiv preprint arXiv:2406.02438. — [arXiv:2406.02438](https://arxiv.org/abs/2406.02438) find
  14. Yongyi Zang, You Zhang, Mojtaba Heydari, and Zhiyao Duan. 2024b. Singfake: Singing voice deepfake detection. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12156–12160. IEEE. find
  15. Yongmao Zhang, Jian Cong, Heyang Xue, Lei Xie, Pengcheng Zhu, and Mengxiao Bi. 2022. Visinger: Variational inference with adversarial learning for end-to-end singing voice synthesis. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7237– 72… find
  16. Hemlata Tak, Massimiliano Todisco, Xin Wang, Jee-weon Jung, Junichi Yamagishi, and Nicholas Evans. 2022. Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation. arXiv preprint arXiv:2202.12233. — [arXiv:2202.12233](https://arxiv.org/abs/2202.12233) find
  17. Massimiliano Todisco, Xin Wang, Ville Vestman, Md Sahidullah, Héctor Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi Kinnunen, and Kong Aik Lee. 2019. find
  18. Asvspoof 2019: Future horizons in spoofed and fake audio detection. arXiv preprint arXiv:1904.05441. find
  19. Guangjing Wang, Qiben Yan, Shane Patrarungrong, Juexing Wang, and Huacheng Zeng. 2023a. Facer: Contrastive attention based expression recognition via smartphone earpiece speaker. In IEEE INFO-COM 2023-IEEE conference on computer communications, pages 1–10. IEEE. find
  20. Yuanda Wang, Bocheng Chen, Hanqing Guo, Guangjing Wang, Weikang Ding, and Qiben Yan. 2025. Clear-mask: Noise-free and naturalness-preserving protection against voice deepfake attacks. In Proceedings of the 20th ACM Asia Conference on Computer and Communications Security, pages 696–709. find
  21. Yuanda Wang, Hanqing Guo, Guangjing Wang, Bocheng Chen, and Qiben Yan. 2023b. Vsmask: Defending against voice synthesis attack via real-time predictive perturbation. In Proceedings of the 16th ACM Conference on Security and Privacy in Wireless and Mobile Networks, pages 239–250. find
  22. Zirui Wang, Zihang Dai, Barnabás Póczos, and Jaime Carbonell. 2019. Characterizing and avoiding negative transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11293–11302. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sharma-2026-unified-feature-mixture-framework-joint/