Elamur Research⌕
☰
home / papers / 2026 / PhonemeDF: A Synthetic Speech Dataset for Audio Deepfake Detection and Naturalness Evaluation

PhonemeDF: A Synthetic Speech Dataset for Audio Deepfake Detection and Naturalness Evaluation

Summary

Modern TTS and VC systems produce speech convincing enough to threaten voice biometrics and spread spoken misinformation. Existing deepfake corpora such as ASVspoof and MLAAD only label audio at the utterance or frame level, which makes it hard to study which speech sounds reveal synthesis. The authors address this with PhonemeDF, a parallel resource that pairs real LibriSpeech recordings with synthetic versions from seven generators. Every file comes with time-aligned phoneme boundaries produced by the Montreal Forced Aligner. The TTS systems generated speech from the original transcripts, while the VC systems transformed the real audio directly. Using these alignments, the authors measure how far synthetic phoneme distributions drift from real ones with Kullback–Leibler divergence and rank the generators by closeness to natural speech. They also train per-phoneme classifiers on handcrafted acoustic features and SSL embeddings. Phonemes with higher divergence proved easier to separate, so KLD can flag the most discriminative phonemes for detection. The resource supports interpretable, phoneme-aware deepfake detection and finer-grained naturalness assessment.

Key points

  • Real speech comes from the LibriSpeech 100-hour subset. Synthetic speech comes from four TTS models (MeloTTS, XTTS v2, Chatterbox TTS, VITS TTS) and three VC models (Chatterbox VC, FreeVC, StarGAN VC).
  • Phoneme segmentation for both real and synthetic audio is distributed as TextGrid files from the Montreal Forced Aligner (MFA).
  • Speaker identity was handled in two ways:
    • Systems lacking built-in voices were conditioned on 10 VCTK reference speakers (5 male, 5 female).
    • MeloTTS and VITS used all of their built-in voices.
    • Original LibriSpeech speakers were not cloned, and all output was resampled to 16 kHz.
  • The detection framework combines per-phoneme KLD with Logistic Regression and SVM classifiers. It compares handcrafted features against SSL embeddings for robustness and generalization.
  • The main finding is a clear correlation between real-vs-synthetic phoneme KLD and classifier performance, which positions KLD as an indicator of where synthesis artifacts concentrate.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5b4zfxnfqwm4

@inproceedings{nallaguntla2026phonemedf,
  title = {{PhonemeDF: A Synthetic Speech Dataset for Audio Deepfake Detection and Naturalness Evaluation}},
  author = {Nallaguntla, Vamshi and Fursule, Aishwarya R. and Kshirsagar, Shruti and Avila, Anderson Raymundo},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5b4zfxnfqwm4},
  url = {https://doi.org/10.63317/5b4zfxnfqwm4}
}

References

40 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, 33:12449–12460. find our card
  2. Oguzhan Baser, Ahmet Ege Tanriverdi, Sriram Vishwanath, and Sandeep P. Chinchali. 2025. Phonemefake: Redefining deepfake realism with language-driven segmental manipulation and adaptive bilevel detection. arXiv preprint arXiv:2506.22783. — [arXiv:2506.22783](https://arxiv.org/abs/2506.22783) find
  3. Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. 2024. Xtts: A massively multilingual zero-shot text-to-speech model. arXiv preprint arXiv:2406.04904. — [arXiv:2406.04904](https://arxiv.org/abs/2406.04904) find
  4. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518. find
  5. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine Learning, 20(3):273–297. find
  6. Rasmus Dall, Junichi Yamagishi, and Simon King. 2014. Rating naturalness in speech synthesis: The effect of style and expectation. In Speech Prosody 2014. find
  7. Hira Dhamyal, Ayesha Ali, Ihsan Ayyub Qazi, and Agha Ali Raza. 2021. Using self attention dnns to discover phonemic features for audio deep fake detection. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1178–1184. IEEE. find
  8. David W. Hosmer, Stanley Lemeshow, and Rodney X. Sturdivant. 2013. Applied Logistic Regression, 3rd edition. John Wiley & Sons, Hoboken, NJ. find
  9. Jee-weon Jung, Hee-Soo Heo, Hemlata Tak, Hyejin Shim, Joon Son Chung, Bong-Jin Lee, Ha-Jin Yu, and Nicholas Evans. 2022. Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Process… find
  10. Jaehyeon Kim, Jungil Kong, and Juhee Son. 2021. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In International Conference on Machine Learning, pages 5530–5540. PMLR. find
  11. Jingyi Li, Weiping Tu, and Li Xiao. 2023. Freevc: Towards high-quality text-free one-shot voice conversion. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  12. Menglu Li, Yasaman Ahmadiadli, and Xiao-Ping Zhang. 2024. Audio anti-spoofing detection: A survey. arXiv preprint arXiv:2404.13914. — [arXiv:2404.13914](https://arxiv.org/abs/2404.13914) find
  13. Yinghao Aaron Li, Ali Zare, and Nima Mesgarani. 2021. Starganv2-vc: A diverse, unsupervised, non-parallel framework for natural-sounding voice conversion. arXiv preprint arXiv:2107.10394. — [arXiv:2107.10394](https://arxiv.org/abs/2107.10394) find
  14. Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. 2017. Montreal forced aligner: Trainable text-speech alignment using kaldi. In Interspeech, pages 498–502. find
  15. Prudhvi Naayini, Praveen Kumar Myakala, Chiranjeevi Bura, Anil Kumar Jonnalagadda, and Srikanth Kamatala. 2025. Ai-powered assistive technologies for visual impairment. arXiv preprint arXiv:2503.15494. — [arXiv:2503.15494](https://arxiv.org/abs/2503.15494) find
  16. Howard C. Nusbaum, Alexander L. Francis, and Anne S. Henly. 1997. Measuring the naturalness of synthetic speech. International Journal of Speech Technology, 2:7–19. find
  17. Christine Nussbaum, Sascha Frühholz, and Stefan R. Schweinberger. 2025. Understanding voice naturalness. Trends in Cognitive Sciences. find
  18. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206–5210. IEEE. find
  19. Ryan Prenger, Rafael Valle, and Bryan Catanzaro. 2019. Waveglow: A flow-based generative network for speech synthesis. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3617–3621. IEEE. find
  20. Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019. Fastspeech: Fast, robust and controllable text to speech. Advances in Neural Information Processing Systems, 32. find
  21. Resemble AI. 2025. Chatterbox-TTS. GitHub repository. https://github.com/resemble-ai/chatterbox. ↗
  22. Davide Salvi, Viola Negroni, Sara Mandelli, Paolo Bestagini, and Stefano Tubaro. 2025. Phoneme-level analysis for person-of-interest speech deepfake detection. arXiv preprint arXiv:2507.08626. — [arXiv:2507.08626](https://arxiv.org/abs/2507.08626) find
  23. Thibault Sellam, Ankur Bapna, Joshua Camp, Diana Mackinnon, Ankur P. Parikh, and Jason Riesa. 2023. Squid: Measuring speech naturalness in many languages. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  24. Ganesh Sivaraman, Hemlata Tak, and Elie Khoury. 2025. Investigating voiced and unvoiced regions of speech for audio deepfake detection. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  25. Gajan Suthokumar, Kaavya Sriskandaraja, Vidhyasaharan Sethu, Chamith Wijenayake, and Eliathamby Ambikairajah. 2019. Phoneme specific modelling and scoring techniques for anti spoofing system. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pag… find
  26. Dia Elhak Temmar, Assia Hamadene, Vamshi Nallaguntla, Aishwarya Fursule, Mohand Saïd Allili, Shruti Kshirsagar, and Anderson R. Avila. 2025. Phonetic analysis of real and synthetic speech using hubert embeddings: Perspectives for deepfake detection. In 2025 IEEE International Conference on Systems,… find
  27. Massimiliano Todisco, Xin Wang, Ville Vestman, Md Sahidullah, Héctor Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi Kinnunen, and Kong Aik Lee. 2019. find
  28. Asvspoof 2019: Future horizons in spoofed and fake audio detection. arXiv preprint arXiv:1904.05441. — [arXiv:1904.05441](https://arxiv.org/abs/1904.05441) find
  29. Christophe Veaux, Junichi Yamagishi, and Kirsten MacDonald. 2017. Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit. Available at: http://dx.doi.org/10.7488/ds/1994. — [doi:10.7488/ds/1994](https://doi.org/10.7488/ds/1994) ↗
  30. Jennifer M. Vojtech, Jacob P. Noordzij Jr, Gabriel J. Cler, and Cara E. Stepp. 2019. The effects of modulating fundamental frequency and speech rate on the intelligibility, communication efficiency, and perceived naturalness of synthetic speech. American Journal of Speech-Language Pathology, 28(2S):… find
  31. Zhizheng Wu, Tomi Kinnunen, Nicholas Evans, Junichi Yamagishi, Cemal Hanilçi, Md Sahidullah, and Aleksandr Sizov. 2015. find
  32. Asvspoof 2015: The first automatic speaker verification spoofing and countermeasures challenge. In INTERSPEECH 2015, Automatic Speaker Verification Spoofing and Countermeasures Challenge, colocated with INTERSPEECH 2015, pages 2037–2041. ISCA. find
  33. Junichi Yamagishi, Xin Wang, Massimiliano Todisco, Md Sahidullah, Jose Patino, Andreas Nautsch, Xuechen Liu, Kong Aik Lee, Tomi Kinnunen, Nicholas Evans, et al. 2021. find
  34. Asvspoof 2021: Accelerating progress in spoofed and deepfake speech detection. arXiv preprint arXiv:2109.00537. — [arXiv:2109.00537](https://arxiv.org/abs/2109.00537) find
  35. Tianle Yang, Chengzhe Sun, Siwei Lyu, and Phil Rose. 2025. Forensic deepfake audio detection using segmental speech features. Forensic Science International, page 112768. find
  36. Jiangyan Yi, Ruibo Fu, Jianhua Tao, Shuai Nie, Haoxin Ma, Chenglong Wang, Tao Wang, Zhengkun Tian, Ye Bai, Cunhang Fan, et al. 2022. find
  37. Add 2022: The first audio deep synthesis detection challenge. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 9216– 9220. IEEE. find
  38. Kuiyuan Zhang, Zhongyun Hua, Rushi Lan, Yushu Zhang, and Yifang Guo. 2025. Phoneme-level feature discrepancies: A key to detecting sophisticated speech deepfakes. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1066–1074. find
  39. Wenliang Zhao, Xumin Yu, and Zengyi Qin. 2023. Melotts: High-quality multi-lingual multi-accent text-to-speech. GitHub repository. https://github.com/myshell-ai/MeloTTS. ↗
  40. Yi Zhu, Surya Koppisetti, Trang Tran, and Gaurav Bharaj. 2024. Slim: Style-linguistics mismatch model for generalized audio deepfake detection. Advances in Neural Information Processing Systems, 37:67901–67928. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nallaguntla-2026-phonemedf-synthetic-speech-dataset-audio/