Elamur Research⌕
☰
home / papers / 2026 / Contrastive and Adversarial Disentanglement for Speaker Representations in Brazilian Portuguese

Contrastive and Adversarial Disentanglement for Speaker Representations in Brazilian Portuguese

Paper · 2026 · Ariadne Nascimento Matos, Arnaldo Candido Junior and Moacir Antonelli Ponti

Summary

Speaker embeddings often absorb traces of the recording environment, such as noise, music or reverberation, and this hurts speaker verification. The authors study how to separate speaker identity from environment in Brazilian Portuguese, which they describe as the first such work for Portuguese. They extend the adversarial disentanglement framework of Nam et al. (2024). An ECAPA-TDNN backbone feeds an autoencoder that splits each embedding into a speaker part and an environment part. Speaker and environment discriminators, a correlation penalty and an L1 reconstruction term enforce the split.

The original framework used only a Triplet Loss. Here the environment branch is also trained with supervised SupCon or self-supervised SimCLR. Evaluation uses a new synthetic corpus, ST1, and the Portuguese portion of CML-TTS, with EER/MinDCF and t-SNE plots.

SupCon gives the best verification on ST1 (EER 4.70%). On the more realistic CML-TTS, stronger environment invariance raises error rates, which suggests the model had partly used environmental cues to tell speakers apart. The work shows the value of controlled synthetic benchmarks for diagnosing disentanglement.

Key points

  • ST1 dataset. Speech was generated with XTTS v2 zero-shot cloning of Braccent speakers: 69 for training and 28 for testing, each reading 50 sentences from classic Brazilian literature. The splits are about 34 h (20,272 utterances) and about 17 h (10,220 utterances). MUSAN conditions were applied: point-source noise, hd-classical, sound-bible and fma-western-art.
  • CML-TTS subset. The Portuguese portion of CML-TTS covers 69 h from 48 speakers (31 male, 17 female).
  • Results on ST1. SupCon reached an EER of 4.70%. The abstract reports MinDCF 0.24, while the body text gives 0.34. SimCLR came close at EER 6% and MinDCF 0.38.
  • Results on CML-TTS. EERs were 14.80% for Triplet Loss with environment dissociation, 18.5% for SupCon and 20.7% for SimCLR. This illustrates a trade-off between discriminating speakers and being invariant to the environment.
  • SupCon EER by environment. It was lowest for hd-classical (4.76%), then point-source (7.69%) and fma-western (8.57%), and highest for sound-bible (11.43%).
  • Ablation on ST1 (SupCon setup). Removing any single loss (adversarial, environment SupCon, reconstruction or correlation) raised the EER to 11.3–12.0%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Ariadne Nascimento Matos, Arnaldo Candido Junior and Moacir Antonelli Ponti. 2026. Contrastive and Adversarial Disentanglement for Speaker Representations in Brazilian Portuguese. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{matos2026contrastive,
  title = {{Contrastive and Adversarial Disentanglement for Speaker Representations in Brazilian Portuguese}},
  author = {Ariadne Nascimento Matos, Arnaldo Candido Junior and Moacir Antonelli Ponti},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.58/}
}

References

31 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. This work was carried out at the Center for Artificial Intelligence (C4AI-USP), with support by FAPESP grant \#2019/07665-4 and IBM Corporation, and was also supported by the Ministry of Science, Technology and Innovation, with resources of Law No. 8.248, of Oct 23, 1991, within the scope of PPI-SOFT… find
  2. Nathalia Alves Rocha Batista. 2019. Estudo sobre identificação automática de sotaques regionais brasileiros baseada em modelagens estatísticas e técnicas de aprendizado de máquina. Master’s thesis, Universidade Estadual de Campinas, Campinas. find
  3. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828.
  4. Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, and Julian Weber. 2024. Xtts: a massively multilingual zero-shot text-to-speech model. pages 4978–4982. find
  5. Edresson Casanova, Christopher Shulby, Alexander Korolev, Arnaldo Candido Junior, Anderson da Silva Soares, Sandra Aluísio, and Moacir Antonelli Ponti. 2023. ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversion. In Proc. find
  6. Interspeech 2023, pages 1244–1248. find
  7. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, and 1 others. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518. find
  8. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning (ICML).
  9. Zhiwei Deng, Rajitha Navarathna, Peter Carr, Stephan Mandt, Yisong Yue, Iain Matthews, and Greg Mori. 2017. Factorized variational autoencoders for modeling audience reactions to movies. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6014–6023. find
  10. Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification. In 21st Annual Conference of the International Speech Communication Association, Interspeech 2020, Virtual Event, Shanghai, China,… find
  11. William J Hardcastle, John Laver, and Fiona E Gibbon. 2012. The handbook of phonetic sciences. John Wiley Sons. find
  12. Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. ArXiv, abs/1807.03748.
  13. Woo Hyun Kang, Sung Hwan Mun, Min Hyun Han, and Nam Soo Kim. 2020. Disentangled speaker and nuisance attribute embedding for robust speaker verification. IEEE Access, 8:141838–141849. find
  14. Xin Wang, Hong Chen, Si’ao Tang, Zihao Wu, and Wenwu Zhu. 2024. Disentangled representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):9677–9696. find
  15. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Advances in Neural Information Processing Systems (NeurIPS). find
  16. Yongqiang Wang and Mark J. F. Gales. 2012. Speaker and noise factorization for robust speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, 20(7):2149–2158. find
  17. Jiachen Lian, Chunlei Zhang, and Dong Yu. 2022. Robust disentangled variational speech representation learning for zero-shot voice conversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6572–6576. IEEE. find
  18. Tianchi Liu, Kong Aik Lee, Qiongqiong Wang, and Haizhou Li. 2023. Disentangling voice and content with self-supervision for speaker recognition. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc. find
  19. Aditya Menon, Raj Gohil, Kumud Tripathi, and Pankaj W. 2025. Laspa: Language agnostic speaker disentanglement with prefix-tuned cross-attention. In Interspeech, pages 3623–3627. find
  20. Nasser-Eddine Monir, Paul Magron, and Romain Serizel. 2024. A phoneme-scale assessment of multi-channel speech enhancement algorithms. Trends in Hearing, 28:23312165241292205. find
  21. KiHyun Nam, Hee-Soo Heo, Jee-weon Jung, and Joon Son Chung. 2024. Disentangled representation learning for environment-agnostic speaker recognition. arXiv preprint arXiv:2406.14559. — [arXiv:2406.14559](https://arxiv.org/abs/2406.14559) find
  22. Yuying Xie, Michael Kuhlmann, Frederik Rautenberg, Zheng-Hua Tan, and Reinhold Haeb-Umbach. 2024. Speaker and style disentanglement of speech based on contrastive predictive coding supported factorized variational autoencoder. In 2024 32nd European Signal Processing Conference (EUSIPCO), pages 436–4… find
  23. X. Xing, M. Xu, and T. F. Zheng. 2024. A joint noise disentanglement and adversarial training framework for robust speaker verification. In Proceedings of Interspeech 2024, pages 707–711. find
  24. Yuhang Xue, Ning Chen, Yixin Luo, Hongqing Zhu, and Zhiying Zhu. 2024. Clessr-vc: Contrastive learning enhanced self-supervised representations for one-shot voice conversion. Speech Communication, 165:103139. find
  25. Ivan Yakovlev, Rostislav Makarov, Andrei Balykin, Pavel Malov, Anton Okhotnikov, and Nikita Torgashov. 2024. Reshape dimensions network for speaker recognition. In Interspeech 2024, pages 3235–3239. find
  26. Olivier Zhang, Olivier Le Blouch, Nicolas Gengembre, and Damien Lolive. 2023. An extension of disentanglement metrics and its application to voice. find
  27. Proc. Interspeech 2023, pages 2878–2882. find
  28. Frederico S. Oliveira, Edresson Casanova, Arnaldo Candido Junior, Anderson S. Soares, and Arlindo R. Galvão Filho. 2023. Cml-tts: A multilingual dataset for speech synthesis in low-resource languages. In Text, Speech, and Dialogue, pages 188–199, Cham. Springer Nature Switzerland. find
  29. Giuseppe Ruggiero, Matteo Testa, Jurgen Van De Walle, and Luigi Di Caro. 2025. Eta-WavLM: Efficient speaker identity removal in self-supervised speech representations using a simple linear equation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 2494–2504, Vienna, Aust… find
  30. David Snyder, Guoguo Chen, and Daniel Povey. 2015. MUSAN: A music, speech, and noise corpus. CoRR, abs/1510.08484. — [arXiv:1510.08484](https://arxiv.org/abs/1510.08484) find
  31. Youzhi Tu, Man-Wai Mak, and Jen-Tzung Chien. 2024. Contrastive self-supervised speaker embedding with sequential disentanglement. IEEE/ACM Trans. Audio, Speech and Lang. Proc., 32:2704–2715. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/matos-2026-contrastive-adversarial-disentanglement-speaker-representations/