Elamur Research⌕
☰
home / papers / 2026 / Spectral Gravity Formant Estimation for Phonetic Segmentation

Spectral Gravity Formant Estimation for Phonetic Segmentation

Paper · 2026 · Michael S. Yantosca and Albert M. K. Cheng

Summary

Modern end-to-end recognizers, built on transformers and CTC, optimize for orthographic accuracy and largely discard timing. This makes them poorly suited to phonetic work, where linguists need precise phone boundaries. The authors extend their earlier real-time segmentation system (Yantosca and Cheng, 2025), whose formant picker tended to place estimates below 1–2 kHz and missed the high-frequency energy of sibilants and other fricatives.

They replace that heuristic with a univariate Gaussian mixture fit to each pyknogram frame. The mixture is estimated by an expectation maximization (EM) procedure that stops when a time budget runs out. Its key novelty is a "spectral gravity" initialization, which treats spectral energy as mass whose pull weakens with the square of frequency distance. This favors dense energy clusters and counters EM's usual weaknesses: dependence on starting values, slow convergence and sensitivity to sparse data.

Evaluated zero-shot on Irish, Twi and Votic Mozilla Common Voice recordings, the method beats recent state-of-the-art systems on key clustering metrics and yields plausible alignments without any training. This matters for low-resource and endangered languages.

Key points

  • The core algorithm is deadline-bounded, amplitude-weighted univariate EM with spectral gravity initialization. It targets real-time deadlines of tens of milliseconds, and the resulting phone labels encode constituent formants as 64-bit integers.
  • The method is implemented in ARTIC/Phonotomizer. Experiments were re-run on v3.0.13 after a timestamp-counter frequency bug was fixed (corrected value 3.696 GHz), and aggregate results were essentially unchanged.
  • Data come from Mozilla Common Voice for Irish, Twi and critically endangered Votic. The gold-label sets (goldgrids-gle, goldgrids-twi, goldgrids-vot) were hand-transcribed by the first author alone.
  • Phonotomizer ran only in zero-shot mode. The Montreal Forced Aligner (MFA) baseline was evaluated only on training alignments.
  • Testing ran only on Ubuntu 24.04. AMD and aarch64 support (e.g., Raspberry Pi 500) was added provisionally but not fully benchmarked.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 worksSignal Processing · 493 works

Cite

Cite as Michael S. Yantosca and Albert M. K. Cheng. 2026. Spectral Gravity Formant Estimation for Phonetic Segmentation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1775

@inproceedings{yantosca2026spectral,
  title = {{Spectral Gravity Formant Estimation for Phonetic Segmentation}},
  author = {Michael S. Yantosca and Albert M. K. Cheng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1775},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1775}
}

References

44 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Paavo Alku, Jouni Pohjalainen, Martti Vainio, Anne-Maria Laukkanen, and Brad H. Story. 2013. Formant frequency estimation of high-pitched vowels using weighted linear prediction. The Journal of the Acoustical Society of America, 134(2):1295–1313. find
  2. Anonymous. 2022a. Software to translate audio to phonemic transcription. Reddit discussion. find
  3. Anonymous. 2022b. Transcribe to IPA (international phonetic alphabet). OpenAI Whisper forum discussion. find
  4. Anonymous. 2023. Speech to phonetic transcription: Does it exist? Reddit discussion. find
  5. Arm Limited. 2025a. CNTFRQ_EL0, Counter-timer Frequency Register. find
  6. Arm Limited. 2025b. CNTVCT_EL0, Counter-timer Virtual Count Register. find
  7. Bishnu S. Atal. 1975. Linear prediction of speech— recent advances with applications to speech analysis. Speech Recognition, pages 221–230. find
  8. Bishnu S. Atal and M.R. Schroeder. 1978. Linear prediction analysis of speech based on a pole-zero representation. The Journal of the Acoustical Society of America, 64(5):1310–1318. find
  9. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. Wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, 33:12449–12460.
  10. Paul Boersma and David Weenink. 2023. Praat: doing phonetics by computer. Software program. find
  11. John Parker Burg. 1975. Maximum Entropy Spectral Analysis. Stanford University. find
  12. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2021. Unsupervised cross-lingual representation learning for speech recognition. find
  13. Travis Downs. 2024. avx-turbo. GitHub repository. find
  14. Mario A. T. Figueiredo and Anil K. Jain. 2002. Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3):381–396. find
  15. Daniel Frisch and Uwe D. Hanebeck. 2021. Gaussian mixture estimation from weighted samples. In 2021 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pages 1–5. IEEE. find
  16. Sean A. Fulop. 2011. Speech Spectrum Analysis. Springer Science & Business Media. find
  17. Reshad Hosseini and Suvrit Sra. 2020. An alternative to EM for gaussian mixture models: batch and stochastic Riemannian optimization. Mathematical Programming, 181(1):187–223. find
  18. Intel Corporation. 2022. Intel® 64 and IA-32 Architectures Software Developer’s Manual. find
  19. Peter Ladefoged. 1993. A Course in Phonetics. Harcourt Brace Jovanovich. find
  20. Xinjian Li, Siddharth Dalmia, Juncheng Li, Matthew Lee, Patrick Littell, Jiali Yao, Antonios Anastasopoulos, David R. Mortensen, Graham Neubig, Alan W. Black, and Metze Florian. 2020. Universal phone recognition with a multilingual allophone system. In ICASSP 2020-2020 IEEE International Conference… find
  21. Xinjian Li, Siddharth Dalmia, Juncheng Li, Matthew Lee, Patrick Littell, Jiali Yao, Antonios Anastasopoulos, David R. Mortensen, Graham Neubig, Alan W. Black, and Metze Florian. 2021. Allosaurus. GitHub repository. find
  22. Richard F. Lyon. 2017. Human and machine hearing. Cambridge University Press. find
  23. Changxue Ma, Yves Kamp, and Lei F. Willems. 1993. Robust signal selection for linear prediction analysis of voiced speech. Speech Communication, 12(1):69–81. find
  24. Elena Markus and Fedor Rozhanskiy. 2022. Votic. In Marianne Bakró-Nagy, Johanna Laakso, and Elena Skribnik, editors, The Oxford Guide to the Uralic Languages, chapter 19, pages 330–349. Oxford University Press. find
  25. Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. 2017. Montreal forced aligner: Trainable text-speech alignment using Kaldi. In Proc. find
  26. Interspeech 2017, pages 498–502. find
  27. John D. McCalpin and Travis Downs. 2023. Find alternative to get_TSC_frequency for AMD processors. GitHub issue (low-overhead-timers). find
  28. Volodymyr Melnykov and Igor Melnykov. 2012. Initializing the EM algorithm in Gaussian mixture models with an unknown number of components. Computational Statistics & Data Analysis, 56(6):1381–1395. find
  29. Isaac Newton. 2009 [orig. 1686]. Philosophiae naturalis principia mathematica. Project Gutenberg. find
  30. Omnilingual ASR team, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, Kevin Chan, Chierh Cheng, Joe Chuang, Caley Droof, Mark Duppenthaler, Paul-Ambroise Duquenne, Alexander Erben, Cynthia Gao, Gabriel Mejia Gonzalez,… — [arXiv:2511.09690](https://arxiv.org/abs/2511.09690) find
  31. Pekka Paalanen, Joni-Kristian Kamarainen, Jarmo Ilo-nen, and Heikki Kälviäinen. 2006. Feature representation and discrimination based on Gaussian mixture model probability densities—practices and algorithms. Pattern Recognition, 39(7):1346–1358. find
  32. Alexandros Potamianos and Petros Maragos. 1995. Speech formant frequency and bandwidth tracking using multiband energy demodulation. In Proc. 1995 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), volume 1, pages 784–787. find
  33. Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, et al. 2024. Scaling speech technology to 1,000+ languages. Journal of Machine Learning Research, 25(97):1–52.
  34. Fedor Rozhanskiy and Elena Markus. 2017. On the mid and high non-labial vowels in luuditsa votic. Linguistica Uralica, 53(4):241–255. find
  35. Christine H. Shadle, Hosung Nam, and D.H. Whalen. 2016. Comparing measurement errors for formants in synthetic and natural vowels. The Journal of the Acoustical Society of America, 139(2):713–727. find
  36. Navid Shokouhi and John H. L. Hansen. 2017. Teager-Kaiser energy operators for overlapped speech detection. 25(5):1035–1047. find
  37. The Mozilla Foundation. 2025. Mozilla Common Voice. Orthographically annotated datasets. find
  38. Linus Torvalds, Patryk Wlazlyn, and Len Brown. 2025. Linux kernel source tree: linux/power/tools/turbostat. GitHub repository. find
  39. Yihong Wu and Pengkun Yang. 2020. Optimal estimation of Gaussian mixtures via denoised method of moments. The Annals of Statistics, 48(4):1981–2007. find
  40. Qiantong Xu, Alexei Baevski, and Michael Auli. 2021. Simple and effective zero-shot cross-lingual phoneme recognition. arXiv preprint arXiv:2109.11680. — [arXiv:2109.11680](https://arxiv.org/abs/2109.11680) find
  41. Michael S. Yantosca and Albert M.K. Cheng. 2025. Phonotomizer: A compact, unsupervised, online training approach to real-time, multilingual phonetic segmentation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12135–12147. find
  42. Tatu Ylonen. 2022. Wiktextract: Wiktionary as machine-readable structured data. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1317–1325. find
  43. Tatu Ylonen. 2025. Votic word forms. Online machine-readable dictionary. find
  44. Baibo Zhang, Changshui Zhang, and Xing Yi. 2004. Competitive EM algorithm for finite mixture models. Pattern recognition, 37(1):131–144. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yantosca-2026-spectral-gravity-formant-estimation-phonetic/