Elamur Research⌕
☰
home / papers / 2025 / English-based acoustic models perform well in the forced alignment of two English-based Pacific Creoles

English-based acoustic models perform well in the forced alignment of two English-based Pacific Creoles

Paper · 2025 · Sam Passmore, Lila San Roque, Kirsty Gillespie et al.

Summary

The paper asks whether forced-alignment tools built for well-resourced languages are good enough to prepare minority-language recordings for sociophonetic vowel research, or whether dedicated models must be built. The authors use newly archived field corpora of two English-based Pacific Creoles: Tok Pisin (59 hours, 147 speakers) and Bislama (38.5 hours, 60 speakers). The recordings are conversational and contain background noise. With the Montreal Forced Aligner (MFA), they compare four acoustic models:

  • a Language-Specific model,
  • a pooled Pacific Creole model,
  • the large English MFA model,
  • that English model adapted with Creole data.

Each model is scored against about 5 minutes of hand-corrected gold boundaries per language, using four measures: Data Retention, Boundary Performance, Midpoint Retention and Formant Accuracy. The English models do acceptably on both Creoles. In vowel environments classed as 'Highly Reliable', they match human performance. Custom models worked better for Bislama than for Tok Pisin. This matters because archived data in under-studied languages could be aligned quickly without building a new model for each language.

Key points

  • The corpora were recorded in the field between 2023 and 2025. Topics were life stories, natural disasters, health and safety. Stretches of code-switching were marked and excluded.
  • The English MFA model (McAuliffe and Sonderegger, 2024) is trained on about 3,614 hours of Indian, Nigerian, UK and US English. The English-Adapted variants total roughly 3,673 hours (Tok Pisin) and 3,653 hours (Bislama).
  • The four metrics are:
    • Data Retention: the correct phone is identified.
    • Boundary Performance: the boundary falls within 20 ms of gold.
    • Midpoint Retention: the aligned interval contains the gold midpoint.
    • Formant Accuracy: the formant differs from gold by less than 10%.
  • Error analysis found the English model consistently undershot vowel midpoints in Bislama. Errors clustered around nasals, whispered or fast speech, rain noise, and Praat formant-tracking failures. Formant errors were 12 for Bislama and 4 for Tok Pisin with the English model, and 13 and 6 with the adapted model.
  • When hand-correcting alignments, the authors advise checking final boundaries, vowels produced in non-modal voice or at unusual speed, and especially vowels before nasal consonants.

Interesting ideas

  • I1. Scoring results separately by how segmentable a vowel's surroundings are ('Highly Reliable' vs. harder contexts) shows where a model is already near human level and where it still fails.
  • I2. A borrowed model is judged with several complementary tests, from identifying the correct unit up to whether the downstream measurement (formants) is still usable.
  • I3. A model from the historically related lexifier language can be used as-is, or lightly adapted by treating the new variety's speakers as "new speakers".
  • I4. A tiny hand-corrected gold sample (about 5 minutes per language) is enough to compare candidate models fairly.
  • I5. Every error is traced to a concrete cause (nasal context, noise, swapped labels, measurement-tool failure), separating aligner faults from faults in the measuring step.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation could be split by construction difficulty, e.g. plain SVO clauses vs. coordination or ellipsis, so silver-data quality is reported per stratum instead of as one average.
  • I2 (tree): Parser variants could be scored not only on LAS but on whether the frames and roles derived from their trees stay unchanged.
  • I3 (tree): English parsers could be applied to dialectal or creole-influenced English, with adaptation on a small treebank instead of training from scratch.
  • I4 (learn): A small, carefully corrected gold set per domain (fables, word problems) could rank witness parsers before their agreement is trusted as silver data.
  • I5 (learn): Self-induced repair rules could be grounded in error logs tagged by cause, so tokenizer faults are not mistaken for parser faults.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of MelbourneMelbourne · Australia102 works · 22,618 citations🎓Australian National UniversityCanberra · Australia24 works · 6,522 citations🏛️Defence Science and Technology GroupCanberra · Australia2 works · 3 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksSpeech & spoken language · 519 works

Cite

Cite as Sam Passmore, Lila San Roque, Kirsty Gillespie, Saurabh Kumar Nath, Kira Davey, Keira Mullan, Tim Cawley, Jennifer Biggs, Rosey Billington, Bethwyn Evans, Nick Thieberger, Nicholas Evans and Danielle Barth. 2025. English-based acoustic models perform well in the forced alignment of two English-based Pacific Creoles. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.1505

@inproceedings{passmore2025english,
  title = {{English-based acoustic models perform well in the forced alignment of two English-based Pacific Creoles}},
  author = {Sam Passmore, Lila San Roque, Kirsty Gillespie, Saurabh Kumar Nath, Kira Davey, Keira Mullan, Tim Cawley, Jennifer Biggs, Rosey Billington, Bethwyn Evans, Nick Thieberger, Nicholas Evans and Danielle Barth},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1505},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1505}
}

References

31 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Danielle Barth. 2023. Tok pisin corpus. find
  2. Danielle Barth, James Grama, Simon Gonzalez Ochoa, and Catherine Travis. 2020. Using forced alignment for sociophonetic research on a minority language. Selected Papers from NWAC47, 25(2). find
  3. Rosey Billington, Hywel Stoakes, and Nick Thieberger. 2021. The pacific expansion: Optimizing phonetic transcription of archival corpora. In Interspeech 2021, pages 4029–4033. ISCA. find
  4. Paul Boersma and David Weenink. 2025. Praat: Doing phonetics by computer. Eleanor Chodroff, Emily P Ahn, and Hossep Dolatian. In Press. Comparing language-specific and cross-language acoustic models for low-resource phonetic forced alignment. Language Documentation and Conservation. find
  5. Terry Crowley. 1990. Beach-la-Mar to Bislama: The emergence of a national language in Vanuatu. Oxford University Press. find
  6. Terry Crowley. 2004. Bislama reference grammar. Oceanic linguistics special publication (31). University of Hawaii Press, Honolulu. find
  7. Marianna Di Paolo and Malcah Yaeger-Dror. 2011. Sociophonetics: a student’s guide. Routledge, London. find
  8. Christian DiCanio, Hosung Nam, Douglas H. Whalen, H. Timothy Bunnell, Jonathan D. Amith, and Rey Castillo García. 2013. Using automatic alignment to analyze endangered language data: Testing the viability of untrained alignment. The Journal of the Acoustical Society of America, 134(3):2235– 2246. ↗
  9. Ksenia Gnevsheva. 2020. The role of style in the ethnolect: Style-shifting in the use of ethnolectal features in first- and second-generation speakers. International Journal of Bilingualism, 24(4):861–880. ↗
  10. Simon Gonzalez, James Grama, and Catherine E. Travis. 2020. Comparing the performance of forced aligners used in sociophonetic research. Linguistics Vanguard, 6(1). ↗
  11. Caroline Jones, Katherine Demuth, Weicong Li, and Andre Almeida. 2017. Vowels in the barunga variety of north australian kriol. In Interspeech 2017, pages 219–223. ISCA. find
  12. Alfred Kik, Martin Adamec, Alexandra Y. Aikhenvald, Jarmila Bajzekova, Nigel Baro, Claire Bowern, Robert K. Colwell, Pavel Drozd, Pavel Duda, Sen-tiko Ibalim, Leonardo R. Jorge, Jane Mogina, Ben Ruli, Katerina Sam, Hannah Sarvasy, Simon Saulei, George D. Weiblen, Jan Zrzavy, and Vojtech Novotny. 2021. Language and ethnobiological skills decline precipitously in papua new guinea, the world’s most linguistically diverse nation. Proceedings of the National Academy of Sciences, 118(22):e2100096118. ↗
  13. Don Kulick. 2019. A Death in the Rainforest: How a Language and a Way of Life Came to an End in Papua New Guinea. Algonquin Books of Chapel Hill, New York, UNITED STATES. find
  14. William Labov. 1963. The social motivation of a sound change. WORD, 19(3):273–309. ↗
  15. William Labov. 1994. Principles of linguistic change, volume 1: Internal factors. Blackwells, Cambridge, MA. find
  16. William Labov. 2001. Principles of linguistic change, volume 3: Cognitive and cultural factors, volume 3. Blackwells, Cambridge, MA. find
  17. William Labov. 2020. The regularity of regular sound change. Language, 96(1):42–59. ↗
  18. C.H. Lee and J.L. Gauvain. 1993. Speaker adaptation based on map estimation of hmm parameters. In 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 2, pages 558– 561 vol.2. find
  19. Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. 2017. Montreal forced aligner: Trainable text-speech alignment using kaldi. In Interspeeh 2017, volume 2017, pages 498–502. find
  20. Michael McAuliffe and Morgan Sonderegger. 2024. English mfa acoustic model v3.1.0. Technical report, https://mfa-models.readthedocs.io/acoustic/English/English MFA acoustic model v3_1_0.html. ↗
  21. Miriam Meyerhoff. 2013. Bislama. Oxford University Press, Oxford. find
  22. Alexis Michaud, Oliver Adams, Trevor Anthony Cohn, Graham Neubig, and Séverine Guillaume. 2018. Integrating automatic transcription into the language documentation workflow: Experiments with na data and the persephone toolkit. Language Documentation and Conservation. find
  23. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, and Petr Schwarz. 2011. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding. IEEE Signal Processing Society. find
  24. Frank Seifart, Nicholas Evans, Harald Hammarström, and Stephen C. Levinson. 2018. Language documentation twenty-five years on. Language, 94(4):e324– e345. ↗
  25. Geoff P. Smith and Jeff Siegel. 2013. Tok Pisin. Oxford University Press, Oxford. find
  26. Rolando Coto Solano, Sally Akevai Nicholas, and Samantha Wray. 2018. Development of natural language processing tools for cook islands māori. In Proceedings of the Australasian Language Technology Association Workshop 2018, page 26–33, Dunedin, New Zealand. Nick Thieberger. Bislama corpus. find
  27. Nick Thieberger and Amanda Harris. 2022. When your data is my grandparents singing. digitisation and access for cultural records, the pacific and regional archive for digital sources in endangered cultures (paradisec). Data Science Journal, 21(1). ↗
  28. Benjamin V. Tucker and Richard Wright. 2020. Introduction to the special issue on the phonetics of under-documented languages. The Journal of the Acoustical Society of America, 147(4):2741–2744. ↗
  29. Alice Turk, Satsuki Nakai, and Mariko Sugahara. 2012. Acoustic Segment Durations in Prosodic Research: A Practical Guide, pages 1–28. De Gruyter. find
  30. James A. Walker and Miriam Meyerhoff. 2020. Pivots of the caribbean? low-back vowels in eastern caribbean english. Linguistics, 58(1):109–130. ↗
  31. Steve Young, Gunnar Evermann, Mark Gales, Thomas Hain, Dan Kershaw, Xunying Liu, Gareth Moore, Julian Odell, Dave Ollason, and Dan Povey. 2002. The HTK book, volume 3.2. Cambridge University Engineering Department, Cambridge. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/passmore-2025-english-based-acoustic-models-perform/