Elamur Research⌕
☰
home / papers / 2013 / A joint model of word segmentation and phonological variation for English word-final /t/-deletion

A joint model of word segmentation and phonological variation for English word-final /t/-deletion

Paper · 2013 · Benjamin Börschinger, Mark Johnson and Katherine Demuth

Summary

Most unsupervised word segmentation models assume every occurrence of a word is pronounced identically. Real speech does not behave this way. The authors target one well-studied case of variation: English word-final /t/-deletion, as when "west" is spoken as [wEs]. They extend the Bayesian Unigram and Bigram segmentation models of Goldwater et al. (2009). The extended model first generates hidden underlying word forms and then passes them through a probabilistic deletion rule whose probability depends on the phonological context. Segmentation and variation therefore live in one mathematically coherent generative model rather than in a pipeline. They test it on the Buckeye Corpus, both with known word boundaries and with boundaries inferred jointly. The Bigram version recovers deleted /t/s fairly well and learns sensible context-specific deletion rates. The Unigram version fails at both tasks. Joint modelling also slightly improves segmentation. The results show that dependencies between neighbouring words help explain away surface variation. The architecture is also a transparent testbed for adding further phonological factors.

Key points

  • The model is a hierarchical Dirichlet Process Bigram model over underlying forms, followed by a rule component P_R that may drop a word-final /t/.
    • The lexical prior only rules out vowel-less words.
    • Rule probabilities have Beta(1,1) priors.
  • Three context settings are compared:
    • uniform, with 1 probability;
    • right context (V, C or utterance boundary $), with 3 probabilities;
    • left-right context, with 6 probabilities.
  • With gold word boundaries and left-right context, the Bigram model reaches about 77% /t/-recovery F-score.
  • In joint segmentation on Buckeye, the Bigram model reaches:
    • about 55% /t/-recovery F-score;
    • about 72.6% segmentation F-score, compared with 70.1% for a Bigram model without a deletion rule (NO-ρ) and 74.0% on deletion-free data (NO-VAR).
    • Unigram segmentation stays near 54–55% in every condition.
  • Error analysis of "just":
    • 1746 of its 2442 tokens surface without /t/.
    • The model attributes only 1081 of those 1746 to an underlying "just".
    • This shows that a single context-level rate cannot capture item-specific behaviour of high-frequency words.

Interesting ideas

  • I1. Separating a hidden underlying form from the observed surface form, linked by an explicit context-conditioned rule, lets variation be explained rather than memorised as new lexical items.
  • I2. Word-to-word predictability is strong evidence for restoring material missing on the surface: "don you" is read as "don't you" because "you" so often follows "don't".
  • I3. Evaluation brackets the model between two references: a version without the rule on noisy data (lower reference) and a version run on clean, variation-free data (upper reference).
  • I4. Estimating a process's overall rate well does not guarantee correct decisions on individual tokens, especially for very frequent items.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Informal spellings like "gonna", "goin'" or "dunno" could be mapped to canonical lemmas through named rewrite rules with logged contexts, so each normalisation stays explainable.
  • I2 (tokens): When a token is ambiguous or truncated, the tagger could score candidate restorations by how well they fit the neighbouring words, not only by the token's own shape.
  • I3 (learn): Any repair rule or normalisation step could be judged against the same two baselines, showing how much of the gap to clean input the rule actually closes.
  • I4 (learn): Evaluation of self-induced repair rules should report per-item errors for high-frequency words alongside aggregate accuracy, and allow item-specific exceptions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Macquarie UniversitySydney · Australia42 works · 5,776 citations🎓Heidelberg UniversityHeidelberg · Germany38 works · 41,356 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksTokenization & segmentation · 119 works

Cite

Cite as Benjamin Börschinger, Mark Johnson and Katherine Demuth. 2013. A joint model of word segmentation and phonological variation for English word-final /t/-deletion. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{borschinger2013joint,
  title = {{A joint model of word segmentation and phonological variation for English word-final /t/-deletion}},
  author = {Benjamin Börschinger, Mark Johnson and Katherine Demuth},
  booktitle = {Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2013},
  url = {https://aclanthology.org/P13-1148/}
}

References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Noam Chomsky and Morris Halle. 1968. The Sound Pattern of English. Haper & Row, New York. find
  2. Andries W. Coetzee and Shigeto Kawahara. 2013. Frequency biases in phonological variation. Natural Language and Linguisic Theory, 31:47–89. ↗
  3. Andries W. Coetzee. 2004. What it Means to be a Loser: Non-Optimal Candidates in Optimality Theory. Ph.D. thesis, University of Massachusetts , Amherst. Laura Dilley, Amanda Millett, J. Devin McAuley, and Tonya R. Bergeson. to appear. Phonetic variation in consonants in infant-directed and adult-directed speech: The case of regressive place assimilation in word-final alveolar stops. Journal of Child Language. ↗
  4. Micha Elsner, Sharon Goldwater, and Jacob Eisenstein. 2012. Bootstrapping a unified model of lexical and phonetic acquisition. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 184–193, Jeju Island, Korea. Association for Computational Linguistics. find
  5. Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2009. A Bayesian framework for word segmentation: Exploring the effects of context. Cognition, 112(1):21–54. ↗
  6. Gregory R. Guy. 1991. Contextual conditioning in variable lexical phonology. Language Variation and Change, 3(2):223–39. ↗
  7. Mark Johnson and Sharon Goldwater. 2009. Improving nonparameteric Bayesian inference: experiments on unsupervised word segmentation with adaptor grammars. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 317–325, Boulder, Colorado, June. Association for Computational Linguistics.
  8. Mark Johnson, Thomas L. Griffiths, and Sharon Goldwater. 2007. Adaptor Grammars: A framework for specifying compositional nonparametric Bayesian models. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 641–648. MIT Press, Cambridge, MA. ↗
  9. Mark Johnson. 2008. Using Adaptor Grammars to identify synergies in the unsupervised acquisition of linguistic structure. In Proceedings of the 46th Annual Meeting of the Association of Computational Linguistics, pages 398–406, Columbus, Ohio. Association for Computational Linguistics. find
  10. Jason Naradowsky and Sharon Goldwater. 2009. Improving morphology induction by learning spelling rules. In Proceedings of the 21st international jont conference on Artifical intelligence, pages 1531– 1536, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  11. Radford M. Neal. 2003. Slice sampling. Annals of Statistics, 31:705–767. find
  12. Dennis Norris, James M. Mcqueen, Anne Cutler, and Sally Butterfield. 1997. The possible-word constraint in the segmentation of continuous speech. Cognitive Psychology, 34(3):191 – 243. ↗
  13. Mark A. Pitt, Laura Dilley, Keith Johnson, Scott Kiesling, William Raymond, Elizabeth Hume, and Eric Fosler-Lussier. 2007. Buckeye corpus of conversational speech. find
  14. Alan Prince and Paul Smolensky. 2004. Optimality Theory: Constraint Interaction in Generative Grammar. Blackwell. ↗
  15. Yee Whye Teh, Michael Jordan, Matthew Beal, and David Blei. 2006. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101:1566–1581. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/borschinger-2013-joint-model-word-segmentation-phonological/