Elamur Research⌕
☰
home / papers / 2013 / Why is English so easy to segment?

Why is English so easy to segment?

Paper · 2013 · Abdellah Fourtassi, Benjamin Börschinger, Mark Johnson et al.

Summary

Unsupervised word segmentation systems do much better on English than on other languages. This paper asks whether that gap comes from models being tuned to English or from real properties of the languages. The authors compare English and Japanese, using both child-directed and adult-directed phonemic corpora. They run a Unigram segmentation model built in the Adaptor Grammar framework. English scores consistently higher in every setting.

To explain this, they introduce Normalized Segmentation Entropy (NSE). NSE measures how many plausible ways an utterance can be split when the true lexicon is already known, weighting each split with unigram word probabilities and normalizing by the number of possible boundaries. English turns out to be far less ambiguous than Japanese, and this ambiguity tracks segmentation performance. The authors link the difference to a trade-off between syllable complexity and word length.

Finally, they show that grammars learning several nested levels of structure jointly narrow the gap. However, the best level to treat as "the word" differs across corpora. The work matters because it reframes cross-language differences as an intrinsic ambiguity problem.

Key points

  • Corpora:
    • Bernstein-Ratner English CDS: 9,790 utterances, transcribed by Brent and Cartwright.
    • Hamasaki Japanese CDS: 10,000 utterances.
    • Buckeye English ADS and Corpus of Spontaneous Japanese ADS: 10,000 utterances each.
    • The CDS corpora come from CHILDES.
  • Setup: The Unigram model (equivalent to MBDP-1) was implemented as an Adaptor Grammar with Pitman-Yor adaptation. It was run twice per corpus for 2,000 iterations, followed by Minimum Bayes Risk decoding.
  • Unigram token F-scores:
    • English CDS 0.77 vs Japanese CDS 0.55.
    • English ADS 0.69 vs Japanese ADS 0.50.
    • Japanese lexicon F was only 0.33 (CDS).
  • Error patterns:
    • English errors mostly merge frequent collocations ("is it") and split off affixes like "-ing" and "-s".
    • Japanese errors split multi-syllabic words and misplace boundaries across words.
  • Three-level hierarchical grammar: An intermediate level gives the best scores everywhere except English CDS. For example, Japanese CDS rises to roughly 0.70 token F.

Interesting ideas

  • I1. Separate intrinsic task difficulty from model weakness by measuring ambiguity under a gold lexicon, so the residual error clearly belongs to the learner.
  • I2. Normalize an entropy over candidate analyses by the number of decision points, so long and short inputs become comparable.
  • I3. Diagnose systematic over- versus under-splitting by comparing boundary precision with boundary recall.
  • I4. Learn several nested structural levels jointly, then pick the level that best serves as the unit, instead of fixing one granularity in advance.
  • I5. Error analysis showed that frequent collocations get swallowed as single units, so frequency alone misleads unit discovery.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Compute an NSE-style entropy over alternative tokenizations of contractions, hyphenations and multiword expressions using the gold lexicon. This would show which errors are inherent and which are tokenizer faults.
  • I2 (tree): Score per-sentence parse ambiguity as entropy over the k-best UD trees divided by sentence length. This would flag the sentences whose attachments need explicit justification.
  • I3 (learn): Add boundary precision/recall gaps as an evaluation signal for tokenizer and chunking repairs. This reveals whether a self-induced rule is too eager or too timid.
  • I4 (world): When grouping tokens into entities and events, keep both finer and coarser candidate spans. Choose among them using downstream consistency of the world model rather than a fixed rule.
  • I5 (frames): When mining recurring verb–argument patterns from silver data, guard against frequent fixed chunks masquerading as single predicates. Do this by checking them against VerbNet roles.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksDevelopmental and Educational Psychology · 499 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksPsycholinguistics & language acquisition · 102 works

Cite

Cite as Abdellah Fourtassi, Benjamin Börschinger, Mark Johnson and Emmanuel Dupoux. 2013. Why is English so easy to segment?. Proceedings of the Fourth Annual Workshop on Cognitive Modeling and Computational Linguistics (CMCL).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{fourtassi2013why,
  title = {{Why is English so easy to segment?}},
  author = {Abdellah Fourtassi, Benjamin Börschinger, Mark Johnson and Emmanuel Dupoux},
  booktitle = {Proceedings of the Fourth Annual Workshop on Cognitive Modeling and Computational Linguistics (CMCL)},
  year = {2013},
  url = {https://aclanthology.org/W13-2601/}
}

References

35 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eleanor Olds Batchelder. 2002. Bootstrapping the lexicon: A computational model of infant speech segmentation. Cognition, 83(2):167–206. ↗
  2. N. Bernstein-Ratner. 1987. The phonology of parent-child speech. In K. Nelson and A. van Kleeck, editors, Children’s Language, volume 6. Erlbaum, Hillsdale, NJ. find
  3. Daniel Blanchard, Jeffrey Heinz, and Roberta Golinkoff. 2010. Modeling the contribution of phonotactic cues to the problem of word segmentation. Journal of Child Language, 37(3):487–511. ↗
  4. Benjamin Börschinger, Katherine Demuth, and Mark Johnson. 2012. Studying the effect of input size for Bayesian word segmentation on the Providence corpus. In Proceedings of the 24th International Conference on Computational Linguistics (Coling 2012), pages 325–340, Mumbai, India. Coling 2012 Organizing Committee. find
  5. Luc Boruta, Sharon Peperkamp, Benoı̂t Crabbé, and Emmanuel Dupoux. 2011. Testing the robustness of online word segmentation: Effects of linguistic diversity and phonetic variation. In Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics, pages 1–9, Portland, Oregon, USA, June. Association for Computational Linguistics. find
  6. M. Brent and T. Cartwright. 1996. Distributional regularity and phonotactic constraints are useful for segmentation. Cognition, 61:93–125. ↗
  7. M. Brent. 1999. An efficient, probabilistically sound algorithm for segmentation and word discovery. Machine Learning, 34:71–105. ↗
  8. Morten H Christiansen, Joseph Allen, and Mark S Seidenberg. 1998. Learning to segment speech using multiple cues: A connectionist model. Language and cognitive processes, 13(2-3):221–268. ↗
  9. Robert Daland and Janet B Pierrehumbert. 2011. Learning diphone-based segmentation. Cognitive Science, 35(1):119–155. ↗
  10. Margaret M. Fleck. 2008. Lexicalized phonotactic word segmentation. In Proceedings of ACL-08: HLT, pages 130–138, Columbus, Ohio, June. Association for Computational Linguistics. find
  11. Sharon Goldwater. 2007. Nonparametric Bayesian Models of Lexical Acquisition. Ph.D. thesis, Brown University. find
  12. Naomi Hamasaki. 2002. The timing shift of two-year-olds responses to caretakers yes/no questions. In Studies in language sciences (2)Papers from the 2nd Annual Conference of the Japanese Society for Language Sciences, pages 193–206. find
  13. Gaja Jarosz and J Alex Johnson. 2013. The richness of distributional cues to word boundaries in speech to young children. Language Learning and Development, (ahead-of-print):1–36. ↗
  14. Mark Johnson and Katherine Demuth. 2010. Unsupervised phonemic Chinese word segmentation using Adaptor Grammars. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 528–536, Beijing, China, August. Coling 2010 Organizing Committee. find
  15. Mark Johnson and Sharon Goldwater. 2009. Improving nonparameteric Bayesian inference: experiments on unsupervised word segmentation with adaptor grammars. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 317–325, Boulder, Colorado, June. Association for Computational Linguistics.
  16. Elizabeth K. Johnson and Peter W. Jusczyk. 2001. Word segmentation by 8-month-olds: When speech cues count more than statistics. Journal of Memory and Language, 44:1–20. ↗
  17. Mark Johnson, Thomas Griffiths, and Sharon Goldwater. 2007. Bayesian inference for PCFGs via Markov chain Monte Carlo. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 139–146, Rochester, New York. Association for Computational Linguistics.
  18. Mark Johnson, Katherine Demuth, Michael Frank, and Bevan Jones. 2010. Synergies in learning words and their referents. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R.S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 1018–1026. find
  19. Mark Johnson. 2008a. Unsupervised word segmentation for Sesotho using Adaptor Grammars. In Proceedings of the Tenth Meeting of ACL Special Interest Group on Computational Morphology and Phonology, pages 20–27, Columbus, Ohio, June. Association for Computational Linguistics. find
  20. Mark Johnson. 2008b. Using Adaptor Grammars to identify synergies in the unsupervised acquisition of linguistic structure. In Proceedings of the 46th Annual Meeting of the Association of Computational Linguistics, pages 398–406, Columbus, Ohio. Association for Computational Linguistics. find
  21. Peter W Jusczyk and Richard N Aslin. 1995. Infants detection of the sound patterns of words in fluent speech. Cognitive psychology, 29(1):1–23. ↗
  22. Peter W. Jusczyk, E. A. Hohne, and A. Bauman. 1999. Infants’ sensitivity to allophonic cues for word segmentation. Perception and Psychophysics, 61:1465–1476. ↗
  23. Brian MacWhinney. 2000. The CHILDES Project: Tools for Analyzing Talk. Transcription, format and programs, volume 1. Lawrence Erlbaum. find
  24. Kikuo Maekawa, Hanae Koiso, Sadaoki Furui, and Hitoshi Isahara. 2000. Spontaneous speech corpus of japanese. In proc. LREC, volume 2, pages 947–952. find
  25. Sven L Mattys, Peter W Jusczyk, Paul A Luce, James L Morgan, et al. 1999. Phonotactic and prosodic effects on word segmentation in infants. Cognitive psychology, 38(4):465–494. ↗
  26. Sven L Mattys, Peter W Jusczyk, et al. 2001. Do infants segment words or recurring contiguous patterns? Journal of experimental psychology, human perception and performance, 27(3):644–655. ↗
  27. Bruna Pelucchi, Jessica F Hay, and Jenny R Saffran. 2009. Learning in reverse: Eight-month-old infants track backward transitional probabilities. Cognition, 113(2):244–247. ↗
  28. J. Pitman and M. Yor. 1997. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25:855–900. ↗
  29. M. A. Pitt, L. Dilley, K. Johnson, S. Kiesling, W. Raymond, E. Hume, and Fosler-Lussier. 2007. Buckeye corpus of conversational speech. find
  30. J. Saffran, R. Aslin, and E. Newport. 1996. Statistical learning by 8-month-old infants. Science, 274:1926–1928. ↗
  31. Claude Shannon. 1948. A mathematical theory of communication. Bell System Technical Journal, 27(3):379–423. ↗
  32. Daniel Swingley. 2005. Statistical clustering and the contents of the infant vocabulary. Cognitive Psychology, 50:86–132. ↗
  33. A. Venkataraman. 2001. A statistical model for word discovery in transcribed speech. Computational Linguistics, 27(3):351–372. ↗
  34. Daniel J Weiss, Chip Gerfen, and Aaron D Mitchel. 2010. Colliding cues in word segmentation: the role of cue strength and general cognitive processes. Language and Cognitive Processes, 25(3):402–422. ↗
  35. Aris Xanthos. 2004. Combining utterance-boundary and predictability approaches to speech segmentation. In First Workshop on Psycho-computational Models of Human Language Acquisition, page 93. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fourtassi-2013-why-is-english-so-easy/