Elamur Research⌕
☰
home / papers / 2002 / Unsupervised Discovery of Morphemes

Unsupervised Discovery of Morphemes

Paper · 2002 · Mathias Creutz and Krista Lagus

Summary

The authors propose an unsupervised method for automatically segmenting words into morpheme-like units. Their main target is languages with rich agglutinative morphology, Finnish in particular. In such languages a word-based language model (for example, an n-gram model in speech recognition) suffers from data sparsity, because the number of word forms is very large. The model treats a word as an arbitrarily long sequence of "morphs" and does not distinguish between stems and affixes. Two methods are compared. The first is an online algorithm that recursively splits a word into two parts and optimizes a cost function based on the MDL principle (the cost of the corpus plus the cost of the morph lexicon). The second is batch sequential segmentation with an ML criterion. Quality is evaluated by comparing the segmentation with an existing linguistic morphological analysis, on Finnish and English data. Goldsmith's Linguistica serves as the baseline. The recursive MDL method performed best, especially for Finnish: it yields a more compact morph lexicon and more correct segmentations. For English the margin is small. The work is significant as an approach to morphological segmentation without annotated data, which is useful for low-resource languages.

Key points

  • Two methods: Recursive MDL (online, binary recursive splitting, lexicon cost of 5 bits per character) and Sequential ML (batch training with Viterbi segmentation). They are compared with Linguistica.
  • Morph lexicon size: for Finnish, Rec. MDL yields 6,302 morphs, Seq. ML 10,977, and Linguistica 22,075. For English, the figures are 3,836, 4,888 and 8,153, respectively.
  • Manual evaluation of 2,500 Finnish tokens: fully correct segmentations amount to 49.6% for Rec. MDL, 47.3% for Seq. ML and 43.1% for Linguistica. Seq. ML has the most incorrect boundaries, at 37.4%.
  • Typical errors: the MDL method does not split very frequent words and oversegments rare ones. Seq. ML tends toward oversegmentation (affecti + on + at + e). Linguistica often leaves Finnish compound words unsplit.
  • Limitations and plans: the model does not take context (allomorphy) or morpheme order (morphotactics) into account. The authors plan to transfer the MDL-like cost to the sequential method. The English data were taken from the Brown corpus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 405 works

💡 Concepts 2Key ideas and methods explained on our concept pages.Natural language processing · 131 worksMorphology · 50 works

Cite

Cite as Mathias Creutz and Krista Lagus. 2002. Unsupervised Discovery of Morphemes. Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1118647.1118650 · arXiv cs/0205057

@inproceedings{creutz2002unsupervised,
  title = {{Unsupervised Discovery of Morphemes}},
  author = {Mathias Creutz and Krista Lagus},
  booktitle = {Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning},
  year = {2002},
  doi = {10.3115/1118647.1118650},
  eprint = {cs/0205057},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/1118647.1118650}
}

References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jerome Bellegarda. 2000. Exploiting latent semantic information in statistical language modeling. Proceedings of the IEEE, 88(8):1279–1296. find
  2. Michael R. Brent. 1999. An efficient, probabilistically sound algorithm for segmentation and word discovery. Machine Learning, 34:71–105. find
  3. Carl de Marcken. 1995. The unsupervised acquisition of a lexicon from continuous speech. Technical Report A.I. Memo 1558, MIT Artificial Intelligence Lab., Cambridge, Massachusetts. find
  4. Carl de Marcken. 1996. Linguistic structure as composition and perturbation. In Meeting of the Association for Computational Linguistics. find
  5. Hervé Déjean. 1998. Morphemes as necessary concept for structures discovery from untagged corpora. In Workshop on Paradigms and Grounding in Natural Language Learning, pages 295–299, Adelaide, Jan. 22. find
  6. Sabine Deligne and Frédéric Bimbot. 1997. Inference of variable-length linguistic and acoustic units by multigrams. Speech Communication, 23:223–241. find
  7. John Goldsmith. 2001. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):153–198. find
  8. Linguistica affect affect + ing affect + ing + ly affect + ion affect + ion + ate affect + ion + s affect + s eläinlääkäri eläinlääkäri + lle eläinmuseo + n eläinmuseo + on eläinpuisto + n eläinpuisto + sta eläintarh + an Yu Hua. 2000. Unsupervised word induction using MDL criterion. In… find
  9. Fred Karlsson. 1987. Finnish Grammar. WSOY, Juva, second edition. find
  10. Fred Karlsson. 1998. Yleinen Yliopistopaino/Helsinki University Press. kielitiede. find
  11. Chunyu Kit and Yorick Wilks. 1999. Unsupervised learning of word boundary with description length gain. In Proceedings of CoNLL99 ACL Workshop, Bergen. find
  12. Kimmo Koskenniemi. 1983. Two-level morphology: A general computational model for word-form recognition and production. Ph.D. thesis, University of Helsinki. find
  13. A. Norman Redlich. 1993. Redundancy reduction as a strategy for unsupervised learning. Neural Computation, 5:289–304. find
  14. Jorma Rissanen. 1989. Stochastic complexity in statistical inquiry. World Scientific Series in Computer Science, 15:79–93. find
  15. Patrick Schone and Daniel Jurafsky. 2000. Knowledge-free induction of morphology using latent semantic analysis. In Proceedings of CoNLL-2000 and LLL-2000, pages 67–72, Lisbon. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/creutz-2002-unsupervised-discovery-morphemes/