Elamur Research⌕
☰
home / papers / 2013 / Minimally-Supervised Morphological Segmentation using Adaptor Grammars

Minimally-Supervised Morphological Segmentation using Adaptor Grammars

Summary

The paper asks how a very small amount of annotated data can improve unsupervised morphological segmentation. Its framework is Adaptor Grammars (AGs), a nonparametric Bayesian approach that combines a PCFG with a Pitman-Yor caching adaptor to infer latent word structure. The authors compare three regimes:

  • Unsupervised AGs, with three hand-designed grammars.
  • Semi-supervised AGs, where gold analyses supply fixed rule counts that constrain the MCMC sampler.
  • AG Select, a new model-selection method. A deliberately over-detailed binary metagrammar proposes many candidate split points. About 1000 gold-segmented word types are then used to choose, per language, which template of nonterminals defines the final morph boundaries.

Experiments cover English, Finnish, Turkish, German and Estonian. Evaluation uses a small dev set and the full Morpho Challenge test sets. To handle data at that scale, the learned posterior grammar is used to decode unseen words. Semi-supervision consistently beats unsupervised training. AG Select achieves the best average results and is competitive with leading semi-supervised systems. This matters because the grammar adapts to each language's morphology rather than imposing one design on all of them.

Key points

  • Three unsupervised baseline grammars are used: MorphSeq (a unigram over morphs), SubMorphs (adds a sub-morph layer) and Compounding (mimics Morfessor's implicit prefix–stem–suffix structure with compounds).
  • AG Select's metagrammar is binary-branching with depth four. The chosen "majority templates" differ by language, for example English M1 M21 M2211 M2212 M222 versus Estonian M11 M12 M2.
  • Test data are the Morpho Challenge sets, reaching up to three million word types. Decoding with the posterior grammar matches full transductive training in accuracy on average.
  • Comparison systems include unsupervised and semi-supervised Morfessor, unsupervised Morsel, and unsupervised AGs. Results were tuned for SBF1, but the method could optimise other metrics or downstream tasks.
  • The main drawback is cost: inference is cubic in word length, and AG Select is heavier than an unsupervised AG. The implementation builds on Mark Johnson's PYAG code.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics Morphology

 Institutions

🎓Tallinn University of TechnologyTallinn · Estonia4 works · 62,003 citations🎓University of EdinburghEdinburgh · United Kingdom202 works · 55,753 citations

Cite

Cite as Kairit Sirts and Sharon Goldwater. 2013. Minimally-Supervised Morphological Segmentation using Adaptor Grammars. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00225

@article{sirts2013minimally,
  title = {{Minimally-Supervised Morphological Segmentation using Adaptor Grammars}},
  author = {Kairit Sirts and Sharon Goldwater},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2013},
  doi = {10.1162/tacl\_a\_00225},
  url = {https://doi.org/10.1162/tacl_a_00225}
}

References

21 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mathias Creutz and Krista Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. ACM Transactions of Speech and Language Processing, 4(1):1–34, February. find
  2. Micha Elsner, Eugene Charniak, and Mark Johnson. 2009. Structured generative models for unsupervised named-entity clustering. In Proceedings of NAACL, pages 164–172. Association for Computational Linguistics. find
  3. John Goldsmith. 2001. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):153–198, June. find
  4. Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2006. Interpolating between types and tokens by estimating power-law generators. In Advances in Neural Information Processing Systems 18, pages 459–466, Cambridge, MA. find
  5. MIT Press. Eric A. Hardisty, Jordan Boyd-Graber, and Philip Resnik. 2010. Modeling perspective using adaptor grammars. In Proceedings of EMNLP, pages 284–292. Association for Computational Linguistics. find
  6. Zellig Harris. 1951. Structural Linguistics. University of Chicago Press. find
  7. Yun Huang, Min Zhang, and Chew Lim Tan. 2011. Non-parametric bayesian machine transliteration with synchronous adaptor grammars. In Proceedings of ACL: short papers - Volume 2, pages 534–539. Association for Computational Linguistics. find
  8. Mark Johnson and Katherine Demuth. 2010. Unsupervised phonemic chinese word segmentation using adaptor grammars. In Proceedings of COLING, pages 528– 536. Association for Computational Linguistics. find
  9. Mark Johnson and Sharon Goldwater. 2009. Improving nonparameteric bayesian inference: experiments on unsupervised word segmentation with adaptor grammars. In Proceedings of NAACL, pages 317–325. Association for Computational Linguistics. find
  10. Mark Johnson, Thomas L. Griffiths, and Sharon Goldwater. 2007. Adaptor grammars: A framework for specifying compositional nonparametric bayesian models. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 641–648. find
  11. MIT Press, Cambridge, MA. Mark Johnson. 2008a. Unsupervised word segmentation for sesotho using adaptor grammars. In Proceedings of ACL Special Interest Group on Computational Morphology and Phonology, pages 20–27. Association for Computational Linguistics. find
  12. Mark Johnson. 2008b. Using adaptor grammars to identify synergies in the unsupervised acquisition of linguistic structure. In Proceedings of ACL, pages 398–406. Association for Computational Linguistics. find
  13. Oskar Kohonen, Sami Virpioja, and Krista Lagus. 2010a. Semi-supervised learning of concatenative morphology. In Proceedings of ACL Special Interest Group on Computational Morphology and Phonology, pages 78–86. Association for Computational Linguistics. find
  14. Oskar Kohonen, Sami Virpioja, Laura Leppänen, and Krista Lagus. 2010b. Semi-supervised extensions to morfessor baseline. In Proceedings of the Morpho Challenge 2010 Workshop, pages 30–34. Aalto University School of Science and Technology. find
  15. Yoong Keok Lee, Aria Haghighi, and Regina Barzilay. 2011. Modeling syntactic context improves morphological segmentation. In Proceedings of CoNLL, pages 1–9. Association for Computational Linguistics. Percy Liang, Hal Daumé, III, and Dan Klein. 2008. Structure compilation: trading structure for fea… find
  16. Constantine Lignos. 2010. Learning from Unseen Data. In Mikko Kurimo, Sami Virpioja, and Ville T. Turunen, editors, Proceedings of the Morpho Challenge 2010 Workshop, pages 35–38. Aalto University School of Science and Technology. find
  17. Jim Pitman and Marc Yor. 1997. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25(2):855–900. find
  18. Hoifung Poon, Colin Cherry, and Kristina Toutanova. 2009. Unsupervised morphological segmentation with log-linear models. In Proceedings of NAACL, pages 209–217. Association for Computational Linguistics. our card
  19. Benjamin Snyder and Regina Barzilay. 2008. Unsupervised multilingual learning for morphological segmentation. In Proceedings of ACL, pages 737–745. Association for Computational Linguistics. our card
  20. Sebastian Spiegler and Christian Monson. 2010. Emma: A novel evaluation metric for morphological analysis. In Proceedings of COLING, pages 1029–1037. Association for Computational Linguistics. find
  21. Sze-Meng Jojo Wong, Mark Dras, and Mark Johnson. 2012. Exploring adaptor grammars for native language identification. In Proceedings of EMNLP, pages 699– 709. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sirts-2013-minimally-supervised-morphological-segmentation-adaptor/