Elamur Research⌕
☰
home / papers / 2008 / Unsupervised Multilingual Learning for Morphological Segmentation

Unsupervised Multilingual Learning for Morphological Segmentation

Summary

Snyder and Barzilay ask whether unsupervised morphological segmentation, which splits words into morphemes without labelled data, improves when several languages are analysed together rather than one at a time. They also ask whether related languages help more than distant ones. Their non-parametric hierarchical Bayesian model uses Dirichlet Process priors and is trained with Gibbs sampling. It segments two languages at once and induces "abstract morphemes": cross-lingual morpheme pairs that share a function and come from one language-independent process. Morphemes without a counterpart are explained by language-specific distributions. A string-edit prior based on Semitic consonant correspondences can raise the likelihood of cognates. The data are short parallel phrases from the Hebrew Bible and its Arabic, Aramaic and English translations. Compared with a monolingual version of the same model, joint learning cuts relative error by up to 24%. Without the phonetic prior, English helps about as much as a related Semitic language. With the prior, related pairs perform best. The work matters because it shows that no annotation in any language is needed to benefit from parallel data.

Key points

  • Corpus:
    • Built from the Hebrew Bible plus the Van Dyke Arabic translation, Targum Onkelos (Aramaic) and the Revised Standard Version (English).
    • Phrases were word-aligned with Giza++ and taken from independent sets in the alignment graph.
    • Phrase pairs seen fewer than five times were removed, leaving 6,139 short parallel phrases.
  • Gold standard and evaluation scope:
    • Arabic gold segmentations come from the Habash and Rambow (2005) analyzer.
    • Only Hebrew and Arabic are evaluated, since English and Aramaic have no gold standard.
    • Hebrew averages 1.8 morphemes per word and Arabic 1.7.
  • Evaluation protocol:
    • The metric is F-score over segmentation points.
    • The test set is a held-out random fifth of the phrases.
    • At test time the model gets no multilingual evidence.
  • Baselines: MORFESSOR, a random baseline weighted by the true segmentation frequency, and the MONOLINGUAL counterpart of the model.
  • Results:
    • Bilingual models clearly beat MONOLINGUAL, except for the Hebrew (+Aramaic) pair.
    • With the phonetic-correspondence prior (from Thackston, 1999), the Hebrew/Arabic pair reaches relative error reductions of 10% and 24%.

Interesting ideas

  • I1. Shared versus language-specific morphemes: The model separates cross-lingually shared units from "stray" units that belong to one language. This lets a joint model use correspondences without forcing every unit to align, which could suit any noisy parallel-data task.
  • I2. Cognate knowledge as a prior: Linguistic knowledge enters only as a prior over cognate likelihood, as a string-edit model built from a letter-correspondence chart. This is a cheap way to add expert knowledge to an unsupervised model, for example between related Slavic languages.
  • I3. Multilingual training, monolingual testing: Parallel data is used only during training, and testing is purely monolingual. This shows the learned model works on ordinary text without translations, which is a useful protocol for other cross-lingual methods.
  • I4. Asking which kind of partner language helps: The paper compares a related partner with a typologically distant but morphologically simple one, such as English. This framing helps when choosing auxiliary languages for low-resource work.
  • I5. Filtering rare aligned phrases: Dropping rare aligned phrase pairs is a simple way to reduce noise from bad translations and alignments before unsupervised training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics Multilingual & cross-lingual NLP Morphology

Institutions

🎓Massachusetts Institute of TechnologyCambridge · United States125 works · 60,031 citations

Cite

Cite as Benjamin Snyder and Regina Barzilay. 2008. Unsupervised Multilingual Learning for Morphological Segmentation. Proceedings of ACL-08: HLT.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{snyder2008unsupervised,
  title = {{Unsupervised Multilingual Learning for Morphological Segmentation}},
  author = {Benjamin Snyder and Regina Barzilay},
  booktitle = {Proceedings of ACL-08: HLT},
  year = {2008},
  url = {https://aclanthology.org/P08-1084/}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Meni Adler and Michael Elhadad. 2006. An unsupervised morpheme-based hmm for hebrew morphological disambiguation. In Proceedings of the ACL/CONLL, pages 665–672. our card
  2. M. M. Bravmann. 1977. Studies in Semitic Philology. Leiden:Brill. find
  3. Lyle Campbell. 2004. Historical Linguistics: An Introduction. Cambridge: MIT Press. find
  4. Mathias Creutz and Krista Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. ACM Transactions on Speech and Language Processing, 4(1). find
  5. Sajib Dasgupta and Vincent Ng. 2007. Unsupervised part-of-speech acquisition for resource-scarce languages. In Proceedings of the EMNLP-CoNLL, pages 218–227. find
  6. Mona Diab and Philip Resnik. 2002. An unsupervised method for word sense tagging using parallel corpora. In Proceedings of the ACL, pages 255–262. find
  7. Umberto Eco. 1995. The Search for the Perfect Language. Wiley-Blackwell. find
  8. Anna Feldman, Jirka Hana, and Chris Brew. 2006. A cross-language approach to rapid creation of new morpho-syntactically annotated resources. In Proceedings of LREC. find
  9. John A. Goldsmith. 2001. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):153–198. find
  10. Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2006. Contextual dependencies in unsupervised word segmentation. In Proceedings of the ACL, pages 673–680. find
  11. Alan Groves and Kirk Lowery, editors. 2006. The Westminster Hebrew Bible Morphology Database. Westminster Hebrew Institute, Philadelphia, PA, USA. find
  12. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambig uation in one fell swoop. In Proceedings of the ACL, pages 573–580. find
  13. Jiri Hana, Anna Feldman, and Chris Brew. 2004. A resource-light approach to russian morphology: Tagging russian using czech resources. In Proceedings of EMNLP, pages 222–229. find
  14. Radford M. Neal. 1998. Markov chain sampling methods for dirichlet process mixture models. Technical Report 9815, Dept. of Statistics and Dept. of Computer Science, University of Toronto, September. find
  15. Thomas Nelson, editor. 1952. The Holy Bible Revised Standard Version. Thomas Nelson & Sons. find
  16. Sebastian Padó and Mirella Lapata. 2006. Optimal constituent alignment with edge covers for semantic projection. In Proceedings of ACL, pages 1161 – 1168. find
  17. Eric Sven Ristad and Peter N. Yianilos. 1998. Learning string-edit distance. IEEE Trans. Pattern Anal. Mach. Intell., 20(5):522–532. find
  18. Monica Rogati, J. Scott McCarley, and Yiming Yang. 2003. Unsupervised learning of arabic stemming using a parallel corpus. In Proceedings of the ACL, pages 391–398. find
  19. Patrick Schone and Daniel Jurafsky. 2000. Knowledge-free induction of morphology using latent semantic analysis. In Proceedings of the CoNLL, pages 67–72. find
  20. Benjamin Snyder and Regina Barzilay. 2008. Cross-lingual propagation for morphological analysis. In Proceedings of AAAI. find
  21. Wheeler M. Thackston. 1999. Introduction to Syriac. Ibex Publishers. find
  22. Chenhai Xi and Rebecca Hwa. 2005. A backoff model for bootstrapping resources for non-english languages. In Proceedings of HLT/EMNLP, pages 851 – 858. find
  23. David Yarowsky, Grace Ngai, and Richard Wicentowski. 2000. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of HLT, pages 161–168. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/snyder-2008-unsupervised-multilingual-learning-morphological-segmentation/