Elamur Research⌕
☰
home / papers / 2010 / Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French

Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French

Paper · 2010 · Mohammed Attia, Jennifer Foster, Deirdre Hogan et al.

Summary

The paper examines how a generative constituency parser should tag words it never saw, or saw only rarely, in training. This matters most for morphologically rich languages and small treebanks. The authors train a Berkeley-style latent-variable PCFG (PCFG-LA) on the Penn Arabic Treebank (ATB), the WSJ part of the Penn-II Treebank and the French Treebank. They compare three ways of handling rare words:

  • Generic symbol: a language-neutral baseline that maps all rare words to one UNKNOWN terminal.
  • Hand-crafted signatures: rare words are mapped to morphological classes. English uses the Berkeley parser's classes, French uses prior work, and Arabic uses new heuristics based on affixes and morphotactics.
  • Learnt signatures: affix classes are chosen automatically by ranking candidate affixes with information gain.

Morphological signatures help most for Arabic and least for English. The Arabic gain of over 3 F-points produces state-of-the-art results for that treebank. Affix signatures learnt automatically perform about as well as hand-written ones. This means the one language-specific part of such a parser could in principle be induced from annotated data alone.

Key points

  • Unknown-word rates: on the development sets, the share of words absent from training is 5.40% for Arabic, 2.83% for English and 5.98% for French. When training sizes are equalised, Arabic has the worst problem and English the mildest.
  • Simple UNKNOWN model: treating words seen at most once as unknown works best (dev F: Arabic 79.53, English 89.47, French 83.88). Raising the cutoff to 10 hurts every language.
  • Learnt signature lists: the information-gain method selected 24 English suffixes, 48 French suffixes and 38 Arabic prefixes and suffixes.
  • Test F-scores (no signature / hand-crafted / information gain):
    • Arabic: 78.34 / 81.59 / 81.33
    • English: 89.48 / 89.65 / 89.77
    • French: 83.74 / 85.77 / 85.55
  • Comparison and setup: on the same Arabic data, Bikel's parser scores 76.50 and the Berkeley parser without signatures scores 75.28. All experiments use gold tokenisation and sentences of up to 40 words.

Interesting ideas

  • I1. Rather than pooling every unseen word into one bucket, back off each one to a class named by its visible affixes (e.g. UNK-ed), so the tag guess depends on the word's form.
  • I2. The rare-word cutoff is a real trade-off: reserving more probability for unknowns damages statistics for known-but-rare words, so the cutoff should be tuned rather than set high.
  • I3. Ranking candidate affixes by information gain for POS prediction gives signature lists nearly as good as expert-written ones, with no linguistic knowledge required.
  • I4. Comparing against gold-tag parsing reveals how much of the parsing error comes from tagging mistakes; for French the gap was striking.
  • I5. The authors measure unknown-word rates on training sets cut to equal size, which separates true language difficulty from mere treebank size.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): The en tagger could map out-of-vocabulary words to affix-signature classes, keeping the guess explainable ("tagged VBD because of -ed").
  • I2 (learn): When training the perceptron parser or tagger, sweep the rare-word threshold on development data instead of hard-coding it.
  • I3 (tokens): Unknown-word classes for the en tagger could be induced and refreshed automatically from treebank data, with the ranked list kept as an inspectable artifact.
  • I4 (tree): Running gparse and the perceptron parser with gold tags as an upper-bound control would show whether tagger fixes or parser fixes are the better investment.
  • I5 (learn): Evaluations across treebanks or UD dialects should control for training size before conclusions are drawn about domain or dialect difficulty.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Dublin City UniversityDublin · Ireland70 works · 9,447 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.English · 2,263 worksSemitic languages · 2,908 worksFrench · 70 works

Cite

Cite as Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph Le Roux, Lamia Tounsi and Josef van Genabith. 2010. Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French. Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{attia2010handling,
  title = {{Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French}},
  author = {Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph {Le Roux}, Lamia Tounsi and Josef {van Genabith}},
  booktitle = {Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages},
  year = {2010},
  url = {https://aclanthology.org/W10-1408/}
}

References

25 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne Abeillé, Lionel Clément, and François Toussenel, 2003. Treebanks: Building and Using Parsed Corpora, chapter Building a Treebank for French. Kluwer, Dordrecht. find
  2. Abhishek Arun and Frank Keller. 2005. Lexicalization in crosslinguistic probabilistic parsing: The case of French. In ACL. The Association for Computer Linguistics. find
  3. Kenneth R. Beesley and Lauri Karttunen. 2003. Finite State Morphology. CSLI studies in computational linguistics. find
  4. Kenneth R. Beesley. 1998. Arabic morphology using only finite-state operations. In The Workshop on Computational Approaches to Semitic Languages.
  5. Ann Bies and Mohammed Maamouri. 2003. Penn Arabic Treebank guidelines. Technical Report TB-1-28-03. find
  6. Dan Bikel. 2004. On the Parameter Space of Generative Lexicalized Parsing Models. Ph.D. thesis, University of Pennslyvania. find
  7. Marie Candito and Benoit Crabbé. 2009. Improving generative statistical parsing with semi-supervised word clustering. In Proceedings of IWPT’09. find
  8. Marie Candito, Benoı̂t Crabbé, and Djamé Seddah. 2009. On statistical parsing of French with supervised and semi-supervised strategies. In Proceedings of the EACL 2009 Workshop on Computational Linguistic Aspects of Grammatical Inference, pages 49–57, Athens, Greece, March. find
  9. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the Annual Meeting of the North American Association for Computational Linguistics (NAACL-00), pages 132–139, Seattle, Washington.
  10. Benoı̂t Crabbé and Marie Candito. 2008. Expériences d’analyse syntaxique statistique du français. In Actes de TALN. find
  11. Yoav Goldberg, Reut Tsarfaty, Meni Adler, and Michael Elhadad. 2009. Enhancing unlexicalized parsing performance using a wide coverage lexicon, fuzzy tag-set mapping, and EM-HMM-based lexical probabilities. In EACL, pages 327–335. The Association for Computer Linguistics. find
  12. Nizar Habash, Owen Rambow, and Ryan Roth. 2009. Mada+tokan: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization. In Proceedings of the 2nd International Conference on Arabic Language Resources and Tools (MEDAR). find
  13. Nizar Habash. 2008. Four techniques for online handling of out-of-vocabulary words in arabic-english statistical machine translation. In Proceedings of Association for Computational Linguistics, pages 57–60. find
  14. Zhongqiang Huang and Mary Harper. 2009. Self-training pcfg grammars with latent annotations across languages. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore, August. find
  15. Mark Johnson. 1998. PCFG models of linguistic tree representations. Computational Linguistics, 24(4):613–632.
  16. Dan Klein and Chris Manning. 2003. Accurate unlexicalised parsing. In Proceedings of the 41st Annual Meeting of the ACL. find
  17. Mohammed Maamouri and Ann Bies. 2004. Developing an Arabic Treebank: Methods, guidelines, procedures, and tools. In Workshop on Computational Approaches to Arabic Script-based Languages, COLING.
  18. Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: Annotating predicate argument structure. In Proceedings of the 1994 ARPA Speech and Natural Language Workshop, pages 114–119, Princeton, New Jersey.
  19. Takuya Matsuzaki, Yusuke Miyao, and Jun’ichi Tsujii. 2005. Probabilistic CFG with latent annotations. In Proceedings of the 43rd Annual Meeting of the ACL, pages 75–82, Ann Arbor, June.
  20. Rani Nelken and Stuart M. Shieber. 2005. Arabic diacritization using weighted finite-state transducers. In ACL-05 Workshop on Computational Approaches to Semitic Languages.
  21. Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In Proceedings of HLT-NAACL, Rochester, NY, April.
  22. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact and interpretable tree annotation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL, Sydney, Australia, July.
  23. Slav Petrov. 2009. Coarse-to-Fine Natural Language Processing. Ph.D. thesis, University of California at Berkeley, Berkeley, CA, USA. find
  24. Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In Proceedings of the International Conference on New Methods in Language Processing (NeMLaP-1), pages 44–49. find
  25. Huihsin Tseng, Daniel Jurafsky, and Christopher Manning. 2005. Morphological features help POS tagging of unknown words across language varieties. In Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/attia-2010-handling-unknown-words-statistical-latent/