Elamur Research⌕
☰
home / papers / 1997 / Automatic Rule Induction for Unknown-Word Guessing

Automatic Rule Induction for Unknown-Word Guessing

Paper · 1997 · Andrei Mikheev

Summary

POS taggers and parsers struggle with words missing from their lexicon, since they cannot look up which tags such words may take. Mikheev presents a fully automatic method for learning rules that predict an unknown word's POS-class from its initial and final segments. Training needs no hand-tagged corpus. It uses only a general-purpose lexicon, built from CELEX and converted to the Penn tag set, plus word frequencies from a raw corpus (Brown Corpus). Three complementary rule sets are induced statistically:

  • prefix morphological rules
  • suffix morphological rules, which confirm that a related stem exists in the lexicon
  • ending-guessing rules, which require no stem

These rules are applied as a cascade, most reliable first. Rule selection relies on precision, recall and coverage. The guessers were integrated into a stochastic and a rule-based tagger and tested on the Brown Corpus after removing hapax words from the lexicon. The author claims higher accuracy than earlier guessers, but the exact figures are not in the supplied excerpt. The approach is cheap to retrain for new tag sets and other affixational languages, such as Ukrainian.

Key points

  • Rules follow one schema, x:{b,e}[−S +M ?I-class → R-class]. For example, e[-ied +y ?(VB VBP) → (JJ VBD VBN)] tags specified as JJ/VBD/VBN when specify is listed as a verb.
  • The induced Ending* guesser has 2,196 rules, compared with 536 in the Xerox tagger's ending guesser.
  • Earlier reference points for unknown-word tagging accuracy in English:
    • about 66% for the hapax-distribution method of Dermatas and Kokkinakis (1995)
    • up to 85% for affix-based methods (Brill 1994; Weischedel et al. 1993)
  • The learned rule sets appear to be independent of domain and corpus but tied to the tag set. The morphological rules largely recover standard English affixes and resemble those in the ispell spell-checker.
  • The method is implemented as modular C++ and AWK tools and can be retrained for other tag sets and affixational languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom255 works · 68,771 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Andrei Mikheev. 1997. Automatic Rule Induction for Unknown-Word Guessing. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@article{mikheev1997automatic,
  title = {{Automatic Rule Induction for Unknown-Word Guessing}},
  author = {Andrei Mikheev},
  journal = {Computational Linguistics},
  year = {1997},
  url = {https://aclanthology.org/J97-3003/}
}

References

24 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baayen, Harald and Richard Sproat. 1995. Estimating lexical priors for low-frequency morphologically ambiguous forms. Computational Linguistics, 22(3):155-166. find
  2. Bod, Rens. 1995. Enriching Linguistics with Statistics: Performance Models of Natural Language. University of Amsterdam ILLC Dissertation Series 1995-14, Academishe Pers, Amsterdam. find
  3. Brill, Eric. 1994. Some advances in transformation-based part of speech tagging. In Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAAI-94). find
  4. Brill, Eric. 1995. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics, 21(4):543-565.
  5. Burnage, G. 1990. CELEX: A Guide for Users. Nijmegen: Centre for Lexical Information. find
  6. Church, Kenneth W. 1988. A stochastic parts program and noun-phrase parser for unrestricted text. In Proceedings of the Second Conference on Applied Natural Language Processing (ANLP-88), pages 136-143.
  7. Cutting, Doug, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1992. A practical part-of-speech tagger. In Proceedings of the Third Conference on Applied Natural Language Processing (ANLP-92), pages 133-140.
  8. Dermatas, Evangelos and George Kokkinakis. 1995. Automatic stochastic tagging of natural language texts. Computational Linguistics, 21(2):137-164. find
  9. DeRose, Stephen. 1988. Grammatical category disambiguation by statistical optimization. Computational Linguistics, 14(1):31-39.
  10. Efron, Bradley and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Brace&Co. find
  11. Francis, W. Nelson and Henry Kucera. 1982. Frequency Analysis of English Usage: Lexicon and Grammar. Houghton Mifflin, Boston. find
  12. Hayslett, H.T. 1981. Frequency Analysis of English Usage Lexicon and Grammar. Heinemann, London. find
  13. Kupiec, Julian. 1992. Robust part-of-speech tagging using a hidden Markov model. Computer Speech and Language, pages 225-241. find
  14. Lewis, David. 1991. Evaluating text categorization. Speech and Natural Language: Proceedings of a Workshop Held at Pacific Grove, CA. find
  15. Marcus, Mitchell, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-329.
  16. Mikheev, Andrei. 1996a. Learning part-of-speech guessing rules from lexicon: Extension to non-concatenative operations. In Proceedings of the 16th International Conference on Computational Linguistics (COLING-96), pages 770-775. find
  17. Mikheev, Andrei. 1996b. Unsupervised learning of word-category guessing rules. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics (ACL-96), pages 327-334. find
  18. Mikheev, Andrei and Liubov Liubushkina. 1995. Russian morphology: An engineering approach. Natural Language Engineering, 1(3):235–260. find
  19. Schmid, Helmut. 1994. Part of speech tagging with neural networks. In Proceedings of the International Conference on Computational Linguistics (COLING-94), pages 172-176. find
  20. Tzoukermann, Evelin, Dragomir R. Radev, and William A. Gale. 1995. Combining linguistic knowledge and statistical learning in French part of speech tagging. find
  21. In Proceedings of the EACL S1GDAT Workshop, pages 51-59. find
  22. Voutilainen, Atro. 1995. A syntax-based part-of-speech analyser. In Proceedings of the Seventh Conference of European Chapter of the Association for Computational Linguistics (EACL-95), pages 157-164. find
  23. Weischedel, Ralph, Marie Meteer, Richard Schwartz, Lance Ramshaw, and Jeff Palmucci. 1993. Coping with ambiguity and unknown words through probabilistic models. Computational Linguistics, 19(2):359-382.
  24. Zhang, Byoung-Tak and Yung-Taek Kim. 1990. Morphological analysis and synthesis by automated discovery and acquisition of linguistic rules. In Proceedings of the 13th International Conference on Computational Linguistics (COLING-90), pages 431-435. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mikheev-1997-automatic-rule-induction-unknown-word/