Elamur Research⌕
☰
home / papers / 1993 / Coping with Ambiguity and Unknown Words through Probabilistic Models

Coping with Ambiguity and Unknown Words through Probabilistic Models

Paper · 1993 · Ralph Weischedel, Marie Meteer, Richard Schwartz et al.

Summary

Rule-based NLP systems built for narrow database-query domains struggle with open-ended text such as newswire. That kind of text brings large vocabularies, frequent unknown words, long sentences and many ambiguities. Weischedel and colleagues at BBN ran a series of pilot experiments between 1990 and 1991. They tested whether corpus-trained probabilistic models can strengthen knowledge-based processing rather than replace it.

The experiments covered three areas:

  • Part-of-speech tagging. They built POST, a Hidden Markov Model tagger in bi-tag and tri-tag variants, trained on the Penn TREEBANK. It includes a probabilistic model of word features for guessing tags of unseen words.
  • Parse selection. A simple context-free probability model ranked competing parses from a unification-based parser and semantic interpreter.
  • Verb case frames. Probabilities guided which generalizations to draw from annotated examples, and the learned frames were evaluated on prepositional-phrase attachment.

Across all tasks, statistical scoring substantially reduced errors, even with modest training data. The work matters as an early, influential argument for hybrid statistical/symbolic NLP. It fed directly into BBN's PLUM text-extraction system.

Key points

  • Motivation came from the MUC-3 terrorism domain: about 20,000 distinct words in a 430,000-word corpus, with sentences averaging 29 words.
  • POST was trained on the University of Pennsylvania TREEBANK, which has roughly 4 million words and 47 tags (36 for words, 11 for punctuation).
  • For unknown words, the probabilistic feature model cut tagging error from 91.5% to 15%.
  • A context-free probability model trained on only 81 sentences reduced parse-selection error by a factor of 2–4 on independent test sets.
  • Used as a preprocessor to Grishman's Proteus system in MUC-3, POST roughly doubled Proteus's speed in one-best mode.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Rensselaer Polytechnic InstituteTroy · United States11 works · 4,917 citations

Also on the byline: Bowdoin College.

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as Ralph Weischedel, Marie Meteer, Richard Schwartz, Lance Ramshaw and Jeff Palmucci. 1993. Coping with Ambiguity and Unknown Words through Probabilistic Models. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@article{weischedel1993coping,
  title = {{Coping with Ambiguity and Unknown Words through Probabilistic Models}},
  author = {Ralph Weischedel, Marie Meteer, Richard Schwartz, Lance Ramshaw and Jeff Palmucci},
  journal = {Computational Linguistics},
  year = {1993},
  url = {https://aclanthology.org/J93-2006/}
}

References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abney, S. (1990). "Rapid incremental parsing with repair." In Proceedings, Sixth Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research. 1-9. find
  2. Ayuso, D. M.; Shaked, V.; and Weischedel, R. M. (1987). "An environment for acquiring semantic information." In Proceedings, 25th Annual Meeting of the Association for Computational Linguistics. 32–40. find
  3. Bates, M. (1989). "Rapid porting of the Parlance TM natural language interface." In Proceedings, Speech and Natural Language Workshop. Morgan Kaufmann Publishers. 83-88. find
  4. Baum, L. E., and Eagon, J. A. (1967). "An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model of ecology." Amer. Math Soc. Bulletin, 73, 360-362. find
  5. Chitrao, M. V., and Grishman, R. (1990). "Statistical parsing of messages." In Proceedings, Speech and Natural Language Workshop. Morgan Kaufmann Publishers. 263-266. find
  6. Church, K. (1988). "A stochastic parts program and noun phrase parser for unrestricted text." In Proceedings, Second Conference on Applied Natural Language Processing. 136-143. de Marcken, C. G. (1990). "Parsing the LOB corpus." In Proceedings, 28th Annual Meeting of the Association for Computationa… our card
  7. DeRose, S. J. (1988). "Grammatical category disambiguation by statistical optimization." Computational Linguistics, 14, 31-39. our card
  8. Grishman, R., and Sterling, J. (1989). "Preference semantics for message understanding." In Proceedings, Speech and Natural Language Workshop. Morgan Kaufmann Publishers. 71-74. find
  9. Grishman, R.; Hirschman, L.; and Nhan, N. T. (1986). "Discovery procedures for sublanguage selectional patterns: Initial experiments." Computational Linguistics, 12, 205-215. find
  10. Hindle, D. (1983). "User manual for Fidditch." Naval Research Laboratory Technical Memorandum \#7590-142. find
  11. Hindle, D., and Rooth, M. (1990). "Structural ambiguity and lexical relations." In Proceedings, Speech and Natural Language Workshop. Morgan Kaufman Publishers. 257-262. find
  12. Jelinek, F. (1985). "Self organizing language modeling for speech recognition." Unpublished Technical Report, IBM T. J. Watson Research Center, Yorktown Heights, NY. find
  13. Katz, S. M. (1987). "Estimation of probabilities from sparse data for the language model component of a speech recognizer." In IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. ASSP-35 No. 3. find
  14. Kuhn, R., and De Mori, R. (1990). "A cache-based natural language model for speech recognition." In IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 570-583. find
  15. Kupiec, J. (1989). "Augmenting a hidden Markov model for phrase-dependent word tagging." In Proceedings, Speech and Natural Language Workshop. Morgan Kaufmann Publishers. 92-98. find
  16. Marcus, M.; Santorini, B.; and Magerman (1990). "First steps towards an annotated database of American English." In Readings for Tagging Linguistic Information in a Text Corpus, edited by Langendoen and Marcus, Tutorial for the 28th Annual Meeting of the Association for Computational Linguistics. find
  17. Santorini, B. (1990). "Annotation manual for the Penn TREEBANK project." Technical Report, CIS Department, University of Pennsylvania. find
  18. Stallard, D. (1989). "Unification-based semantic interpretation in the BBN spoken language system." In Proceedings, Speech and Natural Language Workshop. find
  19. Sundheim, B. M. (1991). "Overview of the Third Message Understanding Evaluation and Conference." In Proceedings, Third Message Understanding Conference (MUC-3). find
  20. Weischedel, R. M.; Bobrow, R.; Ayuso, D. M.; and Ramshaw, L. (1989). "Portability in the Janus natural language interface." In Speech and Natural Language. Morgan Kaufmann Publishers. 112-117. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/weischedel-1993-coping-ambiguity-unknown-words-through/