Elamur Research⌕
☰
home / papers / 1999 / Unsupervised Models for Named Entity Classification

Unsupervised Models for Named Entity Classification

Summary

The paper tackles named entity classification: assigning an already-segmented proper name to Person, Organization, or Location while using almost no labeled data. Collins and Singer parsed 971,746 New York Times sentences with the Collins 96 parser. They kept only names found in two syntactic settings: with an appositive modifier, or as the complement of a preposition attached to a noun. Each example is split into two views. The spelling view covers the full string, the words it contains, capitalization and non-alphabetic patterns. The context view covers the appositive head or the preposition plus its noun. Seven seed rules provide the only supervision. Two algorithms exploit the fact that either view alone often reveals the type:

  • DL-CoTrain is a decision-list bootstrapper in the spirit of Yarowsky 95, adapted using ideas from Blum and Mitchell 98.
  • CoBoost is an AdaBoost-style method that directly minimizes a bound on how often the two view-specific classifiers disagree on unlabeled data.

With about 90,000 unlabeled examples, the methods exceed 91% accuracy. This shows that redundancy in unlabeled text can stand in for large annotated sets.

Key points

  • Seed rules: the only supervision is 7 rules.
    • New York, California and U.S. are locations.
    • Any name containing Mr. is a person.
    • Any name containing Incorporated is an organization.
    • Microsoft and I.B.M. are organizations.
  • Data: roughly 90,000 (spelling, context) pairs were extracted from 971,746 parsed NYT sentences.
  • Test set: 1,000 random examples were hand-labeled as 186 location, 289 person, 402 organization and 123 noise. Removing 38 day and month expressions left 962 items, 85 of them noise.
  • Evaluation: accuracy is reported twice, once counting noise items as errors and once excluding them. When CoBoost abstained (about 2% of the test set), the label defaulted to organization.
  • Decision-list scoring: rules are ranked by a smoothed estimate of label probability given a feature, with α = 0.1 and k = 3. In CoBoost's learning curves, coverage rises over iterations while the two classifiers keep agreeing on more than 94% of examples.

Interesting ideas

  • I1. Two-view feature design: each feature looks only at the spelling or only at the context, never both. This deliberately creates two independent views for co-training. The same design could apply wherever an item has an "internal form" and an "environment", such as place names, product names or trade entities in Ukrainian text.
  • I2. Agreement as a training signal: when an unlabeled example carries strong cues in both views, it implicitly says those cues share a label. Collecting many such hints lets a few seeds spread across the corpus without manual annotation.
  • I3. Disagreement as an objective: CoBoost turns "two classifiers should agree" into a smooth bound that can be optimized. This gives an otherwise heuristic bootstrapping process a principled target, and the idea is reusable for any multi-view semi-supervised task.
  • I4. Syntax as a precision filter: extraction is restricted to two precise patterns (appositives and noun-modifying PPs), which yields clean context predictors cheaply. The trade-off is coverage, and the authors themselves list this limitation.
  • I5. Honest handling of out-of-scope items: reporting accuracy both with and without noise items, and separating out trivially detectable cases such as dates, makes the evaluation of a closed-label classifier more transparent.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition

Cited by

188 papers of our library cite it (of the 140884 whose bibliographies we read); 17 of them have a page here, 171 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{collins1999unsupervised,
  title = {{Unsupervised Models for Named Entity Classification}},
  author = {Collins, Michael and Singer, Yoram},
  booktitle = {1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora},
  year = {1999},
  url = {https://aclanthology.org/W99-0613/}
}

References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Berland and E. Charniak. 1999. Finding Parts in Very Large Corpora. In Proceedings of the the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99). find our card
  2. D. M. Bikel, S. Miller, R. Schwartz, and R. Weischedel. 1997. Nymble: a High-Performance Learning Name-finder. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 194-201. find our card
  3. A. Blum and T. Mitchell. 1998. Combining Labeled and Unlabeled Data with Co-Training. In Proceedings of the l lth Annual Conference on Computational Learning Theory (COLT-98). find our card
  4. E. Brill. 1995. Unsupervised Learning of Disambiguation Rules for Part of Speech Tagging. In Proceedings of the " Third Workshop on Very Large Corpora. find
  5. S. Brin. 1998. Extracting Patterns and Relations from the World Wide Web. In WebDB Wokshop at EDBT '98. find
  6. M. Collins. 1996. A New Statistical Parser Based on Bigram Lexical Dependencies. Proceedings of the 34th Annual ----* .... ..... • .... find
  7. Meeting of the Association for Computational Linguistics, pages 184-191. find
  8. A.P. Dempster, N.M. Laird, and D.B. Rubin, (1977). Maximum Likelihood from Incomplete Data Via the EM Algorithm, Journal of the Royal Statistical Society, Ser B, 39, 1-38. find
  9. Y. Freund. Boosting a weak learning algorithm by majority. Information and Computation, 121 (2):256-285, 1995. find
  10. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55( 1 ): I 19-139, 1997. find
  11. M. Hearst. 1992. Automatic Acquisition of Hyponyms from Large Text Corpora. In Proceedings of the Fourteenth International Conference on Computational Linguistics. find
  12. Michael Kearns. Thoughts on hypothesis boosting. Unpublished manuscript, December 1988. find
  13. J. Lafferty. Additive Models, Boosting, and Inference for Generalized Divergences. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, 1999. find
  14. Proceedings of the Sixth Message Understanding Conference (MUC-6). Morgan Kaufmann, San Mateo, CA. find
  15. E. Riloff and J. Shepherd. 1997. A Corpus-Based Approach for Building Semantic Lexicons. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP-2). find
  16. E. Riloff and R. Jones. 1999. Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping. In Proceedings of the Sixteenth National Conference on Artificial Intelligence (AAAI-99). find
  17. R. E. Schapire. The strength of weak learnability. Machine Learning, 5(2): 197-227, 1990. find
  18. R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pages 80-91, 1998. To appear, Machine Learning. find
  19. L. G. Valiant. A theory of the learnable. Communications of the ACM, 27(11): 1134-1142, November 1984. find
  20. Yarowsky. 1995. Unsupervised Word Sense Disambiguation Rivaling Supervised Methods.In Proceedings of the 33rd find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/collins-1999-unsupervised-models-named-entity-classification/