Elamur Research
☰
home / papers / 1997 / CogNIAC: high precision coreference with limited knowledge and linguistic resources

CogNIAC: high precision coreference with limited knowledge and linguistic resources

Summary

CogNIAC is a rule-based pronoun resolution system designed for settings where deep linguistic knowledge and rich resources are not available. Its guiding idea is to favour precision over recall: a pronoun is resolved only when one of a small, ordered set of high-confidence rules applies, and is otherwise left unresolved rather than guessed. The rules rely on shallow cues such as uniqueness of a compatible antecedent in the current and preceding sentences, reflexives, possessives and grammatical role. Because each rule is simple and its firing is explicit, every resolution decision can be traced back to the rule that produced it. The paper argues that such a conservative resolver is useful as a component in larger systems, where a wrong link costs more than a missing one. It is an early, influential example of the "precise rules first" design later generalised by multi-pass sieve coreference systems.

How Mova uses it

Effectiveness in Mova

The speaker sieve, whose rules are credited jointly to Lee et al. 2013 and this paper, is measured on the GUM test set (UD 2.18): 771 links at 90.7% precision, contributing +21.88 CoNLL F1 (without singletons, cumulative, on gold trees) and +21.99 on Mova's own parses. It is the first and most precise sieve. The share attributable to Baldwin 1997 alone is not measured separately.

👨‍🔬💥

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W97-1306.pdf

Summary

Pronoun resolution is often treated as requiring full world knowledge and inference. Breck Baldwin's CogNIAC challenges that view by focusing on a subset of pronouns that can be resolved without deep reasoning. It uses only shallow resources: sentence detection, part-of-speech tags, basal noun phrase brackets, gender and number information, and, in one setup, partial parses. The system is a small, ordered set of rules written in Perl. Its key feature is that it declines to resolve a pronoun when the context is ambiguous. Ambiguity is detected because the salience ranking is a partial order, so several candidates can tie and the system then leaves the pronoun alone. On narrative training data, this cautious strategy reached very high precision with moderate recall. The paper contrasts it with Hobbs' Naive Algorithm, which always ranks candidates totally and therefore always guesses. A second experiment within the MUC-6 system suggested that the approach carries over to other domains, although errors from other pipeline components lowered performance there. The work shows that precision-oriented, knowledge-light anaphora resolution is practical.

Key points

  • On roughly 200 training pronouns from narrative text, CogNIAC reached 97% precision (121/125) and 60% recall (121/201).
  • There are six core rules, applied in a fixed order:
    • unique antecedent in the whole discourse (8/0 correct/incorrect)
    • reflexive (16/1)
    • unique in current plus prior sentence (114/2)
    • possessive string match (4/1)
    • unique in current sentence (21/1)
    • subject-to-subject (11/0)
  • The baseline for comparison is Hobbs' Naive Algorithm (1976). Unlike CogNIAC, it imposes a total salience order and cannot notice ambiguity, for example in the Winograd "city council" sentence.
  • Experiment 2 embedded CogNIAC in the University of Pennsylvania MUC-6 system. One untested rule ("subject same clause") turned out to have low precision.
  • CogNIAC serves as the pronoun and common noun resolver in Penn's Camp software. Planned extensions include split antecedents, event reference, and verb-based selectional constraints, following Dagan 1990.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{baldwin1997cogniac,
  title = {{CogNIAC: high precision coreference with limited knowledge and linguistic resources}},
  author = {Baldwin, Breck},
  booktitle = {Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts},
  year = {1997},
  url = {https://aclanthology.org/W97-1306/}
}

References

14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baldwin, B., "CogNIAC: A discourse processing engine", University of Pennsylvania, Department of Computer and Information Science, Ph.D. dissertation. find
  2. Baldwin, B., Reynar, J., Collins, M., Eisner, J., Ratnaparkhi, A., Rosenzweig, J., Sarkar, A., Srinivas (1995), "University of Pennsylvania: Description of the University of Pennsylvania system used for MUC-6", Proceedings of Sixth Message Understanding Conference, November 1995. find
  3. Collins, M., "A New Statistical Parser based on Bigram Lexical Dependencies", Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, June, 1996. find
  4. Carter, D. (1986), "Common Sense Inference in a Focus-Guided Anaphor Resolver", Journal of Semantics, 4. find
  5. Chinchor, N., Sundheim, B. (1994) "Coreference Task Definition vl.l", supporting documentation for MUC-6 coreference task. find
  6. Dagan, I., Itai, A., (1990) A Statistical Filter for Resolving Pronoun References, Proceedings of 7th Israeli Symposium on Artificial Intelligence and Computer Vision. find
  7. Grosz, B. J, Joshi, A. K., and Weinstein, S. (1986) "Towards a Computational Theory of Discourse Interpretation" Ms. find
  8. Hobbs, J. (1976) "Pronoun Resolution", Research Report \#76-1, City College, City University of New York. find
  9. Hobbs, J. (1977) "38 Examples of Elusive Antecedents from Published Texts", Research Report \#77-2, City College, City University of New York. find
  10. C. Kennedy, B. Boguraev, "Anaphora for everyone: pronominal anaphora resolution without a parser." Proceedings of the 16th International Conference on Computational Linguistics COLING'96, Copenhagen, Denmark, 5-9 August 1996 find
  11. Lappin, S., Leass, H. (1994) "An Algorithm for Pronominal Anaphora Resolution", Computational Linguistics. find our card
  12. Mitkov R., "Pronoun resolution: the practical alternative", In S. Botley, T. McEnery (Eds) "Discourse Anaphora and Anaphor Resolution". University College London Press, 1997 find
  13. Sidner, C. L. (1986) "Focusing in the Comprehension of Definite Anaphora", Readings in Natural Language Processing, eds Grosz, Jones, Webber. Morgan Kaufman, Los Altos CA. find
  14. Winograd, T. 1972, "Understanding Natural Language", New York: Academic Press. find