Evaluating Automated and Manual Acquisition of Anaphora Resolution Strategies

Summary

The paper tackles anaphora resolution, a persistent bottleneck for applications such as information extraction and machine translation. Real systems must cope with noisy upstream analysis and with anaphoric phenomena that vary by language and domain. The authors previously built a Manually-Designed Resolver (MDR) that relies on hand-selected, hand-ordered discourse knowledge sources. Here they build a Machine Learning-based Resolver (MLR) and compare the two. They annotated Japanese newspaper articles about joint ventures with coreference links using their GUI annotation tool, DTTool. They then trained C4.5 decision trees on feature vectors that each describe an anaphor paired with a candidate antecedent. Several MLR variants beat the hand-built system, and with only 50 training texts the learned resolver already surpasses the MDR. Decision trees were chosen partly because they are readable, so the features the trees select can be compared with linguistic theories of Japanese zero pronouns. The work suggests that annotated corpora can replace costly manual engineering of discourse preferences.

Key points

Annotation scheme: The corpus uses a detailed SGML tag set covering definite NPs (prefixed by "dou", "ryou", "kono" or "sono"), proper-name anaphora (NAME), reflexives, and indexicals. It also covers zero pronouns, split into ZPRO and QZPRO, with semantic subtypes such as ORG and P.
Features: The trees use 66 lexical, syntactic, semantic and positional features, both unary and binary. These are computed automatically from the system's discourse markers, so some feature values are noisy.
Training variants: Three settings were varied:
- whether all pairs on the anaphoric chain (transitive closure) count as positive examples;
- whether the tree predicts the anaphoric type or only a yes/no answer;
- the pruning confidence factor (25, 50, 75 or 100%).
Learning curve: Performance levels off at an F-measure of about 77.4 once roughly 250 training examples are used.
Weak spots: Resolution of ZPRO-ORG and DNP-ORG is weaker, likely because the batch-mode MLR cannot pass along links it has already resolved. For QZPRO-ORG, the trees relied on topicalization, distance, semantic class and subjecthood, which partly overlaps with the ranking proposed by Walker et al.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Coreference resolution

Cited by

54 papers of our library cite it (of the 140673 whose bibliographies we read); 25 of them have a page here, 29 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/981658.981675

@inproceedings{aone1995evaluating,
  title = {{Evaluating Automated and Manual Acquisition of Anaphora Resolution Strategies}},
  author = {Aone, Chinatsu and Bennett, Scott William},
  booktitle = {33rd Annual Meeting of the Association for Computational Linguistics},
  year = {1995},
  doi = {10.3115/981658.981675},
  url = {https://doi.org/10.3115/981658.981675}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Chinatsu Aone and Scott W. Bennett. 1994. Discourse Tagging Tool and Discourse-tagged Multilingual Corpora. In Proceedings of International Workshop on Sharable Natural Language Resources (SNLR). find
Chinatsu Aone and Douglas McKee. 1993. Language-Independent Anaphora Resolution System for Understanding Multilingual Texts. In Proceedings of 31st Annual Meeting of the ACL. find
Chinatsu Aone, Sharon Flank, Paul Krause, and Doug McKee. 1993. SRA: Description of the SOLOMON System as Used for MUC-5. In Proceedings of Fourth Message Understanding Conference (MUC-5). find
Chinatsu Aone. 1994. Customizing and Evaluating a Multilingual Discourse Module. In Proceedings of the 15th International Conference on Computational Linguistics (COLING). find
Susan Brennan, Marilyn Friedman, and Carl Pollard. 1987. A Centering Approach to Pronouns. In Proceedings of 25th Annual Meeting of the ACL. find
Dennis Connolly, John D. Burger, and David S. Day. 1994. A Machine Learning Approach to Anaphoric Reference. In Proceedings of International Conference on New Methods in Language Processing (NEMLAP). find
Marti A. Hearst. 1994. Multi-Paragraph Segmentation of Expository Text. In Proceedings of 32nd Annual Meeting of the ACL. find
Jerry R. Hobbs. 1976. Pronoun Resolution. Technical Report 76-1, Department of Computer Science, City College, City University of New York. find
Megumi Kameyama. 1988. Japanese Zero Pronominal Binding, where Syntax and Discourse Meet. In Papers from the Second International Worksho on Japanese Syntax. find
Hans Kamp. 1981. A Theory of Truth and Semantic Representation. In J. Groenendijk et al., editors, Formal Methods in the Study of Language. Mathematical Centre, Amsterdam. find
Hiromi Nakaiwa and Satoru Ikehara. 1992. Zero Pronoun Resolution in a Japanese to English Machine Translation Systemby using Verbal Semantic Attribute. In Proceedings of the Fourth Conference on Applied Natural Language Processing. find
Rebecca J. Passonneau and Diane J. Litman. 1993. Intention-Based Segmentation: Human Reliability and Correlation with Linguistic Cues. In Proceedings of 31st Annual Meeting of the ACL. find
J. Ross quinlan. 1993. C~.5: Programs forMachine Learning. Morgan Kaufmann Publishers. find
Stephen Soderland and Wendy Lehnert. 1994. Corpus-driven Knowledge Acquisition for Discourse Analysis. In Proceedings of AAAI. find
Marilyn Walker, Masayo Iida, and Sharon Cote. 1994. Japanese Discourse and the Process of Centering. Computational Linguistics, 20(2). find
Marilyn A. Walker. 1989. Evaluating Discourse Processing Algorithms. In Proceedings of 27th Annual Meeting of the ACL. find

↑ back to top