Evaluating a Focus-Based Approach to Anaphora Resolution

Summary

The paper tests whether a theoretically motivated, focus-based method for pronoun resolution works on real newswire text, rather than only on the small hand-picked examples common in earlier literature. The authors build on Sidner's focusing algorithm, as extended by Azzam (1996), and integrate it into the coreference module of LaSIE, Sheffield's MUC-6/MUC-7 Information Extraction system. Focus registers are updated after each elementary event (roughly a simple clause) instead of after each sentence, which allows pronouns to find antecedents within the same sentence. Separate interpretation rules handle agent pronouns, non-agent pronouns, and possessive/reciprocal/reflexive pronouns. Evaluated with the MUC-7 corpora and scorer, the focus-based version performs slightly worse than LaSIE's original simple heuristics. The authors attribute this mainly to its dependence on accurate and complete parses, and to its narrower set of candidate antecedents. The study matters because it gives one of the first corpus-scale quantitative checks of focusing theory. It also exposes weaknesses in that theory's assumptions and points toward hybrid approaches.

Key points

Data: MUC-7 coreference training set (60 texts, about 581 words each) and blind evaluation set (20 texts, about 605 words each), scored with the MUC scorer based on Vilain et al. (1995).
Baseline without pronoun resolution reached an f-measure of 52.6% on training and 55.7% on evaluation. LaSIE's simple heuristics raised this to 64.1% and 62.3%, respectively.
The focus-based algorithm reached f = 61.9% on training and 60.4% on evaluation, below the simple heuristics. Recall on evaluation was 53.3% versus 56.0%.
The algorithm adds two new registers to Sidner's original set: AFS (actor focus stack) and Intra-AFL (candidate foci from the current elementary event). It also sets an initial "expected focus" from the first clause.
Corpus examples show the focus is not always the preferred antecedent. Neither clause-level nor sentence-level focus updating clearly won. The authors suggest combining focus rules with the heuristics, or learning interpretation-rule priorities from data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/980845.980858

@inproceedings{azzam1998evaluating,
  title = {{Evaluating a Focus-Based Approach to Anaphora Resolution}},
  author = {Azzam, Saliha and Humphreys, Kevin and Gaizauskas, Robert},
  booktitle = {36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1},
  year = {1998},
  doi = {10.3115/980845.980858},
  url = {https://doi.org/10.3115/980845.980858}
}

References

11 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

S. Azzam, K. Humphreys, and R. Gaizauskas. 1998. Coreference resolution in a multilingual information extraction system. In Proceedings of the First Language Resources and Evaluation Conference (LREC). Linguistic Coreference Workshop. find
S. Azzam. 1995. Anaphors, P P s and Disambiguation Process for conceptual analysis. In Proceedings of l~th IJCAL S. Azzam. 1996. Resolving anaphors in embedded sentences. In Proceedings of 34th ACL. find
R. Gaizauskas, T. Wakao, K Humphreys, H. Cunningham, and Y. Wilks. 1995. Description of the LaSIE system. In Proceedings of MUC-6, pages 207-220. Morgan Kaufmann. find
R. Gaizauskas. 1995. XI: A Knowledge Representation Language Based on Cross-Classification and Inheritance. Technical Report CS-95-24, University of Sheffield. find
R. Grishman and B. Sundheim. 1996. Message Understanding Conference - 6: A brief history. In Proceedings of 16th IJCAI, pages 466-471. find
J.S. Gruber. 1976. Lexical structures in syntax and semantics. North-Holland. find
K. Humphreys, R. Gaizauskas, S. Azzam, C. Huyck, B. Mitchell, H. Cunningham, and Y. Wilks. 1998. Description of the LaSIE-II system. In Proceedings of MUC-7. Forthcoming. find
D. Lin. 1995. Description of the PIE System. In Proceedings of MUC-6, pages 113-126. find
Morgan Kaufmann. C. Sidner. 1981. Focusing for interpretation of pronouns. American Journal of Computational Linguistics, 7:217-231. find
G. Thurmair. 1996. AVENTINUS System Architecture. AVENTINUS project report LE1-2238. find
M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of MUC-6, pages 45-52. Morgan Kaufmann. find our card

📄 Evaluating a Focus-Based Approach to Anaphora Resolution