Elamur Research⌕
☰
home / papers / 2000 / Importance of Pronominal Anaphora Resolution in Question Answering Systems

Importance of Pronominal Anaphora Resolution in Question Answering Systems

Summary

The paper asks whether open-domain Question Answering systems lose useful information when they ignore pronouns in documents. The authors first measure how often pronouns occur in ten text collections used in TREC-8 QA and in IR testing. Pronoun use is heavy in newswire and much lower in technical abstracts, where "it/its" dominates. They then build a full pipeline to test the effect. A Salton-style IR engine (idf weighting, Porter stemming, cosine similarity) retrieves documents. The SUPAR parser, which uses Slot Unification Grammar, resolves pronominal anaphora in both queries and documents without semantic resources. A sentence-extraction QA module ranks sentences, scoring a resolved pronoun with the weight of its antecedent. Tests on the LAT corpus compare the system with and without resolution. Resolution raises the share of correct answers from 29.03% to 54.84%. The gain is largest when query terms appear only as pronouns in the answer sentence. The authors conclude that pronoun resolution should be a standard part of open-domain QA, and that it is essential for noun-phrase extraction systems.

Key points

  • The pronominal reference ratio ranges from 55.20% in LAT down to 9.05% in CRANFIELD. News collections (LAT, FBIS, TIME, FT) lie between 35.96% and 55.20% and contain many third-person pronouns.
  • SUPAR combines a POS tagger, partial parsing, statistical knowledge, c-command and morphological agreement constraints, and preference heuristics. It does not use WordNet or other semantic knowledge.
  • Evaluation used 93 questions, the subset of 150 for which the IR step found the relevant document. These were split into groups A, B and C according to how pronouns relate to the answer.
  • Overall, target sentences in the system's output rose from 41.94% to 54.84%, and correct answers from 29.03% to 54.84%. For group C, target sentences rose from 29.03% to 67.74%.
  • On a rebalanced set of 83 questions, matched to LAT's pronoun ratio, the gains were similar: +10.84 points for target sentences and +22.89 points for correct answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 11 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1075218.1075288

@inproceedings{vicedo2000importance,
  title = {{Importance of Pronominal Anaphora Resolution in Question Answering Systems}},
  author = {Vicedo, José L. and Ferrández, Antonio},
  booktitle = {Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics},
  year = {2000},
  doi = {10.3115/1075218.1075288},
  url = {https://doi.org/10.3115/1075218.1075288}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eric Breck, John Burger, Lisa Ferro, David House, Marc Light, and Inderjeet Mani. 1999. find
  2. A Sys Called Quanda. In Eighth Text REtrieval Conference (TRE, 1999). find
  3. Gordon V. Cormack, Charles L. A. Clarke, Christopher R. Palmer, and Derek I. E. Kisman. 1999. Fast Automatic Passage Ranking (MultiText Experiments for TREC-8). In Eighth Text REtrieval Conference (TRE, 1999). find
  4. Antonio Ferrandez, Manuel Palomar, and Lidia Moreno. 1998. Anaphora resolution in unre-striced texts with partial parsing. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Lingustics COLING-ACL . find
  5. Antonio Ferrandez, Manuel Palomar, and Lidia Moreno. 1999. An empirical approach to Spanish anaphora resolution. To appear in Machine Translation . find
  6. David A. Hull. 1999. Xerox TREC-8 Question Answering Track Report. In Eighth Text REtrieval Conference (TRE, 1999). find
  7. Kevin Humphreys, Robert Gaizauskas, Mark Hepple, and Mark Sanderson. 1999. University of Sheeld TREC-8 Q&A System. In Eighth Text REtrieval Conference (TRE, 1999). 18 0 9 27 find
  8. Julian Kupiec, 1999. MURAX: Finding and Organising Answers from Text Search , pages 311{ 331. Kluwer Academic, New York. find
  9. Dan Moldovan, Sanda Harabagiu, Marius Pasca, Rada Mihalcea, Richard Goodrum, Roxana G^rju, and Vasile Rus. 1999. LASSO: A Tool for Sur ng the Answer Net. In Eighth Text REtrieval Conference (TRE, 1999). find
  10. Thomas S. Morton. 1999. Using Coreference in Question Answering. In Eighth Text REtrieval Conference (TRE, 1999). find
  11. Douglas W. Oard, Jianqiang Wang, Dekang Lin, and Ian Soboro . 1999. TREC-8 Experiments at Maryland: CLIR, QA and Routing. In Eighth Text REtrieval Conference (TRE, 1999). find
  12. John Prager, Dragomir Radev, Eric Brown, Anni Coden, and Valerie Samn. 1999. The Use of Predictive Annotation for Question Answering. In Eighth Text REtrieval Conference (TRE, 1999). find
  13. Gerard A. Salton. 1989. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer . Addison Wesley, New York. find
  14. Amit Singhal, Steve Abney, Michiel Bacchiani, Michael Collins, Donald Hindle, and Fernando Pereira. 1999. ATT at TREC-8. In Eighth Text REtrieval Conference (TRE, 1999). find
  15. Toru Takaki. 1999. NTT DATA: Overview of system approach at TREC-8 ad-hoc and question answering. In Eighth Text REtrieval Conference (TRE, 1999). find
  16. TREC-8. 1999. Eighth Text REtrieval Conference . find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vicedo-2000-importance-pronominal-anaphora-resolution-question/