Elamur Research⌕
☰
home / papers / 2005 / Handling Biographical Questions with Implicature

Handling Biographical Questions with Implicature

Paper · 2005 · Donghui Feng and Eduard Hovy

Summary

Feng and Hovy address a weakness of standard pipeline QA systems: they take a question's hidden assumptions for granted. Asked when Beethoven married, such a system dutifully hunts for a date, though he never married. Asked for Reagan's wife, it returns one name when there were two. The authors add a "cascading guidance" step that runs before answer search, after the person name and answer type are known. For spouse and child questions in their web-based biography QA system PowerBio, they retrieve the top N Google pages about the person and keep only sentences that mention both the person and a topic keyword. From these they take unigram, bigram and trigram features in a window around the keywords. Two classifiers then run in sequence: the first decides whether any answer exists, and the second decides whether to expect one answer or several. They compare Decision Tree (C5.0), Naïve Bayes, SVM and ME. Trigram features work best, and adding more documents gives little benefit. The work matters because it treats false or narrowing presuppositions as a step to detect explicitly, rather than leaving them to answer ranking.

Key points

  • The motivation draws on TREC 2003, where most systems answered "NIL" correctly only about 10–30% of the time for questions that had no answer.
  • Two question types are studied, SPOUSE_QUESTION and CHILD_QUESTION. The data set covers 75 persons across training and testing.
  • Each feature score multiplies the conditional probability of the last word in the N-gram by the N-gram's frequency in the retrieved documents.
  • For first-level Child classification with Naïve Bayes:
    • trigram features reach about 86.6% precision (top 10 or top 30 documents);
    • bigram features reach up to about 82.9%;
    • unigram features stay around 24–35%;
    • 4-grams were judged too sparse to be useful.
  • Out of 49,320 scanned bios, only 2 stated outright that the subject never attended school. This shows how rarely text states negative facts.

Interesting ideas

  • I1. The presupposition check is a separate gate placed before answer extraction: first decide whether the asked-about fact exists at all, then decide how many values to expect, and only then search. This turns "no answer" and "several answers" into explicit outcomes instead of ranking failures.
  • I2. Expected cardinality ("one or many?") is a decision made before extraction, and that decision then guides how many answers to pull. It prevents a system from stopping at the first hit for multi-valued attributes such as spouses or children.
  • I3. Evidence is gathered only from a narrow window around topic keywords in sentences that mention the target, which cuts noise from the rest of the document. This is a cheap way to focus statistical evidence on the relevant relation.
  • I4. Absent facts are almost never stated explicitly, and the authors suggest cross-checking related facts against each other, such as marriage against children. Inferring absence from what is present can therefore beat searching for explicit negation.
  • I5. The authors test whether more retrieved data helps and find a plateau, with slight degradation for bigrams at 50–70 documents. Testing data size as an explicit variable is a useful habit.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could mark the existential and uniqueness presuppositions of a wh-question as checkable claims and send them to verification before any answer slot is filled.
  • I2 (world): the entity store could carry a cardinality expectation per relation, such as spouse being possibly multiple over time, so the world model does not wrongly collapse several values into one.
  • I3 (learn): when harvesting silver examples for a relation, restricting attention to clauses near a trigger word could keep the training signal cleaner.
  • I4 (answer): QA readers on fables and tales could answer "none" by consistency checks over the world of the text, for example finding no marriage event for a character, instead of searching for a negative statement.
  • I5 (learn): evaluations of silver-data growth should include size sweeps, so that extra noisy material is shown to add signal before it is kept.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Southern CaliforniaLos Angeles · United States162 works · 35,836 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Implicature · 138 works

Cite

Cite as Donghui Feng and Eduard Hovy. 2005. Handling Biographical Questions with Implicature. Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{feng2005handling,
  title = {{Handling Biographical Questions with Implicature}},
  author = {Donghui Feng and Eduard Hovy},
  booktitle = {Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
  doi = {10.3115/1220575.1220650},
  url = {https://doi.org/10.3115/1220575.1220650}
}

References

34 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Boser, B.E., Guyon, I. and Vapnik, V. 1992. A training algorithm for optimal margin classifiers. Proceedings of the ACM COLT 1992. find
  2. Chang, C. and Lin, C. 2001. LIBSVM — A library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm/ ↗
  3. Chu-Carroll, J., Czuba, K., Prager, J., and Ittycheriah, A. 2003. In question answering, two heads are better than one. Proceedings of HLT-NAACL-2003. find
  4. Clark, S., Steedman, M. and Curran, J.R. 2004. Object-extraction and question-parsing using CCG. Proceedings EMNLP-2004, pages 111-118, Barcelona, Spain. find
  5. Dumais, S., Banko, M., Brill, E., Lin, J., and Ng, A. 2002. Web question answering: is more always better? Proceedings of SIGIR-2002. find
  6. Echihabi, A. and Marcu, D. 2003. A noisy channel approach to question answering. Proceedings of ACL-2003. find
  7. Fleischman, M., Hovy, E.H., and Echihabi, A. 2003. Offline strategies for online question answering: answering questions before they are asked. Proceedings of ACL-2003. find
  8. Gazdar, G. 1979. Pragmatics: Implicature, presupposition, and logical form. New York: Academic Press. find
  9. Harabagiu, S. 2001. Just-In-Time Question Answering. Invited talk in Proceedings of the Sixth Natural Language Processing Pacific Rim Symposium 2001. find
  10. Hermjakob, U., Echihabi, A., and Marcu, D. 2002. Natural language based reformulation resource and web exploitation for question answering. Proceedings of TREC-2002. find
  11. Hermjakob, U., Hovy, E.H., and Lin, C. 2000. Knowledge-based question answering. TREC-2000. find
  12. Hovy, E.H., Gerber, L., Hermjakob, U., Junk, M., and Lin, C. 2000. Question answering in Webclopedia. Proceedings of TREC-2000. find
  13. Hovy, E.H., Hermjakob, U., Lin, C., and Ravichandran, D. 2002. Using knowledge to facilitate factoid answer pinpointing. Proceedings of COLING-2002. find
  14. Hsu, C.-W., Chang, C.-C., and Lin, C.-J. 2003. A Practical Guide to Support Vector Classification. Available at: http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide. pdf. ↗
  15. Levinson, S. 1983. Pragmatics. Cambridge University Press. find
  16. Lita L.V. and Carbonell, J. 2004. Instance-based question answering: a data driven approach. Proceedings of EMNLP 2004. find
  17. Magnini, B., Romagnoli, S., Vallin, A., Herrera, J., Peñas, A., Peinado, V., Verdejo, F., Rijke, M. 2003. The Multiple Language Question Answering Track at CLEF 2003. CLEF 2003: 471-486. find
  18. Mann, G. and Yarowsky, D. 2005. Multi-field information extraction and cross-document fusion. Proceedings of ACL-2005. find
  19. Moldovan, D., Clark, D., Harabagiu, S., and Maiorano, S. 2003. Cogex: A logic prover for question answering. Proceedings of ACL-2003. find
  20. Moldovan, D., Harabagiu, S., Pasca, M., Mihalcea, R., Girju, R., Goodrum, R., and Rus, V. 2000. The structure and performance of an open-domain question answering system. Proceedings of ACL-2000. find
  21. Nyberg, E. et al. 2003. A multi strategy approach with dynamic planning. Proceedings of TREC-2003. find
  22. Och, F. J.and Ney, H. 2002. Discriminative training and maximum entropy models for statistical machine translation. Proceedings of ACL 2002 pp. 295-302.
  23. Pasca, M. and Harabagiu, S. 2001. High Performance Question/Answering. Proceedings of SIGIR-2001. find
  24. Prager, J. M., Chu-Carroll, J., and Czuba, K.W.. 2004. Question answering using constraint satisfaction. Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL'04). find
  25. Quinlan, J. R. 1993. C4.5: Programs for machine learning.
  26. Morgan Kaufmann, San Mateo, CA, 1993. find
  27. Ravichandran, D. and Hovy, E.H. 2002. Learning Surface Text Patterns for a Question Answering System. Proceedings of ACL-2002.
  28. Saquete, E., Martínez-Barco, P., Muñoz, R., and Vicedo, J.L. 2004. Splitting complex temporal questions for question answering systems. Proceedings of ACL'04. find
  29. Schiffman, B., Mani, I., and Concepcion, K.J. 2001. Producing biographical summaries: combining linguistic knowledge with corpus statistics. Proceedings of ACL/EACL-2001. ↗
  30. Soricut, R. and Brill, E. 2004. Automatic question answering: beyond the factoid. Proceedings of HLT/NAACL-2004, Boston, MA. find
  31. Voorhees, E.M. 2003. Overview of the trec 2003 question answering track. Proceedings of TREC-2003. find
  32. Xu, J., Licuanan, A., Weischedel, R. 2003. TREC 2003 QA at BBN: Answering Definitional Questions. Proceedings of TREC 2003. find
  33. Yang, H., Chua, T.S., Wang, S., and Koh, C.K. 2003. Structured use of external knowledge for eventbased open domain question answering. Proceedings of SIGIR-2003. find
  34. Zhou, L., Ticrea, M., and Hovy, E.H. 2004. Multi-document biography summarization. Proceedings of EMNLP-2004. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/feng-2005-handling-biographical-questions-implicature/