Elamur Research⌕
☰
home / papers / 2003 / A Machine Learning Approach to Pronoun Resolution in Spoken Dialogue

A Machine Learning Approach to Pronoun Resolution in Spoken Dialogue

Summary

Pronouns in spoken dialogue are hard to resolve automatically. Many point back to clauses or verb phrases rather than noun phrases, and many have no identifiable antecedent at all, being expletive or vague. Strube and Müller adapt their decision-tree anaphora resolver, originally built for written text, to this setting. They annotated twenty Switchboard dialogues with markables for NPs, VPs, sentences and disfluencies. Candidate antecedents for it and that include S- and VP-markables drawn from the two preceding valid sentences, filtered by a rough approximation of the right frontier. Starting from standard NP-level and coreference-level features, the authors added features aimed at spoken dialogue and selected among them in a wrapper-style manner. These included verb argument preferences, agreement-sensitive distance measures and IR-style metrics. The verb-preference features were ultimately dropped. Simple distance measures proved more useful. The combined model reached an F-measure of 47.42, compared with 45.85 for the baseline. Unlike earlier work, the evaluation keeps non-referential pronouns, and the authors argue the result is comparable to Byron's (2002) system restricted to domain-independent knowledge. To the authors' knowledge, this is the first implemented corpus-based resolver that also handles non-NP antecedents.

Key points

  • Corpus: 20 randomly selected Switchboard dialogues, totalling 30,810 tokens, 3,275 sentences, 1,771 turns and 16,601 annotated markables. Some attributes come from the Penn Treebank version of Switchboard.
  • Feature set: 25 features in total. Nine are new for dialogue: antecedent type (NP/S/VP), verb preferences for NP/VP/S arguments taken from a verb list built over 553 Switchboard dialogues, mdist_3mf3p, mdist_3n, ante_tfidf, ante_ic and wdist_ic.
  • Combined results for all pronouns:
    • Baseline: precision 61.71, recall 36.48, F 45.85.
    • Final model: precision 56.74, recall 40.72, F 47.42.
  • Main error source: pronouns without antecedents. The model assigns wrong antecedents to many of them, and most are vague rather than true expletives.
  • Comparison with Byron (2002): her TRAINS93 system scores 72% with domain-specific semantics and 51% without, but it assumes only referential pronouns as input. The authors consider their approach more robust and easier to port to new domains.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

47 papers of our library cite it (of the 140673 whose bibliographies we read); 18 of them have a page here, 29 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1075096.1075118

@inproceedings{strube2003machine,
  title = {{A Machine Learning Approach to Pronoun Resolution in Spoken Dialogue}},
  author = {Strube, Michael and Müller, Christoph},
  booktitle = {Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics},
  year = {2003},
  doi = {10.3115/1075096.1075118},
  url = {https://doi.org/10.3115/1075096.1075118}
}

References

18 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asher, Nicholas (1993). Reference to Abstract Objects in Discourse. Dordrecht, The Netherlands: Kluwer. find
  2. Breiman, Leo, Jerome H. Friedman, Charles J. Stone & R.A. Olshen (1984). Classification and Regression Trees. Belmont, Cal.: Wadsworth and Brooks/Cole. find
  3. Briscoe, Ted & John Carroll (1997). Automatic extraction of subcategorization from corpora. In Proceedings of the 5th Conference on Applied Natural Language Processing, Washington, D.C., 31 March – 3 April 1997, pp. 356–363. find
  4. Byron, Donna K. (2002). Resolving pronominal reference to abstract entities. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Penn., 7–12 July 2002, pp. 80–87. find
  5. Byron, Donna K. & James F. Allen (1998). Resolving demonstrative pronouns in the TRAINS93 corpus. In New Approaches to Discourse Anaphora: Proceedings of the Second Colloquium on Discourse Anaphora and Anaphor Resolution (DAARC2), pp. 68–81. find
  6. Dagan, Ido & Alon Itai (1991). A statistical filter for resolving pronoun references. In Y.A. Feldman & A. Bruckstein (Eds.), Artificial Intelligence and Computer Vision, pp. 125– 135. Amsterdam: Elsevier. find
  7. Eckert, Miriam & Michael Strube (2000). Dialogue acts, synchronising units and anaphora resolution. Journal of Semantics, 17(1):51–89. find
  8. Ihaka, Ross & Robert Gentleman (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5:299–314. find
  9. Kohavi, Ron & George H. John (1997). Wrappers for feature subset selection. Artificial Intelligence Journal, 97(1-2):273–324. find
  10. Müller, Christoph, Stefan Rapp & Michael Strube (2002). Applying Co-Training to reference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Penn., 7–12 July 2002, pp. 352–359. find
  11. Ng, Vincent & Claire Cardie (2002). Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Penn., 7–12 July 2002, pp. 104–111. find our card
  12. Paul, Michael, Kazuhide Yamamoto & Eiichiro Sumita (1999). Corpus-based anaphora resolution towards antecedent preference. In Proc. of the 37th ACL, Workshop Coreference and Its Applications, College Park, Md., 1999, pp. 47–52. find
  13. Soon, Wee Meng, Hwee Tou Ng & Daniel Chung Yong Lim (2001). A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521– 544. find our card
  14. Strube, Michael, Stefan Rapp & Christoph Müller (2002). The influence of minimum edit distance on reference resolution. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, Philadelphia, Pa., 6–7 July 2002, pp. 312–319. find
  15. Tetrault, Joel R. (2001). A corpus-based evaluation of centering and pronoun resolution. Computational Linguistics, 27(4):507–520. find our card
  16. Therneau, Terry M. & Elizabeth J. Atkinson (1997). An introduction to recursive partitioning using the RPART routines. Technical Report: Mayo Foundation. Distributed with the RPART package. find
  17. Vilain, Marc, John Burger, John Aberdeen, Dennis Connolly & Lynette Hirschman (1995). A model-theoretic coreference scoring scheme. In Proceedings of the 6th Message Understanding Conference (MUC-6), pp. 45–52. San Mateo, Cal.: Morgan Kaufmann. find our card
  18. Webber, Bonnie L. (1991). Structure and ostension in the interpretation of discourse deixis. Language and Cognitive Processes, 6(2):107–135. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/strube-2003-machine-learning-approach-pronoun-resolution/