Elamur Research⌕
☰
home / papers / 2011 / Learning the Information Status of Noun Phrases in Spoken Dialogues

Learning the Information Status of Noun Phrases in Spoken Dialogues

Paper · 2011 · Altaf Rahman and Vincent Ng

Summary

The paper tackles automatic information-status classification: deciding whether a noun phrase in a conversation is old (already mentioned or tied to the dialogue participants), mediated (inferable from earlier material or generally known), or new to the hearer. Using Nissim et al.'s annotated Switchboard dialogues, the authors build SVM classifiers on top of Nissim's (2006) seven morpho-syntactic features. They add two feature families. Lexical features stand in, shallowly, for world knowledge about which entities are commonly known. Structured features encode the NP's syntactic context through parse trees, used via kernels. They also split the data by dialogue rather than by instance, which makes evaluation more realistic. The extended classifier beats the Nissim-feature baseline by 8.1%. The predicted labels then act as a filter for coreference: only NPs labeled old are sent to the resolver. This raises F-measure of mention-pair and cluster-ranking coreference models by roughly 1–2.6%. The work matters because anaphora resolvers otherwise try to link every NP, which hurts precision.

Key points

  • Data: 147 Switchboard dialogues (later released in the NXT corpus) with 68,992 NPs. Of these, 51.2% are old, 34.5% mediated and 14.3% new. The split is 117 training and 30 test dialogues.
  • Classifier: SVM light, with three one-versus-all binary classifiers. Each test NP gets the class whose classifier returns the most positive margin. Binary classifiers were chosen because SVM multiclass cannot use the non-linear kernels that parse-tree features need.
  • Baseline features (Nissim 2006): full previous mention, mention time, partial previous mention, determiner, NP type, NP length and grammatical role.
  • Adding lexical and parse-tree features gives an 8.1% improvement over the Nissim-only system.
  • Coreference was scored with B3 and CEAF. Restricting resolution to predicted-old NPs helps both models. With gold information status, F-measure rises a further 1.8–4.1% (MP) and 2.7–3.1% (CR).

Interesting ideas

  • I1. Instead of building a list of "generally known" entities by hand, let lexical features pick up such world knowledge implicitly from the annotated data; this is a cheap substitute for an explicit knowledge base.
  • I2. Use a cheap upstream classifier as a gate, so the expensive resolver only considers mentions predicted to have an antecedent; this trades a little recall for precision.
  • I3. Split train and test by whole documents rather than by shuffled instances, because instance-level shuffling leaks material from the same document and blocks per-document downstream tasks.
  • I4. Better accuracy on the auxiliary task did not translate into better coreference: newly correct "old" labels exposed NPs to wrong antecedents, while errors in the weaker system stayed harmless.
  • I5. An oracle run with gold information status gives an upper bound on how much the feature can help coreference.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Mova could flag entities as hearer-known or text-introduced when populating the world of a text, learning the flag from annotated mentions instead of curating a full list.
  • I2 (frames): Mova's coreference step could first mark each mention old, mediated or new, and attempt antecedent search only for old ones, with the mediated label explaining bridging cases such as part-whole.
  • I3 (learn): Mova's evaluation sets should hold out whole texts, so that discourse-level components are measured honestly.
  • I4 (learn): when a component improves, Mova should measure its effect on the end task and inspect the cases that changed, rather than trusting the component's own score.
  • I5 (learn): Mova could add gold-input oracle runs for each pipeline stage, which shows where improving upstream accuracy actually pays off.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of Texas at DallasRichardson · United States47 works · 5,363 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 401 works

Cite

Cite as Altaf Rahman and Vincent Ng. 2011. Learning the Information Status of Noun Phrases in Spoken Dialogues. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{rahman2011learning,
  title = {{Learning the Information Status of Noun Phrases in Spoken Dialogues}},
  author = {Altaf Rahman and Vincent Ng},
  booktitle = {Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing},
  year = {2011},
  url = {https://aclanthology.org/D11-1099/}
}

References

38 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David J. Allerton. 1978. The notion of ’givenness’ and its relations to presupposition and to theme. Lingua, 44:133–168. ↗
  2. Chinatsu Aone and Scott William Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, pages 122–129.
  3. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the Linguistic Coreference Workshop at the First International Conference on Language Resources and Evaluation, pages 563–566.
  4. David Bean and Ellen Riloff. 1999. Corpus-based identification of non-anaphoric noun phrases. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 373–380.
  5. Sasha Calhoun, Jean Carletta, Jason Brenier, Neil Mayo, Dan Jurafsky, Mark Steedman, and David Beaver. 2010. The NXT-format Switchboard corpus: A rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue. Language Resources and Evaluation, 44(4):387–419. ↗
  6. Michael Collins and Nigel Duffy. 2001. Convolution kernels for natural language. In Advances in Neural Information Processing Systems 14, pages 625–632. ↗
  7. Michael Collins. 2002. Ranking algorithms for named-entity extraction: Boosting and the voted perceptron. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 489– 496. find
  8. Aron Culotta, Michael Wick, and Andrew McCallum. 2007. First-order probabilistic models for coreference resolution. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 81–88.
  9. Chad Cumby and Dan Roth. 2003. On kernel methods for relational learning. In Proceedings of the 20th International Conference on Machine Learning, pages 107–114. find
  10. Pascal Denis and Jason Baldridge. 2007. Global, joint determination of anaphoricity and coreference resolution using integer programming. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 236–243.
  11. Pascal Denis and Jason Baldridge. 2008. Specialized models and ranking for coreference resolution. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 660–669.
  12. Miriam Eckert and Michael Strube. 2001. Dialogue acts, synchronising units and anaphora resolution. Journal of Semantics, 17(1):51–89. find
  13. Eva Hajičová. 1984. Topic and focus. In Contributions to Functional Syntax, Semantics, and Language Comprehension (LLSEE 16), pages 189–202. John Benjamins, Amsterdam. ↗
  14. Michael A. K. Halliday. 1976. Notes on transitivity and theme in English. Journal of Linguistics, 3(2):199– 244. find
  15. Laurence R. Horn. 1986. Presupposition, theme, and variations. In A. Farley et al., editor, Papers from the Parasession on Pragmatics and Grammatical Theory at the 22nd Regional Meeting, pages 168–192. CLS. find
  16. Thorsten Joachims. 1999. Making large-scale SVM learning practical. In Bernhard Scholkopf and Alexander Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 44–56. MIT Press. ↗
  17. Harold W. Kuhn. 1955. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(83). ↗
  18. Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 135–142.
  19. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25– 32.
  20. Joseph McCarthy and Wendy Lehnert. 1995. Using decision trees for coreference resolution. In Proceedings of the Fourteenth International Conference on Artificial Intelligence, pages 1050–1055. ↗
  21. Alessandro Moschitti. 2004. A study on convolution kernels for shallow statistic parsing. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 335–342. find
  22. Vincent Ng and Claire Cardie. 2002. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In Proceedings of the 19th International Conference on Computational Linguistics, pages 730–736. find
  23. Vincent Ng. 2004. Learning noun phrase anaphoricity to improve conference resolution: Issues in representation and optimization. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 151–158.
  24. Vincent Ng. 2009. Graph-cut-based anaphoricity determination for coreference resolution. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 575– 583.
  25. Malvina Nissim, Shipra Dingare, Jean Carletta, and Mark Steedman. 2004. An annotation scheme for information status in dialogue. In Proceedings of the 4th International Conference on Language Resources and Evaluation. find
  26. Malvina Nissim. 2006. Learning information status of discourse entities. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 94–102. find
  27. Ellen Prince. 1981. Toward a taxonomy of given-new information. In P. Cole, editor, Radical Pragmatics, pages 223–255. New York, N.Y.: Academic Press. find
  28. Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 968–977.
  29. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. ↗
  30. Mark Steedman. 2000. The Syntactic Process. The MIT Press, Cambridge, MA. ↗
  31. Michael Strube. 1998. Never look back: An alternative to centering. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics, pages 1251–1257. ↗
  32. Olga Uryupina. 2003. High-precision identification of discourse new and unique noun phrases. In Proceedings of the ACL Student Research Workshop, pages 80– 86. find
  33. Enric Vallduvı́. 1992. The Informational Component. Garland, New York. find
  34. Yannick Versley, Alessandro Moschitti, Massimo Poesio, and Xiaofeng Yang. 2008a. Coreference systems based on kernels methods. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 961–968.
  35. Renata Vieira and Massimo Poesio. 2000. An empirically-based system for processing definite descriptions. Computational Linguistics, 26(4):539– 593. ↗
  36. Xiaofeng Yang, Jian Su, Jun Lang, Chew Lim Tan, and Sheng Li. 2008. An entity-mention model for coreference resolution with inductive logic programming. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 843–851.
  37. Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2003. Kernel methods for relation extraction. Journal of Machine Learning Research, 3:1083–1106.
  38. GuoDong Zhou and Fang Kong. 2009. Global learning of noun phrase anaphoricity in coreference resolution via label propagation. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 978–986.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rahman-2011-learning-information-status-noun-phrases/