Elamur Research⌕
☰
home / papers / 2008 / Coreference-inspired Coherence Modeling

Coreference-inspired Coherence Modeling

Summary

Local coherence models for sentence ordering usually treat a sentence as a bag of words or as entities with syntactic roles. They ignore how an entity is actually expressed. Elsner and Charniak bring two ideas from coreference research into coherence modeling.

The first is a maximum-entropy classifier that decides whether a noun phrase introduces a new referent or points back to a known one. It uses features adapted from Uryupina (2003), drops order-dependent cues, and relies on a "same head" approximation of coreference. The second is the probabilistic pronoun model of Ge et al. (1998), which scores pronoun–antecedent pairs using Hobbs distance, mention counts, and gender and number agreement.

Each model is multiplied with the entity grid of Lapata and Barzilay (2005). The authors test on Wall Street Journal documents using two tasks: telling an original document from a shuffled version (discrimination), and reinserting a removed sentence (insertion). On its own, the entity grid is stronger than either new model. The full combination, however, clearly beats the grid on both tasks. This shows that the form of referring expressions carries useful coherence signal.

Key points

  • The discourse-new classifier reaches 87.4% F-score on the MUC-7 formal test set, slightly above Uryupina's 86.9. Its training labels come from the "same head" heuristic, so no gold coreference annotation is needed.
  • The evaluation uses Penn Treebank WSJ sections 14–24 (1004 documents). On the AIRPLANE corpus the approach was ineffective, because those texts contain few pronouns or discourse-new markers.
  • Discrimination accuracy and F-score:
    • Entity Grid: 76.17 accuracy, 77.55 F
    • EGrid+Disc-New: 78.88 accuracy, 80.31 F
    • Combined (all three models): 79.60 accuracy, 81.02 F
  • Insertion score rises from 19.57 for the entity grid to 22.98 for the combined model. All differences are significant under Wilcoxon's signed-rank test (p < .001).
  • The pronoun model alone is weak (55.77 discrimination accuracy), since a displaced pronoun often finds another plausible antecedent nearby. Even so, removing it significantly hurts the combined system.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

24 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 20 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{elsner2008coreference,
  title = {{Coreference-inspired Coherence Modeling}},
  author = {Elsner, Micha and Charniak, Eugene},
  booktitle = {Proceedings of ACL-08: HLT, Short Papers},
  year = {2008},
  doi = {10.3115/1557690.1557702},
  url = {https://doi.org/10.3115/1557690.1557702}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Regina Barzilay and Mirella Lapata. 2005. Modeling local coherence: an entity-based approach. In ACL 2005. find
  2. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In HLT-NAACL 2004, pages 113–120. find
  3. Regina Barzilay, Noemie Elhadad, and Kathleen McKeown. 2002. Inferring strategies for sentence ordering in multidocument news summarization. Journal of Artificial Intelligence Results (JAIR), 17:35–55. find
  4. David L. Bean and Ellen Riloff. 1999. Corpus-based identification of non-anaphoric noun phrases. In ACL’99, pages 373–380. find our card
  5. Erdong Chen, Benjamin Snyder, and Regina Barzilay. 2007. Incremental text structuring with online hierarchical ranking. In Proceedings of EMNLP. find
  6. Hal Daumé III. 2004. Notes on CG and LM-BFGS optimization of logistic regressio n. Paper available at http://pub.hal3.name\#daume04cg-bfgs, implementation available at http://hal3.name/megam/, August. ↗
  7. Peter Foltz, Walter Kintsch, and Thomas Landauer. 1998. The measurement of textual coherence with 7 We attempted this method for pronouns using our model, but found it ineffective. latent semantic analysis. Discourse Processes, 25(2&3):285–307. find
  8. Kari Fraurud. 1990. Definiteness and the processing of noun phrases in natural discourse. Journal of Semantics, 7(4):395–433. find
  9. Niyu Ge, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 161–171. find our card
  10. Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225. find our card
  11. John A. Hawkins. 1978. Definiteness and indefiniteness: a study in reference and grammaticality prediction. Croom Helm Ltd. find
  12. Jerry R. Hobbs. 1976. Pronoun resolution. Technical Report 76-1, City College New York. find
  13. Mirella Lapata and Regina Barzilay. 2005. Automatic evaluation of text coherence: Models and representations. In IJCAI, pages 1085–1090. find
  14. Mirella Lapata. 2006. Automatic evaluation of information ordering: Kendall’s tau. Computational Linguistics, 32(4):1–14. find
  15. E. Miltsakaki and K. Kukich. 2004. Evaluation of text coherence for electronic essay scoring systems. Nat. Lang. Eng., 10(1):25–55. find
  16. Ani Nenkova and Kathleen McKeown. 2003. References to named entities: a corpus study. In NAACL ’03, pages 70–72. find
  17. Vincent Ng and Claire Cardie. 2002. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In COLING. find
  18. Massimo Poesio and Renata Vieira. 1998. A corpus-based investigation of definite description use. Computational Linguistics, 24(2):183–216. find
  19. Massimo Poesio, Mijail Alexandrov-Kabadjov, Renata Vieira, Rodrigo Goulart, and Olga Uryupina. 2005. Does discourse-new detection help definite description resolution? In Proceedings of the Sixth International Workshop on Computational Semantics, Tillburg. find
  20. Ellen Prince. 1981. Toward a taxonomy of given-new information. In Peter Cole, editor, Radical Pragmatics, pages 223–255. Academic Press, New York. find
  21. Radu Soricut and Daniel Marcu. 2006. Discourse generation using utility-trained coherence models. In ACL-2006. find
  22. Olga Uryupina. 2003. High-precision identification of discourse new and unique noun phrases. In Proceedings of the ACL Student Workshop, Sapporo. find
  23. Renata Vieira and Massimo Poesio. 2000. An empirically-based system for processing definite descriptions. Computational Linguistics, 26(4):539– 593. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/elsner-2008-coreference-inspired-coherence-modeling/