Elamur Research
☰
home / papers / 2008 / Modeling Local Coherence: An Entity-Based Approach

Modeling Local Coherence: An Entity-Based Approach

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J08-1001.pdf

Summary

Systems that generate or summarize text need a way to judge whether their output flows well from one sentence to the next. Earlier coherence models used hand-written rules or manually annotated corpora, and they did not carry over well to new domains. Barzilay and Lapata introduce the entity grid. Its rows are sentences and its columns are discourse entities. Each cell records whether an entity is mentioned and in what grammatical role. A text is then described by how often particular entity transition patterns occur, and a learned model uses these features. This turns coherence assessment into a machine learning task instead of a direct implementation of Centering Theory. All features are extracted automatically, using a parser and either the Ng and Cardie (2002) coreference system or simple string matching. The authors test the approach on three tasks: ordering sentences, ranking automatic summaries against human coherence ratings, and predicting readability. It beats state-of-the-art methods on the first two and improves the Schwarm and Ostendorf (2005) readability system. The entity grid became a standard reference point for data-driven discourse coherence modeling.

Key points

  • Grid cells use four labels: subject (S), object (O), other role (X), or absent (–). The experiments vary three knowledge sources: coreference resolution, syntactic roles, and salience (how often an entity is mentioned).
  • Combining syntax and salience gives consistently strong results on all three tasks.
  • Sentence ordering and summary evaluation are treated as ranking problems, where the model picks the most coherent of several alternatives. For readability, discourse-level grid features are added to the sentence-level features of existing systems.
  • Automatic coreference (Ng and Cardie 2002, trained on newspaper text) helps on human-written texts but not on machine-generated summaries. Plain string matching works as a reliable substitute for coreference.
  • Pronoun use turns out to be a useful signal of coherence, but the models depend on domain and genre. Proposed future work includes grids built on lexical chains, other languages such as Chinese and German, and combining the grid with Rhetorical Structure Theory.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Discourse analysis · Coreference resolution

Cited by

241 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 233 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli.2008.34.1.1

@article{barzilay2008modeling,
  title = {{Modeling Local Coherence: An Entity-Based Approach}},
  author = {Barzilay, Regina and Lapata, Mirella},
  journal = {Computational Linguistics},
  year = {2008},
  doi = {10.1162/coli.2008.34.1.1},
  url = {https://doi.org/10.1162/coli.2008.34.1.1}
}

References

77 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Althaus, Ernst, Nikiforos Karamanis, and Alexander Koller. 2004. Computing locally coherent discourses. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 399–406, Barcelona, Spain. find
  2. Ariel, Mira. 1988. Referring and accessibility. Journal of Linguistics, 24:65–87. find
  3. Asher, Nicholas and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press, Cambridge, England. find
  4. Barzilay, Regina. 2003. Information Fusion for Multi-Document Summarization: Praphrasing and Generation. Ph.D. thesis, Columbia University, New York. find
  5. Barzilay, Regina and Noemie Elhadad. 2003. Sentence alignment for monolingual comparable corpora. In Proceedings of the 8th Conference on Empirical Methods in Natural Language Processing, pages 25–32, Sapporo, Japan. find
  6. Barzilay, Regina and Mirella Lapata. 2005. Modeling local coherence: An entity-based approach. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 141–148, Ann Arbor, MI. find
  7. Barzilay, Regina and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In Proceedings of the 2nd Human Language Technology Conference and Annual Meeting of the North American Chapter of the Association for Computational Linguistic… find
  8. Berry, Michael W., Susan T. Dumais, and Gavin W. O’Brien. 1994. Using linear algebra for intelligent information retrieval. SIAM Review, 37(4):573–595. find
  9. Brennan, Susan E., Marilyn W. Friedman, and Charles J. Pollard. 1987. A centering approach to pronouns. In Proceedings of the 25th Annual Meeting of the Association for Computational Linguistics, pages 155–162, Palo Alto, CA. find
  10. Briscoe, Ted and John Carroll. 2002. Robust accurate statistical annotation of general text. In Proceedings of the 3rd International Conference on Language Resources and Evaluation, pages 1499–1504, Las Palmas, Canary Islands. find
  11. Chafe, Wallace L. 1976. Givenness, contrastiveness, definiteness, subjects, topics, and point of view. In Charles N. Li, editor, Subject and Topic. Academic Press, New York, pages 25–55. find
  12. Chall, Jeanne S. 1958. Readability: An Appraisal of Research and Application. Number 34 in Bureau of Educational Research Monographs. Ohio State University Press, Columbus. find
  13. Chall, Jeanne S. and Edgar Dale. 1995. Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books, Cambridge, MA. find
  14. Charniak, Eugene. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 132–139, Seattle, WA. find
  15. Clark, Herbert H. and Susan E. Haviland. 1977. Comprehension and the given-new contract. In Roy O. Freedle, editor, Discourse Production and Comprehension. Ablex, Norwood, NJ, pages 1–39. find
  16. Clarkson, Philip and Ronald Rosenfeld. 1997. Statistical language modeling. In Proceedings of ESCA EuroSpeech’97, pages 2707–2710, Rhodes, Greece. find
  17. Collins, Michael. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics, pages 16–23, Madrid, Spain. find
  18. Collins, Michael. 2002. Discriminative reranking for natural language parsing. In Proceedings of the 17th International Conference on Machine Learning, pages 175–182, Palo Alto, CA. find our card
  19. Foltz, Peter W., Walter Kintsch, and Thomas K. Landauer. 1998. Textual coherence using latent semantic analysis. Discourse Processes, 25(2&3):285–307. find
  20. Freund, Yovav, Raj Iyer, Robert E. Schapire, and Yoram Singer. 2003. An efficient boosting algorithm for combining preferencs. Machine Learning, 4:933–969. find
  21. Germann, Ulrich, Michael Jahr, Kevin Knight, Daniel Marcu, and Kenji Yamada. 2004. Fast and optimal decoding for machine translation. Artificial Intelligence, 154(1–2):127–143. find
  22. Givon, Talmy. 1987. Beyond foreground and background. In Russell S. Tomlin, editor, Coherence and Grounding in Discourse. Benjamins, Amsterdam/Philadelphia, pages 175–188. find
  23. Grosz, Barbara, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225. find our card
  24. Gundel, Jaenette K., Nancy Hedberg, and Ron Zacharski. 1993. Cognitive status and the form of referring expressions in discourse. Language, 69(2):274–307. find
  25. Gunning, Robert. 1952. The Technique of Clear Writing. McGraw Hill, New York. find
  26. Halliday, M. A. K. and Ruqaiya Hasan. 1976. Cohesion in English. Longman, London. find
  27. Hasler, Laura. 2004. An investigation into the use of centering transitions for summarisation. In Proceedings of the 7th Annual CLUK Research Colloquium, pages 100–107, Birmingham, UK. find
  28. Hoey, Michael. 1991. Patterns of Lexis in Text. Oxford University Press, Oxford, England. find
  29. Hudson, S. B., M. K. Tanenhaus, and G. S. Dell. 1986. The effect of the discourse center on the local coherence of a discourse. In Proceedings of the 8th Annual Meeting of the Cognitive Science Society, pages 96–101, Amherst, MA. find
  30. Joachims, Thorsten. 1998a. Making large-scale support vector machine learning practical. In Bernard Schölkopf, Christopher Burges, and Alexander Smola, editors, Advances in Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA. find
  31. Joachims, Thorsten. 1998b. Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the European Conference on Machine Learning, pages 137–142, Berlin, Springer. find our card
  32. Joachims, Thorsten. 2002. Optimizing search engines using clickthrough data. In Proceedings of ACM Conference on Knowledge Discovery and Data Mining, pages 133–142, Chicago, IL. find
  33. Kameyama, Megumi. 1986. A property-sharing constraint in centering. In Proceedings of the 24th Annual Meeting of the Association for Computational Linguistics, pages 200–206, New York. find
  34. Karamanis, Nikiforos. 2003. Entity Coherence for Descriptive Text Structuring. Ph.D. thesis, University of Edinburgh, Edinburgh, Scotland. find
  35. Karamanis, Nikiforos, Massimo Poesio, Chris Mellish, and Jon Oberlander. 2004. Evaluating centering-based metrics of coherence for text structuring using a reliably annotated corpus. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 391–398, Barcelona,… find
  36. Karttunen, Lauri. 1976. Discourse referents. In James D. McCawley, editor, Syntax and Semantics: Notes from the Linguistic Underground, volume 7. Academic Press, New York, pages 363–386. find
  37. Katz, Irvin R. and Malcolm I. Bauer. 2001. Sourcefinder: Course preparation via linguistically targeted web search. Journal of Educational Technology and Society, 4(3):45–49. find
  38. Kibble, Rodger and Richard Power. 2004. Optimising referential coherence in text generation. Computational Linguistics, 30(4):401–416. find
  39. Kincaid, J. Peter, James Aagard, John O’Hara, and Larry Cottrell. 1981. Computer readability editing system. IEEE Transactions on Professional Communication, 1(24):34–81. find
  40. Kincaid, Peter J., Robert P. Fishburne, Richard L. Rodgers, and Brad S. Chissom. 1975. Derivation of new readability formulas for Navy enlisted personnel. Research Branch Report 8-75, U.S. Naval Air Station, Memphis, TN. find
  41. Knight, Kevin and Vasileios Hatzivassiloglou. 1995. Two-level, many-path generation. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, pages 252–260, Cambridge, MA. find
  42. Kudo, Taku and Yuji Matsumoto. 2001. Chunking with support vector machines. In Thorsten Joachims, editor, Proceedings of the 2nd Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 192–199, Pittsburgh, PA. find
  43. Kuhn, R. and R. De Mori. 1990. A cache-based natural language model for speech recognition. IEEE Transactions on PAMI, 12(6):570–583. find
  44. Kuno, Susumu. 1972. Functional sentence perspective. Linguistic Inquiry, 3:269–320. find
  45. Landauer, Thomas K. and Susan T. Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction and representation of knowledge. Psychological Review, 104(2):211–240. find
  46. Langkilde, Irene and Kevin Knight. 1998. Generation that exploits corpus-based statistical knowledge. In Proceedings of the 17th International Conference on Computational Linguistics and 36th Annual Meeting of the Association for Computational Linguistics, pages 704–710, Montréal, Canada. find
  47. Lapata, Mirella. 2003. Probabilistic text structuring: Experiments with sentence ordering. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 545–552, Sapporo, Japan. find
  48. Lin, Chin-Yew and Eduard H. Hovy. 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of the 2nd Human Language Technology Conference and Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 71–78, Boston, MA. find
  49. Lin, Dekang. 2001. LaTaT: Language and text analysis tools. In Proceedings of the 1st International Conference on Human Language Technology Research, pages 222–227, San Francisco, CA. find
  50. Mann, William C. and Sandra A. Thomson. 1988. Rhetorical structure theory. Text, 8(3):243–281. find
  51. Marcu, Daniel. 2000. The Theory and Practice of Discourse Parsing and Summarization. MIT Press, Cambridge, MA. find
  52. McKoon, Gail and Roger Ratcliff. 1992. Inference during reading. Psychological Review, 99(3):440–446. find
  53. Mellish, Chris, Mick O’Donnell, Jon Oberlander, and Alistair Knott. 1998. Experiments using stochastic search for text planning. In Proceedings of the 9th International Workshop on Natural Language Generation, pages 98–107, New Brunswick, NJ. find
  54. Miltsakaki, Eleni and Karen Kukich. 2000. The role of centering theory’s rough-shift in the teaching and evaluation of writing skills. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 408–415, Hong Kong. find
  55. Mitchell, James V. 1985. The Ninth Mental Measurements Yearbook. University of Nebraska Press, Lincoln. find
  56. Morris, Jane and Graeme Hirst. 1991. Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics, 1(17):21–43. find our card
  57. Ng, Vincent and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 104–111, Philadelphia, PA. find our card
  58. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, PA. find our card
  59. Poesio, Massimo, Rosemary Stevenson, Barbara Di Eugenio, and Janet Hitzeman. 2004. Centering: A parametric theory and its instantiations. Computational Linguistics, 30(3):309–363. find
  60. Pradhan, Sameer, Kadri Hacioglu, Valerie Krugler, Wayne Ward, James H. Martin, and Dan Jurafsky. 2005. Support vector learning for semantic argument classification. Machine Learning, 60(1):11–39. find
  61. Prince, Ellen. 1978. A comparison of wh-clefts and it-clefts in discourse. Language, 54:883–906. find
  62. Prince, Ellen. 1981. Toward a taxonomy of given-new information. In Peter Cole, editor, Radical Pragmatics. Academic Press, New York, pages 223–255. find
  63. Reiter, Ehud and Robert Dale. 2000. Building Natural-Language Generation Systems. Cambridge University Press, Cambridge, England. find
  64. Schwarm, Sarah E. and Mari Ostendorf. 2005. Reading level assessment using support vector machines and statistical language models. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 523–530, Ann Arbor, MI. find
  65. Scott, Donia and Clarisse Sieckenius de Souza. 1990. Getting the message across in RST-based text generation. In Robert Dale, Chris Mellish, and Michael Zock, editors, Current Research in Natural Language Generation. Academic Press, New York, pages 47–73. find
  66. Sidner, Candace L. 1979. Towards a Computational Theory of Definite Anaphora Comprehension in English Discourse. Ph.D. thesis, MIT. find
  67. Soon, W. M., Hwee Tou Ng, and D. C. Y. Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. find our card
  68. Stenner, A. Jackson. 1996. Measuring reading comprehension with the lexile framework. Presented at the California Comparability Symposium, Burlingame, CA. find
  69. Strube, Michael and Udo Hahn. 1999. Functional centering—Grounding referential coherence in information structure. Computational Linguistics, 25(3):309–344. find
  70. Toutanova, Kristina, Penka Markova, and Christopher D. Manning. 2004. The leaf projection path view of parse trees: Exploring string kernels for HPSG parse selection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 166–173, Barcelona, Spain. find
  71. Vapnik, Vladimir. 1998. Statistical Learning Theory. Wiley, Chichester, UK. find
  72. Vilain, Marc, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th Message Understanding Conference (MUC-6), pages 45–52, San Francisco, CA. find our card
  73. Walker, Marilyn, Masayo Iida, and Sharon Cote. 1994. Japanese discourse and the process of centering. Computational Linguistics, 20(2):193–232. find
  74. Walker, Marilyn, Aravind Joshi, and Ellen Prince, editors. 1998. Centering Theory in Discourse. Clarendon Press, Oxford, UK. find
  75. Walker, Marilyn A., Owen Rambow, and Monica Rogati. 2001. Spot: A trainable sentence planner. In Proceedings of the 2nd Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 17–24, Pittsburgh, PA. find
  76. Weiss, Sholom M. and Casimir A. Kulikowski. 1991. Computer Systems that Learn: Classification and Prediction Methods from, Statistics, Neural Nets, Machine Learning, and Expert Systems. Morgan Kaufmann, San Mateo, CA. find
  77. Witten, Ian H. and Eibe Frank. 2000. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufman, San Mateo, CA. find