Lexical Cohesion Computed by Thesaural relations as an indicator of the structure of text

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J91-1002.pdf

Summary

Morris and Hirst tackle a long-standing difficulty in discourse analysis: locating the boundaries of text segments without domain-specific knowledge. They build on Halliday and Hasan's notion of lexical cohesion and introduce lexical chains, which are sequences of nearby, semantically related words that can cross sentence boundaries. Using a general thesaurus (Roget's Thesaurus) as the only knowledge source, they link words that share a category or a thesaural connection. The resulting chains are then compared against the linguistic and intentional structure defined in Grosz and Sidner's discourse theory, using a long magazine excerpt ("Outland" by Jay Teitel). The analysis was carried out manually. It shows that chains broadly track segment structure: a chain ending tends to coincide with a segment ending, a new chain signals a new segment, and a returning chain marks a resumed earlier segment. This matters because chains can be computed for ordinary, domain-independent text, giving a cheap structural cue. They also supply local context for word sense disambiguation, which could extend sentence-bound methods such as Hirst's Polaroid Words.

Key points

Cohesion types. The paper adopts Halliday and Hasan's five classes of lexical cohesion. Three involve reiteration (same referent, different referent, superordinate). Two involve collocation (systematic relations such as antonyms or part-whole, and nonsystematic ones such as {garden, digging}).
Target structure. Chains are mapped onto Grosz and Sidner's discourse model, which has three parts: linguistic structure, intentional structure (with dominance and satisfaction-precedence relations), and a stack-based attentional state. The paper notes that identifying segments is the weak spot of that theory.
Chain returns. When an earlier chain resumes, this is treated as strong evidence that the text is returning to a prior segment.
Disambiguation example. In one chain from the example text (afflicted, darkness, eerie, cold, …), the context makes clear that barren and sterile mean cold and bleak, not infertile.
Limitations and future work. A general thesaurus misses domain vocabulary; for example, Volks/Volkswagen were not linked to the car chain. The authors call for automating the method, running it on large corpora, defining chain strength, and adding other resources such as domain thesauri, dictionaries, and corpus statistics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Discourse analysis · Word sense disambiguation

Cited by

155 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 148 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@article{morris1991lexical,
  title = {{Lexical Cohesion Computed by Thesaural relations as an indicator of the structure of text}},
  author = {Morris, Jane and Hirst, Graeme},
  journal = {Computational Linguistics},
  year = {1991},
  url = {https://aclanthology.org/J91-1002/}
}

References

17 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Bryan, Robert M. (1973). "Abstract thesauri and graph theory applications to thesaurus research," in Automated language analysis, edited by Sally Yeates Sedelow, University of Kansas. find
Carroll, Lewis (1872). Through the Looking Glass. find
Charniak, Eugene (1986). "A neat theory of marker parsing." In Proceedings, 5th National Conference on Artificial Intelligence, Philadelphia, August 1986, 584-588. find
Grosz, Barbara and Sidner, Candance (1986). "Attention, intentions and the structure of discourse." Computational Linguistics, 12(3), 175-204. find our card
Hahn, Udo (1985). "On lexically distributed text parsing. A computational model for the analysis of textuality on the level of text cohesion and text coherence." In Linking in text, edited by Ferenc Kiefer, Universit/it Konstanz. find
Halliday, Michael and Hasan, Ruqaiya (1976). Cohesion in English. Longman Group. find
Hirst, Graeme (1987). Semantic Interpretation and the Resolution of Ambiguity. Studies in Natural Language Processing. Cambridge University Press. find
Hirst, G. (1981). Anaphora in Natural Language Understanding: A Survey. Lecture Notes in Computer Science. Springer Verlag. find
Hobbs, Jerry (1978). "Coherence and coreference." Technical note 168, SRI International. find
McKeown, K. (1985). Text Generation: Using Discourse Strategies and Focus Constraints to Generate Natural Language Text. Studies in Natural Language Processing. Cambridge University Press. find
Morris, Jane (1988). "Lexical cohesion, the thesaurus, and the structure of text." Technical report CSRI-219, Department of Computer Science, University of Toronto. find
Postman, Leo and Keppel, Geoffrey, editors (1970). Norms of Word Association. Academic Press. find
Reichman, Rachel (1985). Getting Computers to Talk Like You and Me: Discourse Context, Focus, and Semantics (An ATN Model). The MIT Press. find
Roget, P. (1977). Roget's International Thesaurus, Fourth Edition. Harper and Row Publishers Inc. find
Sedelow, Sally and Sedelow, Walter (1987). "Semantic space." Computers and translation, 2, 235-245. find
Sedelow, Sally and Sedelow, Walter (1986). "Thesaural knowledge representation." In Proceedings, 2nd Annual Conference of the University of Waterloo Centre for the New Oxford English Dictionary: Advances in Lexicology. University of Waterloo. find
Ventola, E. (1987). The Structure of Social Interaction: A Systemic Approach to the Semiotics of Service Encounters. Open Linguistic Series. Frances Pinter Publishers. find