Elamur Research
☰
home / papers / 2005 / A Framework for Annotating Information Structure in Discourse

A Framework for Annotating Information Structure in Discourse

What it is about

The paper proposes a framework for analysing information structure in conversational English, using the Switchboard corpus. It has two layers. The first is information status (old, mediated or new), which tells whether an entity is accessible to the listener. The second is information structure proper: theme/rheme, which says how each intonation phrase is organised, and kontrast, which marks how salient the speaker wants an entity to be. The authors argue that information structure in English can be analysed only together with prosodic prominence and phrasing. They therefore propose annotating a subset of the corpus for prosody, in stand-off XML in the NXT tool, so that the layers stay independent of the surface syntax. As a demonstration, they annotate 14 noun phrases of one "Pie in the Sky" sentence group, and they show how the theme/rheme split and the prosodic phrase boundaries differ.

Ideas

For Mova

The chapters are languages/en/29-ellipsis-word-order-information-structure.md and languages/en/30-pragmatics-overview.md. The code is prag/src/frames.rs for the discourse status and en/src/lin.rs for word order. The measure is agreement of the rule-based status and theme/rheme labels with a hand-labelled set, and the size of that set is itself a limit to report.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W05-0307.pdf

Summary

The paper tackles the lack of agreed annotation standards for information structure in spontaneous dialogue. The authors propose a layered framework for the Switchboard corpus, built with NXT stand-off XML on top of existing Penn Treebank syntax and dialog-act markup. It has two parts.

Information status. Noun phrases are labelled as old, mediated or new, with optional finer subtypes. These subtypes cover bridging, generally known entities, and coreference links.

Information structure. Theme/rheme and background/kontrast are annotated. The authors argue these cannot be separated from prosody: phrasing tracks the theme/rheme split, while prominence reflects information status and kontrast. So a subset of the corpus also receives prosodic labels.

In a two-annotator trial, Kappa agreement on the top-level categories was judged reliable. Agreement fell when subtypes were included, which supports the hierarchical design because annotators can always back off to the main classes. The scheme was then applied to 147 dialogues. The authors also illustrate it on the shared Pie-in-the-Sky sentences.

The resulting resource combines syntax, disfluency, dialog acts, discourse and prosody. It targets work such as paraphrase generation, information extraction, topic detection and dialogue speech synthesis.

Key points

  • Corpus and tools: Switchboard contains 2,430 phone conversations, about 3 million words. One third is parsed in the Penn Treebank and tagged with dialog acts. Annotation layers are integrated with NXT, and the TIGER Switchboard filter was used for XML conversion.
  • Category inventory: The nine mediated subtypes include aggregation, func value and poss, along with set, event, situation, part, bound and general. Old has six subtypes, including identity, event, general, generic, ident generic and relative. A decision tree resolves cases where several labels fit. WordNet and FrameNet guide bridging decisions.
  • Validation trial: Two annotators labelled three dialogues containing 1,738 markables. Of these, 1,502 were evaluated after removing 222 traces and 14 not-understood cases. Old proved easier to agree on than mediated or new.
  • Full annotation: Coverage is 147 dialogues, 43,358 sentences and 69,004 markables. These split into 35,299 old, 23,816 mediated and 9,889 new. A further 8,127 were non-applicable, and there are 16,324 coreference links.
  • Prosody layer: ToBI break indices 3 and 4 are conflated to mark phrase boundaries. Word-level prominence will be rated on a graded scale, still being calibrated in a pilot. In the Pie-in-the-Sky demo, 14 NPs were labelled for information status.

Topics: Discourse analysis

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1608829.1608836

@inproceedings{calhoun2005framework,
  title = {{A Framework for Annotating Information Structure in Discourse}},
  author = {Calhoun, Sasha and Nissim, Malvina and Steedman, Mark and Brenier, Jason},
  booktitle = {Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky},
  year = {2005},
  doi = {10.3115/1608829.1608836},
  url = {https://doi.org/10.3115/1608829.1608836}
}

References

39 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. G. M. Ayers. 1994. Discourse functions of pitch range in spontaneous and read speech. In J. Venditti, editor, OSU Working Papers in Linguistics, volume 44, pages 1–49. find
  2. C. F. Baker, C. J. Fillmore, and J. B. Lowe. 1998. The Berkeley FrameNet project. In C. Boitet and P. Whitelock, editors, Proc. COLING-ACL, pages 86–90. find our card
  3. E. Bard, D. Robertson, and A. Sorace. 1996. Magnitude estimation of linguistic acceptability. Language, 72(1):32–68. S. Baumann, C. Brinckmann, S. Hansen-Schirra, G-J. find
  4. Kruijff, I. Kruijff-Korbayová, S. Neumann, and E. Teich. 2004. Multi-dimensional annotation of linguistic corpora for investigating information structure. In Proc. NAACL/HLT ”Frontiers in Corpus Annotation”, Boston, MA. find
  5. M. Beckman and G. Elam. 1997. Guidelines for ToBI Labelling.. The OSU Research Foundation, v.3.0. find
  6. P. Boersma and D. Weenink. 2003. Praat:doing phonetics by computer. http://www.praat.org. ↗
  7. J. M. Brenier, D. M. Cer, and D. Jurafsky. 2005. Emphasis detection in speech using acoustic and lexical features. In LSA Annual Meeting, Oakland, CA. find
  8. S. Calhoun. 2004a. Overloaded ToBI and what to do about it: An argument for function-based phonological intonation categories. In Univ. of Edinburgh Ling. Postgrad. Conf.. find
  9. S. Calhoun. 2004b. Phonetic dimensions of intonational categories - L+H* and H*. In Prosody 2004, Nara, Japan. find
  10. J. Carletta, S. Dingare, M. Nissim, and T. Nikitina. 2004. Using the NITE XML Toolkit on the Switchboard Corpus to study syntactic choice: a case study. In Proc. of LREC2004, Lisbon. find
  11. J. Carletta. 1996. Assessing agreement on classification tasks: the kappa statistic. Comp. Ling., 22(2):249–254. find our card
  12. H. H. Clark. 1975. Bridging. In R. Schank and B. Nash-Webber, eds, Theoretical Issues in NLP. MIT Press, Cambridge, MA. find
  13. A. Cutler, D. Dahan, and W. van Donselaar. 1997. Prosody in the comprehension of spoken language: A literature review. Lang. and Sp., 40(2):141–201. find
  14. S. Davies, M. Poesio, F. Bruneseaux, and L. Romary. 1998. Annotating coreference in dialogues: Proposal for a scheme for MATE, http://www.hcrc.ed.ac.uk/˜poesio/anno_manual.html . ↗
  15. G. Dogil, J. Kuhn, J. Mayer, G. Mhler, and S. Rapp. 1997. Prosody and discourse structure: Issues and experiments. In Proc. of the ESCA Workshop on Intonation: Theory, Models and Applications, pages 99–102, Athens, Greece. find
  16. M. Eckert and M. Strube. 2001. Dialogue acts, synchronising units and anaphora resolution. J. of Semantics, 17(1):51–89. find
  17. C. Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. find
  18. J. Godfrey, E. Holliman, and J. McDaniel. 1992. SWITCH-BOARD: Telephone speech corpus for research and development. In Proc. ICASSP-92, pages 517–520. N. Hedberg and JM. find
  19. Sosa. 2001. The prosodic structure of topic and focus in spontaneous english dialogue. In LSA Workshop on Topic and Focus, Santa Barbara. find
  20. M. Heldner, E. Strangert, and T. Deschamps. 1999. A focus detector using overall intensity and high frequency emphasis. In Proc. ICPhS-99, vol 2, 1491–1493, San Francisco. find
  21. J. Hirschberg. 1993. Pitch accent in context: Predicting intonational prominence from text. AI, 63:305–340. find
  22. YCZ L. Hirschman and N. Chinchor. 1997. MUC-7 coreference task definition. In Proc. of 7 Conf. on Message Understanding. find
  23. D. R. Ladd. 1996. Intonational Phonology. CUP, UK. find
  24. K. Lambrecht. 1994. Information structure and sentence form. Topic, focus, and the mental representation of discourse referents. Camb. U. Press, UK. find
  25. S. Löbner. 1985. Definites. J. of Semantics, 4:279–326. M. Marcus, B. Santorini, and MA. find
  26. Marcinkiewicz. 1993. Building a large annotated corpus of english: The Penn treebank. Comp. Ling., 19:313–330. find our card
  27. A. Mengel and W. Lezius. 2000. An XML-based encoding format for syntactically annotated corpora. In Proc. LREC2000,121–126. find
  28. M. Nissim. 2003. Annotation scheme for information status in dialogue. HCRC, University of Edinburgh. Unpub. ms. find
  29. M. Nissim, S. Dingare, J. Carletta, and M. Steedman. 2004. An annotation scheme for information status in dialogue. In Proc. LREC2004, Lisbon. find
  30. R. Passonneau. 1996. Instructions for applying discourse reference annotation for multiple applications (DRAMA). Unpub. ms.. find
  31. J. Pitrelli, M. Beckman, and J. Hirschberg. 1994. Evaluation of prosodic transcription labelling reliability in the ToBI framework. In Proc. of the 3 Intl. Conf. on Spoken Lge. Proc., vol. 2, pages 123–126. []\ M. Poesio. 2000. The GNOME annotation scheme manual (v.4), http://www.hcrc.ed.ac.uk/˜gnome/anno_manual.html. ↗
  32. E. F. Prince. 1981. Toward a taxonomy of given-new information. In P. Cole, ed., Radical Pragmatics. Acad. Press, NY. find
  33. E. Prince. 1992. The ZPG letter: subjects, definiteness, and information-status. In S. Thompson and W. Mann, eds., Discourse description: diverse analyses of a fund raising text, pages 295–325. John Benjamins, Philadelphia/Amsterdam. find
  34. H.H. Rump and R. Collier. 1996. Focus conditions and the prominence of pitch-accented syllables. Lang. and Sp., 39:1–17. find
  35. E. Shriberg, P. Taylor, R. Bates, A. Stolcke, K. Ries, D. Jurafsky, N. Coccaro, R. Martin, M. Meteer, and C.V. Ess-Dykema. 1998. Can prosody aid the automatic classification of dialog acts in conversational speech? Lang. and Sp., 41(3-4):439–487. find
  36. E. Shriberg, A. Stolcke, D. Hakkani-Tür, and G. Tür. 2000. Prosody-based automatic segmentation of speech into sentences and topics. Sp. Comm., 32(2):127–154. find
  37. M. Steedman. 2000. Information Structure and the Syntax-Phonology Interface. LI, 31(4):649-689. find
  38. H. Truckenbrodt. 2002. Upstep and embedded register levels. Phonology, 19:77–120. find
  39. E. Vallduvı́ and M. Vilkuna. 1998. On Rheme and Kontrast. Syntax and Semantics, 29:79-108. find