Elamur Research⌕
☰
home / papers / 2004 / The ICSI Meeting Recorder Dialog Act (MRDA) Corpus

The ICSI Meeting Recorder Dialog Act (MRDA) Corpus

Summary

Naturally occurring multi-party meetings involve heavy speaker overlap, interruptions, floor competition and real social dynamics. Large annotated resources for studying these phenomena at the discourse level have been scarce. The authors introduce the MRDA corpus, a set of annotations layered on the ICSI Meeting Corpus: about 72 hours of audio from 75 meetings recorded at ICSI. Annotators segmented speech into dialog act (DA) units, assigned DA labels and linked related utterances through adjacency pairs. The tag scheme adapts SWBD-DAMSL to meeting talk. Each DA receives one mandatory general tag plus optional specific tags, and a pipe symbol marks prosodic units that contain several DAs. Reliability was assessed in two ways: with a deliberately strict, word-based segmentation agreement metric the authors devised, and with Kappa on DA labels under several class groupings. The six-way grouping gives good agreement. The corpus also ships with word-level timings, annotator comments, suggested label groupings and rising-intonation marks. This makes it a valuable benchmark for DA classification, meeting understanding and prosody modeling.

Key points

  • Scale: 180,218 DA tags (excluding the rising-tone marker rt) across 75 meetings of about one hour each. The meetings include 53 speakers (40 male, 13 female), averaging around 6 per meeting.
  • Tag set: derived from SWBD-DAMSL, with 11 general tags and 39 specific tags. It also includes disruption marks (%-, %--) and indecipherable markers (x, %).
  • Reliability: overall strict segmentation agreement is 66.5%. Kappa is 0.80 for the basic six-way class map (Map 1), with pairwise values of 0.72–0.82 across the maps.
  • Distribution: splitting at the pipe gives 113,560 general tags. Statements (s) make up 68.00%, backchannels (b) 13.37% and floor holders (fh) 7.38%.
  • Process: annotation was done in 10-minute chunks, each taking roughly 3 hours to label. About one third of all words occur in overlapping speech.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

122 papers of our library cite it (of the 140673 whose bibliographies we read); 18 of them have a page here, 104 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{shriberg2004icsi,
  title = {{The ICSI Meeting Recorder Dialog Act (MRDA) Corpus}},
  author = {Shriberg, Elizabeth and Dhillon, Raj and Bhagat, Sonali and Ang, Jeremy and Carvey, Hannah},
  booktitle = {Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004},
  year = {2004},
  doi = {10.21236/ada460980},
  url = {https://doi.org/10.21236/ada460980}
}

References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexandersson, J., Buschbeck-Wolf, B., Fujinami, T., et al. Dia logue Acts in VERBMOBIL-2 Second Edition. VM-Report 226, DFKI Saarbrücken, Germany, July 1998. find
  2. Anderson, A. H., Bader, M., Bard, E. G., et al. (1991). The HCRC Map Task Corpus. Language and Speech, 34(4), 351-366. ↗
  3. Carletta, J., 1996. Assessing agreement on classification tasks: The Kappa Statistic. Computational Linguistics, 22:2, 249-254. ↗ our card
  4. Cieri, C., Miller, D. & Walker, K., 2002. Research methodologies, observations, and outcomes in conversational speech data collection. Proc. HLT 2002. find
  5. Clark, A. & Popescu-Belis, A., 2004. Multi-level Dialogue Act Tags. In Proceedings of SIGDIAL ’04 (5 th SIGDIAL Workshop on Discourse and Dialog). Cambridge, MA. find our card
  6. Core, M. & Allen, J., 1997. Coding dialogs with the DAMSL annotation scheme. Working Notes: AAAI Fall Symposium, AAAI, Menlo Park, CA, pp. 28-35. find
  7. Dhillon, R., Bhagat, S., Carvey, H., & Shriberg, E., 2004. Meeting Recorder Project: Dialog Act Labeling Guide. ICSI Technical Report TR-04-002, International Computer Science Institute. ↗
  8. Finke, M., Lapata, M., Lavie, A., et al., 1998. CLARITY: Inferring discourse structure from speech. AAAI ’98 Spring Symposium Series, March 23-25, 1998, Stanford University, California. find
  9. Janin, A. et al., 2003. The ICSI Meeting Corpus. Proc. ICASSP-2003. find
  10. Jekat, S., Klein, A., Maier, E., et al. Dialogue Acts in Verbmobil, Verbmobil-Report No. 65, April 1995. find
  11. Jurafsky, D., Shriberg, E., & Biasca, D., 1997. Switchboard SWBD-DAMSL Labeling Project Coder’s Manual, Draft 13. Technical Report 97-02, Univ. of Colorado Institute of Cognitive Science. find
  12. Levinson, S., 1983. Pragmatics. Cambridge: Cambridge University Press. find
  13. Waibel, A., et al., 2001. Advances in automatic meeting record creation and access. Proc. ICASSP-2001. find
  14. Wrede, B. & Shriberg, E., 2003. The relationship between dialogue acts and hot spots in meetings. Proc. IEEE Speech Recognition and Understanding Workshop, St. Thomas. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shriberg-2004-icsi-meeting-recorder-dialog-act/