Elamur Research⌕
☰
home / papers / 2004 / Multi-level Dialogue Act Tags

Multi-level Dialogue Act Tags

Summary

Dialogue act (DA) annotation has no agreed tagset, and multi-dimensional schemes like DAMSL or ICSI-MR allow millions of tag combinations, which makes automatic labelling hard. The authors work on meeting recording and retrieval in the (IM)2 project. They set out criteria a DA tagset should meet: a theoretical basis, fit with real utterances, reliable human annotation, feasibility of automatic tagging, usefulness for the application, and mappability to earlier schemes. Using these criteria they derive MALTUS from the ICSI-MR and Switchboard tagsets. MALTUS keeps a small primary layer of general tags plus a few optional secondary tags, with explicit rules about which tags exclude each other.

On the ICSI Meeting Recorder corpus, they train Maximum Entropy classifiers with lexical and contextual features. They compare two designs: one integrated classifier that predicts whole labels, and a set of separate per-level classifiers whose outputs are combined. Coarse distinctions turn out to be learnable. However, the multi-level setup did slightly worse than the integrated one, which points to dependencies between the tag layers.

Key points

  • The ICSI-MR corpus contains 75 one-hour staff meetings with up to eight speakers each. After processing there are 113,560 labelled utterances with 776 distinct label types, while about 7.3 million combinations are theoretically possible.
  • Allowing only one tag per label would cap tagging accuracy on ICSI-MR at 0.601. Allowing two tags raises the cap to 0.934.
  • Setup: a Maximum Entropy classifier trained on 40 meetings and tested on 5 meetings (6,771 utterances).
  • Accuracy on coarser tasks:
    • Six-way task: 77.9% vs. a 54.0% baseline (reported human kappa for a similar task: 0.79).
    • Four general MALTUS tags (S, H, Q, B): 84.9% vs. 64.1%.
  • Full MALTUS labels (baseline 41.9%):
    • A single integrated classifier reached 73.2%.
    • Six combined level-specific classifiers reached 70.5%. These covered the primary tag, politeness, attention, order, restatement and response.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory · Part-of-speech tagging

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{clark2004multi,
  title = {{Multi-level Dialogue Act Tags}},
  author = {Clark, Alexander and Popescu-Belis, Andrei},
  booktitle = {Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004},
  year = {2004},
  url = {https://aclanthology.org/W04-2328/}
}

References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. James F. Allen and Mark G. Core. 1997. DAMSL: Dialog act markup in several layers (draft 2.1). Technical report, Multiparty Discourse Group, Discourse Research Initiative, September/October 1997. find
  2. Susan Armstrong, Alexander Clark, Giovanni Coray, Maria Georgescul, Vincenzo Pallotta, Andrei Popescu-Belis, David Portabella, Martin Rajman, and Marianne Starlander. 2003. Natural language queries on natural language data: a database of meeting dialogues. In NLDB’2003 (8th International Conference on Applications of Natural Language to Information Systems), Burg/Cottbus, Germany. find
  3. Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2003. Automatically generated prosodic cues to lexically ambiguous dialog acts in multi-party meetings. In ICPhS 2003, Barcelona. find
  4. Jean Carletta, Amy Isard, Stephen Isard, Jacqueline C. Kowtko, Gwyneth Doherty-Sneddon, and Anne H. Anderson. 1997. The reliability of a dialogue structure coding scheme. Computational Linguistics, 23:13–31. ↗
  5. Mark G. Core and James F. Allen. 1997. Coding dialogues with the DAMSL annotation scheme. In David Traum, editor, Working Notes: AAAI Fall Symposium on Communicative Action in Humans and Machines, pages 28–35, Menlo Park, CA. American Association for Artificial Intelligence. find
  6. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting recorder project: Dialog act labeling guide. Technical Report TR-04-002, ICSI (International Computer Science Institute), Berkeley, CA. ↗
  7. Daniel Jurafsky, Elizabeth Shriberg, and Debra Biasca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation (coders manual, draft 13). Technical Report 97-02, University of Colorado, Institute of Cognitive Science. find
  8. Daniel Jurafsky, Elizabeth Shriberg, Barbara Fox, and Traci Curl. 1998. Lexical, prosodic, and syntactic cues for dialog acts. In ACL/COLING-98 Workshop on Discourse Relations and Discourse Markers, pages 114–120. find our card
  9. Marion Klein and Claudia Soria. 1998. Dialogue acts. In Marion Klein, Niels Ole Bernsen, Sarah Davies, Laila Dybkjaer, Juanma Garrido, Henrik Kasch, Andreas Mengel, Vito Pirrelli, Massimo Poesio, Silvia Quazza, and Claudia Soria, editors, MATE Deliverable 1.1: Supported Coding Schemes, MATE (Multilevel Annotation, Tools Engineering) European Project LE4-8370. find
  10. Stephen C. Levinson. 1983. Pragmatics. Cambridge University Press, Cambridge, UK. find
  11. Agnes Lisowska. 2003. Multimodal interface design for the multimodal meeting domain: Preliminary indications from a query analysis study. Technical report, IM2.MDM, 11/2003. find
  12. Christopher Manning and Dan Klein. 2003. Optimization, maxent models, and conditional estimation without magic. In Tutorial at HLT-NAACL 2003 and ACL 2003. ACL, Edmonton, Canada. find
  13. Nelson Morgan, Don Baron, Sonali Bhagat, Hannah Carvey, Rajdip Dhillon, Jane A. Edwards, David Gelbart, Adam Janin, Ashley Krupski, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, and Chuck Wooters. 2003. Meetings about meetings: research at ICSI on speech in multiparty conversations. In ICASSP 2003 (International Conference on Acoustics, Speech, and Signal Processing), Hong Kong, China. find
  14. Andrei Popescu-Belis. 2003. Dialogue act tagsets for meeting understanding: an abstraction based on the DAMSL, Switchboard and ICSI-MR tagsets. Technical report, IM2.MDM, v1.1, 09/2003. find
  15. John R. Searle. 1969. Speech Acts. Cambridge University Press, Cambridge, UK. find our card
  16. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The ICSI meeting recorder dialog act (MRDA) corpus. In Proceedings of SIGDIAL ’04 (5th SIGdial Workshop on Discourse and Dialog), Cambridge, MA. ↗ our card
  17. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Marie Meteer, and Carol Van Ess-Dykema. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–371. ↗ our card
  18. David R. Traum. 2000. 20 questions for dialogue act taxonomies. Journal of Semantics, 17(1):7–30. find
  19. Daniel Vanderveken. 1990. Meaning and speech acts. Cambridge University Press, Cambridge, UK. find
  20. Britta Wrede and Elizabeth Shriberg. 2003. The relationship between dialogue acts and hot spots in meetings. In IEEE Speech Recognition and Understanding Workshop, St. Thomas, U.S. Virgin Islands. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/clark-2004-multi-level-dialogue-act-tags/