📄 The DialogBank

Summary

Dialogue act corpora have been annotated under many incompatible schemes, such as DAMSL, SWBD-DAMSL, HCRC Map Task, DIT++ and MRDA, which makes them hard to combine or compare. The DialogBank addresses this by collecting dialogues and giving them gold-standard annotations under the ISO 24617-2 standard. Some dialogues are re-annotated from existing corpora and some are annotated for the first time. ISO 24617-2 goes beyond simply labelling turns. It assigns several functions per segment across nine semantic dimensions, allows overlapping or discontinuous functional segments, and records qualifiers, dependence relations and rhetorical relations.

The authors build on the standard's split between an abstract syntax (set-theoretic annotation structures) and concrete renderings. They show that three formats are "ideal", meaning complete and unambiguous with respect to that abstract syntax: the XML reference format DiAML-XML and two human-friendly tables, DiAML-TabSW and DiAML-MultiTab. Any ideal format can be mapped to any other without losing meaning, so the three formats are interoperable. The result is a reusable, standards-based multilingual resource (English and Dutch) for dialogue research.

Key points

• Source corpora: English material comes from HCRC Map Task, Switchboard, TRAINS and DBOX. Dutch material comes from DIAMOND, Schiphol, OVIS and Dutch Map Task. Multi-party AMI dialogues are planned.
• Dimensions: ISO 24617-2 has nine, namely Task, Auto-Feedback, Allo-Feedback, Turn Management, Time Management, Own Communication Management, Partner Communication Management, Discourse Structuring and Social Obligation Management.
• Annotation process: Map Task dialogues were re-segmented from raw speech using the ANVIL tool. Switchboard SWBD-DAMSL tags were first converted semi-automatically to ISO function tags (Fang et al., 2012) as an intermediate step. Dutch Map Task, OVIS and Schiphol had no prior dialogue act annotation.
• Table layouts: DiAML-MultiTab uses four columns for markable, speaker, addressee and other participants, followed by one column per dimension. DiAML-TabSW instead lists all of a segment's acts in a single column, each tagged with its dimension.
• Release status: The resource is in beta; its first public release was in December 2015.

Interesting ideas

• I1. Defining one abstract annotation structure and proving each concrete format complete and unambiguous, via an encoding function and its inverse, guarantees lossless conversion between formats instead of relying on ad-hoc converters.
• I2. Treating an utterance as carrying several acts at once, one per semantic dimension (task content, feedback, turn-taking, stalling, social niceties), avoids forcing a single label onto a fragment.
• I3. Annotation units are minimal functional segments rather than turns or sentences, and they may overlap, be discontinuous, or span speakers.
• I4. Dependence links (which question an answer addresses, what a feedback act reacts to) and rhetorical links with named argument roles (e.g. reason/result for Cause) are recorded as relations between acts.
• I5. Mapping legacy labels semi-automatically to the target scheme and using that only as an intermediate step toward full re-annotation is a cheap bootstrap for existing data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): UD dialect conversion could be specified as invertible maps to and from one abstract dependency structure, with round-trip tests confirming no information is lost.
• I2 (frames): The speech-act layer in prag could output a set of dimension-tagged acts per segment, so that "oh okay" is recorded as positive feedback and a turn-taking move at the same time.
• I3 (frames): Speech acts and implicatures could attach to explicit token spans that need not match sentence boundaries, which keeps every act traceable to its exact evidence.
• I4 (world): Event ordering and causal structure in the world model could store typed, role-labelled links between acts or events, so answers about "why" can cite the linked pair.
• I5 (learn): Old treebank or frame labels could be auto-converted into silver data that witness parsers and the LLM teacher then refine, rather than being trusted as gold.