📄 Abstracting a Dialog Act Tagset for Meeting Processing

Summary

The paper targets meeting processing and retrieval (MPR), where finding things like unanswered questions requires labelling each utterance with its dialogue function. Popescu-Belis first sets out six requirements for a dialogue act tagset: a theoretical grounding, fit with real data, reliable human annotation, feasibility of automatic tagging, relevance to the application, and compatibility with earlier schemes. He then formalises DAMSL, SWBD-DAMSL and ICSI-MR with rewriting rules. DAMSL and ICSI-MR allow millions of tag combinations, which makes automatic tagging very hard. SWBD-DAMSL is tractable but too coarse for multiparty meetings.

From this analysis he derives MALTUS, a layered and abstract tagset. Each label has one primary tag and optional secondary tags, and explicit exclusiveness constraints limit the inventory to 770 possible labels. ICSI-MR meeting annotations were converted to MALTUS, and the conversion also served as a consistency check of the source corpus. The work matters because it shows how to keep a functionally rich annotation scheme that is still learnable, while reusing existing corpora.

Key points

• Size of the tag spaces:
  • DAMSL allows over 4 million tag combinations.
  • SWBD-DAMSL grouped the 220 combinations observed in about 200,000 Switchboard utterances into 42 mutually exclusive tags.
  • ICSI-MR also allows millions of combinations; about 400 distinct labels appeared in roughly 6 hours (about 7,000 utterances).
• MALTUS structure:
  • An utterance is either U (undecipherable) or carries one level-1 tag: S, Q, B or H.
  • It can then take optional level-2 tags: answer polarity (RP/RN/RU), AT, DO, RIC/RIR and PO.
  • A disruption mark D can be added independently.
  • The result is 770 possible labels.
• ICSI-MR data analysed:
  • The analysis covers 50 of the 75 hours.
  • After splitting prosodic units and setting aside disruption marks, 65,188 utterances remained, with 685 observed label types.
  • At most six tags occurred in a single label.
• Annotator agreement: on ICSI-MR, agreement became acceptable (kappa 0.8) only when labels were collapsed into five classes.
• Automatic tagging:
  • With 42 SWBD-DAMSL tags, earlier statistical work reached about 70%.
  • A small ICSI-MR tag subset reached about 78%, against a 61% baseline of always predicting "statement".
  • Preliminary MALTUS experiments reached 70%.
  • About 500 elicited user queries show that questions, requests and offers matter for MPR.

Interesting ideas

• I1. Writing each tagset's label syntax as rewriting rules makes it possible to count the space of legal labels exactly and to compare schemes formally.
• I2. Use a layered label: one mandatory exclusive primary function (statement, question, backchannel, hold) plus a few independent optional modifiers (answer polarity, politeness, attention). This keeps expressiveness while shrinking the search space.
• I3. Mapping between tagsets only works from the finer scheme to the coarser one, and it also needs the coarser scheme's exclusiveness rules stated explicitly.
• I4. The conversion procedure doubled as a validator, catching incoherent source labels such as two general tags in one label, which were reported back to the corpus maintainers.
• I5. Tagging accuracy is reported against both a uniform baseline and a majority-class baseline (61% for "statement"), and tagset design is checked against real user queries.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): the speech-act label grammar in prag could be specified the same way, so its combinatorial size and exclusiveness constraints become explicit and checkable.
• I2 (frames): speech-act output could carry one core act plus orthogonal flags such as politeness or restatement, instead of free combinations.
• I3 (tree): when converting UD dialects or treebanks, record which direction is lossless and which constraints the target imposes, rather than assuming symmetric mappings.
• I4 (learn): every silver-data or treebank conversion step could log constraint violations as a cheap error detector for the input data.
• I5 (learn): speech-act and frame classifiers should always be scored against a majority-label control, and label inventories should be pruned using what the answer stage actually needs.