📄 Dialogue Act Tagging with Transformation-Based Learning

Summary

The paper addresses automatic recognition of dialogue acts: labels such as SUGGEST, ACCEPT or REJECT that capture what a speaker intends, which literal wording often fails to reveal. The authors adapt Brill's Transformation-Based Learning (TBL), which until then had not been used for discourse. TBL learns an ordered list of readable relabeling rules from templates. To avoid sparse data, utterances are described by abstract features: speaker change, punctuation, utterance length, and the acts on neighbouring utterances. The authors also introduce "dialogue act cues", which are 1–3-word substrings chosen automatically because the acts they co-occur with have low entropy. These cues are then pruned by a redundancy filter and grouped by semantic clustering. Two further additions address TBL's limits: a Monte Carlo strategy that speeds up training, and a committee of models that yields confidence scores. On the VERBMOBIL appointment-scheduling data, the system matches the best previously reported accuracy (Reithinger and Klesen). Automatically mined cues beat both hand-made cue phrase lists and the full set of word n-grams, which shows that selecting features carefully matters for rule learners.

Key points

• Direct comparison used Reithinger and Klesen's VERBMOBIL split: 143 training dialogues (2701 utterances) and 20 test dialogues (328 utterances), labeled with 18 dialogue acts.
• The full system averaged 75.12% accuracy over five runs (σ = 1.34%), with a best run of 77.44%. Settings were improvement threshold 1 and Monte Carlo sample size R = 14.
• An ablation varied the word-substring features:
  • no substrings: 41.16%
  • literature cue phrases (Hirschberg & Litman, Knott, plus 45 in-house): 61.74%
  • all 1–3-word n-grams: 69.21%
  • entropy-selected cues with filtering and clustering: 71.22%, using far fewer substrings
• The cue set C keeps a substring s only if H(D|s) is below one threshold and its corpus count is above another. Cues fall into three kinds: traditional ("but", "so"), potential ("thanks", "see you") and domain-specific ("what time", "busy").
• In one run, TBL produced a model of 213 rules. Its first rule tags every utterance SUGGEST, and later rules make conditional fixes, e.g. "no" after a SUGGEST becomes REJECT.

Interesting ideas

• I1. Select lexical cue features by how sharply they predict the label: keep a short phrase only when the labels it co-occurs with have low entropy and it occurs often enough, which avoids both incomplete hand lists and a flood of noisy n-grams.
• I2. Prune a long cue if a shorter cue it contains already predicts the label equally well or better. The longer form adds only redundancy, and fewer candidates means fewer chance-selected spurious rules.
• I3. TBL builds an ordered list of human-readable "if context, then relabel" rules. It starts from a default tag and keeps greedily adding the rule with the best net gain until the gain falls below a threshold, so the learned model doubles as an explanation.
• I4. Encode the speaker as a simple boolean, "speaker changed from previous utterance", rather than pairing the act label with speaker direction. This keeps the label space small and generalizes across speakers.
• I5. Train several models into a committee and read their agreement as a confidence score. The authors propose using high-confidence outputs on untagged data as seed examples for weakly supervised learning.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Speech-act and irony detection in prag could mine its trigger phrases from labeled dialogue this way, so each act decision cites a specific cue with a known entropy.
• I2 (learn): Self-induced repair rules could undergo the same subsumption check, dropping any rule whose condition is a strict specialization of an equally good general rule.
• I3 (learn): This suits explainable repair layers on top of the parser or tagger, where every correction must be traceable to a named rule with a measured improvement on silver data.
• I4 (frames): For dialogue in fables and tales, the turn-change flag could serve as a cheap context feature when assigning speech acts like reply, refusal or acceptance.
• I5 (learn): This matches the multi-witness silver-data scheme: only labels on which several witness models agree would be promoted to training data, with disagreements sent to an LLM teacher tier.