📄 Cross-Domain Dialogue Act Tagging

Summary

The paper asks whether dialogue act (DA) cues learned automatically from one corpus still work on conversations from a different domain. The authors had earlier built a very simple DA classifier that uses only features inside the utterance. It scores word n-grams of length 1–4 by how strongly each one predicts a DA label, which is the conditional probability P(d | n). On SWITCHBOARD, this classifier matched far heavier HMM-based models.

Here they take the cues learned from SWITCHBOARD, which consists of open-topic chats between strangers, and apply them to the AMITIÉS GE corpus, which consists of transactional banking calls to call centres. Both corpora are annotated with DAMSL-derived tag sets. The cues turn out to port well across the domains. Many highly ranked cues are generic markers of conversational function, such as "can you" for yes/no questions or "ok" for agreement, rather than domain vocabulary. The authors suggest that the model's simplicity may be part of the reason it transfers so well. This matters because labelled dialogue data is expensive, and cues that transfer reduce the annotation needed for each new application.

Key points

• SWITCHBOARD data: 1155 annotated conversations. The 220 fine-grained labels are clustered into 42 classes, following Jurafsky et al. (1998). Inter-annotator kappa is about .80.
• AMITIÉS GE data: 1000 English human-human call-centre dialogues (balances, payments, lost cards). The data is anonymised and annotated with XDML, a DAMSL variant combined with domain semantics. Overall kappa is only .59, rising to about .70 for major classes. The most frequent tag, Information-request, covers 20% of utterances.
• Method:
  • Candidate cues are all 1–4-grams in the training data.
  • Each n-gram is kept according to its maximal predictivity over DA labels.
  • Predictivity and frequency thresholds prune the list. These thresholds are tuned on a validation set, never on the test data.
• Classification:
  • An utterance gets the label of its single most predictive cue.
  • Ties go to the cue with the higher occurrence count, and any remaining ties are broken at random.
• Baseline and future work: The reference point is Stolcke et al. (2000), with 71.0% on SWITCHBOARD (198k training / 4k test utterances). Planned work includes merging both corpora for cue learning and testing on VERBMOBIL and ICSI-MRDA.

Interesting ideas

• I1. Scoring each n-gram by its best P(label | n-gram) produces both a classifier and a ranked, human-readable list of cues, so every decision can be traced to one phrase.
• I2. Applying a minimum-frequency threshold alongside predictivity removes rare n-grams that look perfectly predictive only by chance, and these are what distort results.
• I3. Training on one domain and testing on a structurally different one is a direct check of whether a model learned general function or memorised topic words.
• I4. Thresholds are set automatically on held-out validation data, keeping the test set untouched and the procedure reproducible.
• I5. A fixed, deterministic tie-break order (predictivity, then frequency, then chance) keeps a simple rule-based classifier well defined.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): crate prag could tag speech acts from a compact, inspectable cue table, with the winning cue recorded as the explanation.
• I2 (learn): the same double threshold could filter self-induced repair rules, so that a rule seen only a few times cannot fire with false confidence.
• I3 (learn): evaluations could routinely pair in-domain tests with an out-of-domain set (for example fables versus school tests) as a control against domain leakage.
• I4 (learn): parser confidence and silver-data agreement cut-offs could be tuned the same way instead of by hand.
• I5 (frames): when several speech-act or implicature rules conflict, an explicit priority chain makes the chosen reading reproducible and easy to explain.