📄 A Study of a Segmentation Technique for Dialogue Act Assignation (short paper)

Summary

Dialogue systems need to attach Dialogue Act (DA) labels to the utterances inside each user turn. Most earlier work assumed the turns were already split into utterances, but real systems and most corpora do not provide that split. The paper uses an HMM plus N-gram model that finds the best DA sequence for a turn. It can work either with a fixed segmentation or by searching over all possible segmentations. It also adds NGT (N-gram transducers), a segmenter built on the GIATI idea. Words and utterance-boundary marks are merged into one token stream, an N-gram is trained on that stream, and Viterbi decoding then uses the N-gram as a transducer.

Experiments on SwitchBoard and Dihana (2- and 3-level labels) used cross-validation. NGT segments turns more accurately than the HMM's implicit segmentation, but this gives a clear DA gain only on SwitchBoard. The main finding is that predicting the right number of utterances matters more than placing the boundaries exactly. This points toward future models that first estimate how many utterances a turn contains.

Key points

• Data and setup: SwitchBoard and Dihana (2-level and 3-level labels), with cross-validation. There are three conditions: correct (manual) segmentation, NGT segmentation, and unsegmented turns.
• Segmentation error (bigram):
  • NGT: 26.1 / 9.6 / 13.6 on SwitchBoard / Dihana-2 / Dihana-3.
  • HMM-based model: 41.4 / 14.2 / 38.8.
  • This is a relative reduction above 20%.
• Error in the number of utterances:
  • NGT: 23.0 / 9.6 / 12.8.
  • HMM: 34.7 / 6.0 / 17.3.
  • The HMM does better only on Dihana 2-levels.
• Whole-turn DA error (bigram):
  • Correct segmentation: 38.3 / 21.6 / 24.6.
  • NGT: 48.1 / 27.1 / 30.6.
  • Unsegmented: 59.6 / 24.6 / 32.5.
• DAER:
  • Correct segmentation: 38.3 / 20.1 / 24.7.
  • NGT: 43.2 / 29.0 / 30.6.
  • Unsegmented: 60.7 / 24.0 / 33.2.
  • On Dihana 2-levels, the unsegmented approach beats NGT.

Interesting ideas

• I1. When no segmentation is available, the decoder can search over all splits, so the boundaries come out as a side product of choosing the best label sequence. This avoids committing early to a separate segmenter.
• I2. The GIATI-style relabelling merges each input word with its output tag (here a boundary mark) into one combined symbol. A plain N-gram trained on these symbols then becomes a cheap sequence transducer that Viterbi can decode.
• I3. The error analysis separates two things: getting the count of segments wrong and getting the boundary positions wrong. Only the first strongly drives labelling errors.
• I4. The proposed future step is to predict how many units a turn contains first, then use that number to prune the segmentation search.
• I5. Two complementary metrics are used: an exact whole-turn match, and DAER, an edit-distance rate at the label level. Together they show whether errors are total or partial.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): speech-act labelling in prag could jointly decide where one act ends and the next begins, rather than trusting sentence splits blindly.
• I2 (tokens): a small, inspectable boundary model for sentence or clause splitting in crate en, where every boundary decision can be traced to N-gram counts.
• I3 (learn): evaluation of splitters and chunkers should report count errors separately, since a wrong count makes every downstream alignment fail.
• I4 (frames): estimating the number of speech acts or clauses beforehand could constrain the frame builder and cut implausible analyses.
• I5 (learn): report both strict per-sentence correctness and an edit-distance score for frame or act sequences, so near-misses are visible.