📄 A Universality-Individuality Integration Model for Dialog Act Classification

Summary

The paper addresses dialog act classification: deciding which communicative function an utterance serves (question, answer, statement, backchannel and so on). The authors argue that word embeddings alone miss useful signals. They add two other cues: part-of-speech tags, and a small hand-built "statistics" vector that marks which punctuation is present and which length bucket the utterance falls into. These three inputs differ in kind, so simply concatenating them may hide their relationships. The proposed UIIM model therefore maps each cue into two representations. A shared universality vector is pushed toward similarity across cues, and a private individuality vector keeps what is specific to that cue. The fused result is passed to a BiLSTM over the conversation and then to a classifier. On SwDA and MRDA, BERT+UIIM reaches 78.6% and 89.9% accuracy, which the authors report as state of the art. The work shows that cheap linguistic signals, combined carefully, still add value on top of strong pretrained encoders.

Key points

• Benchmarks are SwDA (Switchboard) and MRDA (ICSI Meeting Recorder), evaluated with accuracy on the standard train/validation/test splits.

• The three input cues are:

  • word vectors, from 300-d fastText or 768-d BERT;
  • one-hot POS tags, taken from SwDA annotations or produced with NLTK for MRDA, encoded by an LSTM;
  • a fixed-length binary vector of punctuation presence plus utterance-length intervals of size 5, with an open last bucket.

• Each cue is first projected to a common hidden size and then split into universality and individuality vectors. Context comes from a BiLSTM across utterances.

• Accuracy results:

  Model	SwDA	MRDA
  BERT+UIIM	78.6	89.9
  BERT only	77.9	89.2
  fastText+UIIM	76.4	89.3
  fastText only	75.6	88.5

  The headline margins of +3.5 and +5.2 points are measured against older published baselines. The gain over the authors' own BERT-only baseline is about 0.7 points.

• Reported hyperparameters: batch 64, dropout 0.3, learning rate 1e-4, hidden size 224, loss weights α/β/γ = 1/0.7/0.7. Mini-batches group conversations that have the same number of utterances.

Interesting ideas

• I1. A tiny surface vector can carry real signal for utterance function: binary flags for each punctuation mark plus a bucketed length, since very short replies tend to be answers or acknowledgements.
• I2. Give each heterogeneous input a shared projection constrained toward agreement, plus a private one that keeps its own character, instead of naively concatenating them.
• I3. POS acts as a disambiguator for words whose function changes with category, such as "bear" as a verb versus a noun, so even a crude one-hot tag sequence helps a downstream classifier.
• I4. Always report the gain over your own plain-encoder baseline: here that gain is much smaller than the gain over older published systems.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): prag's speech-act detector could use punctuation flags and length buckets as explicit, inspectable cues alongside the UD tree, for example to separate "OK." acknowledgements from full assertions.
• I2 (learn): when several witness parsers produce silver data, separating what they agree on from what each contributes uniquely could guide which disagreements to trust or inspect.
• I3 (tokens): this supports keeping tagger output as a first-class feature exposed to later stages, not only as an intermediate step for the parser.
• I4 (learn): our evaluations should pair every added component with an ablation on the identical backbone, so that improvements are not credited to a new technique when they actually come from a stronger encoder.