📄 Latent Semantic Analysis for Dialogue Act Classification

Summary

The paper asks whether Latent Semantic Analysis (LSA), a method that works only from word co-occurrence, can serve as a dialogue act classifier, and whether adding non-lexical information improves it. The authors test four variants. Plain LSA labels a test utterance with the tag of its nearest training utterance by cosine. Clustered LSA builds a word-by-tag matrix. SemiClustered LSA projects tag aggregates into a space learned from documents. FLSA appends the preceding dialogue acts as extra rows of the matrix. They evaluate on DIAG, a small corpus of computer-mediated tutoring dialogues, and on Spanish CallHome, whose tags they collapse into CallHome37 and CallHome10. All variants of plain LSA beat the majority-class baseline by a wide margin. SemiClustered LSA wins on the small DIAG data but fails badly on the larger CallHome tag sets. Adding previous acts through FLSA did not help. The work matters as an early, cheap baseline for dialogue act tagging and as an honest test of the claim that bag-of-words models benefit from extra features.

Key points

• DIAG has 23 dialogues, 607 distinct words and 660 tutor moves. Its four tags are problem solving, judgement, domain knowledge and other.
• Spanish CallHome has 128 phone calls, 12,066 distinct words, 44,628 dialogue acts and 232 original tags. The authors collapsed these into CallHome37 and CallHome10 (8 acts plus "%" for abandoned utterances and "x" for noise).
• Plain LSA reached 75.73% on DIAG (k=50), 65.36% on CallHome37 (k=50) and 68.91% on CallHome10 (k=25). The majority baselines were 43.64% and 42.69%.
• SemiClustered LSA gave the best DIAG result at 78.78%. Clustered LSA dropped to 22.08% on CallHome37.
• FLSA scored slightly below plain LSA everywhere (74.26%, 62.59%, 66.57%). Using more than one previous act gave no gain. A prior neural-network result of 76.2% (Ries, 1999) is hard to compare because its tag set is unclear and it used POS tags.

Interesting ideas

• I1. Non-word evidence, such as the previous dialogue act, can be stacked as extra rows next to word counts so that both live in one shared space; here this did not pay off, which is itself informative.
• I2. Each class can be represented by a single prototype vector, ideally projected into a space learned from individual examples. Matching against a few prototypes is much faster than matching against every training item, and it worked best when data and label sets were small.
• I3. Rare fine-grained tags can be collapsed into several coarser levels (232 → 37 → 10), with the model evaluated at each level, because method rankings change with granularity.
• I4. Every score is paired with a majority-class baseline, and the authors note that comparisons with prior work break down when the target label set is not stated.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): when prag picks a speech act, the previous act could be one explicit, named input next to tree-based cues, and its real contribution should be measured rather than assumed.
• I2 (frames): per-speech-act prototype profiles built from frame features could give a fast nearest-prototype check whose output is easy to inspect and explain.
• I3 (learn): speech-act and UD-dialect evaluations could be reported at several explicitly documented label granularities so results stay comparable across runs.
• I4 (learn): each evaluation table could carry a most-frequent-label control and a written label-set definition alongside the negative controls already used.