Elamur Research⌕
☰
home / papers / 2009 / Unsupervised Classification of Dialogue Acts using a Dirichlet Process Mixture Model

Unsupervised Classification of Dialogue Acts using a Dirichlet Process Mixture Model

Paper · 2009 · Nigel Crook, Ramon Granell and Stephen Pulman

Summary

Hand-labelling dialogue corpora with Dialogue Act (DA) tags is slow and costly. This paper tests whether utterances can be grouped automatically, with no labels, so that the groups line up with DA categories. The authors use a Dirichlet Process Mixture Model (DPMM) through the Chinese Restaurant Process (CRP). Because the model is nonparametric, it infers how many clusters to create instead of requiring that number beforehand. Each utterance is represented as a bag of words, and each cluster is a multinomial over the vocabulary with a Dirichlet prior. The data is the Spanish Dihana corpus of Wizard-of-Oz train-information dialogues, which carries a three-level DA annotation. The clusters are scored against a random-clustering baseline, using similarity measures and two error measures. The CRP separated system turns from user turns almost perfectly. Many user clusters matched first-level acts such as Answering and Asking, and some matched second-level pairs such as Answering:Day. This suggests unsupervised DA discovery is feasible, although the authors note the domain is narrow.

Key points

  • Dihana has 900 dialogues and an 823-word vocabulary. After named-entity replacement and separating user words from system words, there are 964 distinct tokens.
  • The annotation has three levels, giving 248 full labels (153 user, 95 system). Levels 1+2 give 72 labels (45 user, 27 system), and level 1 alone gives 16 labels (7 user, 9 system).
  • The CRP was run for 0 epochs (a random baseline), 1000K epochs and 1500K epochs. The 1000K run found 44 clusters and the 1500K run found 54, against an ideal of 72.
  • In the 1500K run, 30 clusters held only system utterances, 22 held only user utterances and 2 were mixed.
  • Cluster 15 merged Closing and Rejection, since the same phrasing (e.g. "No thank you") can serve either act. Future work proposes n-grams, syntactic features, the DA of the previous utterance and the Switchboard corpus.

Interesting ideas

  • I1. A Dirichlet-process prior lets the data decide the number of categories, which avoids repeated runs over a grid of cluster counts when the true inventory is unknown.
  • I2. The authors score the untrained (0-epoch) random clustering with the same metrics, so every later number is compared against chance.
  • I3. Evaluation uses two complementary errors: one per cluster (how impure it is) and one per gold class (how scattered that class is across clusters). This catches both mixed clusters and over-split classes.
  • I4. Looking at which gold labels end up sharing a cluster exposes acts that the surface form cannot tell apart, such as closing versus rejection.
  • I5. A hierarchical label set (act, then topic slot, then value) is scored separately at each level, which shows how deep the unsupervised signal reaches.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova could cluster unexplained repair cases or failed parses with a CRP to discover how many distinct error types exist, rather than fixing a taxonomy up front.
  • I2 (learn): every unsupervised or self-induced component in Mova could report its scores next to a random-assignment negative control on the same data.
  • I3 (learn): when induced frame or sense groups are compared to VerbNet or FrameNet classes, Mova can report both directions so splitting and merging failures are visible.
  • I4 (frames): the speech-act layer in prag can treat such confusable pairs as cases that need dialogue context or world state, and log which context feature resolved each one.
  • I5 (frames): Mova's speech-act labels could be checked level by level, from the coarse act down to its role or slot, to locate where interpretation starts to fail.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of OxfordOxford · United Kingdom104 works · 157,183 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 624 worksSpeech act theory · 229 works

Cite

Cite as Nigel Crook, Ramon Granell and Stephen Pulman. 2009. Unsupervised Classification of Dialogue Acts using a Dirichlet Process Mixture Model. Proceedings of the SIGDIAL 2009 Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{crook2009unsupervised,
  title = {{Unsupervised Classification of Dialogue Acts using a Dirichlet Process Mixture Model}},
  author = {Nigel Crook, Ramon Granell and Stephen Pulman},
  booktitle = {Proceedings of the SIGDIAL 2009 Conference},
  year = {2009},
  doi = {10.3115/1708376.1708427},
  url = {https://doi.org/10.3115/1708376.1708427}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Toine Andernach, Mannes Poel, and Etto Salomons. 1997. Finding classes of dialogue utterances with kohonen networks. In In Daelemans, pages 85–94. find
  2. J.A. Andernach. 1996. A machine learning approach to the classification and prediction of dialogue utterances. In Proceedings of the 2nd International Conference on New Methods in Language Processing, pages 98–109. find
  3. Charles E. Antoniak. 1974. Mixtures of dirichlet processes with applications to bayesian nonparametric problems. The Annals of Statistics, 2(6):1152–1174. ↗
  4. J.L. Austin. 1962. How to do things with words. Oxford: Clarendon Press. find
  5. Michael D. Escobar and Mike West. 1995. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90(430):577–588. ↗
  6. Sheila Garfield and Stefan Wermter. 2006. Call classification using recurrent neural networks, support vector machines and finite state automata. Knowl. Inf. Syst., 9(2):131–156. ↗
  7. J. J. Godfrey, E. C. Holliman, and J. Mcdaniel. 1992. SWITCHBOARD: telephone speech corpus for research and development. In Proc. ICASSP, volume 1, pages 517–520 vol.1. find
  8. Gang Ji and J. Bilmes. 2005. Dialog act tagging using graphical models. In Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP ’05). IEEE International Conference on, volume 1, pages 33– 36. find
  9. J.M.Benedı́, E.Lleida, A. Varona, M.J.Castro, I.Galiano, R.Justo, I. López, and A. Miguel. 2006. Design and acquisition of a telephone spontaneous speech dialogue corpus in spanish: Dihana. In Fifth International Conference on Language Resources and Evaluation (LREC), pages 1636–1639, Genova, Italy, May. find
  10. S. Keizer, M. Gasic, F. Mairesse, B. Thomson, K. Yu, and S. Young. 2008. Modelling user behaviour in the his-pomdp dialogue manager. In IEEE SLT, pages 121–124, Dec. find
  11. Steven N. Maceachern and Peter Müller. 1998. Estimating mixture of dirichlet process models. Journal of Computational and Graphical Statistics, 7(2):223–238. ↗
  12. C. D. Martı́nez-Hinarejos, J. M. Benedı́, and R. Granell. 2008. Statistical framework for a spanish spoken dialogue corpus. Speech Communication, 50:992–1008. ↗
  13. Geoffrey Mclachlan and David Peel. 2000. Finite Mixture Models. Wiley Series in Probability and Statistics. Wiley-Interscience, October. ↗
  14. Gabriel Murray, Steve Renals, Jean Carletta, and Johanna Moore. 2006. Incorporating speaker and discourse features into speech summarization. In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 367–374, Morristown, NJ, USA. Association for Computational Linguistics. find
  15. J. Pitman and M. Yor. 1997. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25(2):855–900. ↗
  16. J. Pitman. 2002. Combinatorial stochastic processes. find
  17. Fabrizio Sebastiani. 2002. Machine learning in automated text categorization. ACM Comput. Surv., 34(1):1–47, March. ↗
  18. Andreas Stolcke, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Comput. Linguist., 26(3):339–373. ↗
  19. E. Zovato and J. Romportl. 2008. Speech synthesis and emotions: a compromise between flexibility and believability. In Proceedings of Fourth International Workshop on Human-Computer Conversation, Bellagio, Italy. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/crook-2009-unsupervised-classification-dialogue-acts-dirichlet/