Elamur Research⌕
☰
home / papers / 2015 / The Influence of Context on Dialogue Act Recognition

The Influence of Context on Dialogue Act Recognition

Paper · 2015 · Eugenio Ribeiro, Ricardo Ribeiro and David Martins de Matos

Summary

The paper asks how much the surrounding dialog helps a classifier decide which dialog act a segment performs, and how far back that help extends. The authors used the large Switchboard corpus and data following the ISO 24617-2 standard. The standard-annotated data came from the Tilburg DialogBank and from part of the LEGO (Let's Go) corpus, whose original labels were converted into ISO communicative functions. The classifier was an SVM. Its base features were n-grams, punctuation and wh-words. Context came from one to five earlier segments and was encoded in three ways: raw n-grams mixed with the current ones, n-grams tagged with their distance, or the dialog act labels of earlier segments. Only past information was used, so the setup suits live systems.

Mixing context words in without marking them usually hurt accuracy. Distance-tagged n-grams helped a lot, but only for the nearest segment. Previous act labels gave steady gains that fade with distance. The Switchboard results beat earlier published numbers, and the ISO results serve as a baseline for later work.

Key points

  • Datasets: Switchboard (about 224k segments, 44-tag set, with 41–44 tag variants in the literature), Tilburg DialogBank, and LEGO mapped onto ISO 24617-2 communicative functions.
  • Untagged n-gram context: this was generally harmful. The only exception was the single preceding segment in LEGO, which the authors attribute to the rigid wording of system turns.
  • Distance-tagged n-grams: these gave a large improvement from the first preceding segment and nothing visible beyond it.
  • Dialog act label context: this improved accuracy gradually, with influence shrinking roughly exponentially with distance and leveling off near the third segment. The same pattern appeared with automatically predicted labels, not just gold ones.
  • ISO dimensions: adding information from dimensions other than Task gave at most about 2 percentage points, mostly not significant. For comparison, an earlier HMM baseline (Stolcke et al.) reached 71.0% on Switchboard.

Interesting ideas

  • I1. Context features should be kept separate from current-segment features, either by tagging each with its distance or by using a different feature type, because blending them adds noise instead of signal.
  • I2. Compact summaries of earlier turns (their act labels) carried context better and more stably than their raw words, and about three turns back was enough.
  • I3. The authors checked that the context effect still held when gold labels were swapped for the system's own predictions, a realistic test of error propagation.
  • I4. Converting a domain-specific annotation scheme into a shared standard (LEGO into ISO 24617-2) produced extra comparable data and a reusable baseline.
  • I5. The authors restricted themselves to past context only, so the setup reflects what a live system actually knows at decision time.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): when crate prag classifies a speech act, features drawn from earlier sentences should carry a distance marker rather than being merged into the current sentence's evidence.
  • I2 (frames): the speech-act layer can condition on the act labels of the last two or three utterances instead of their full text, which keeps the decision small and easy to explain.
  • I3 (learn): any feature fed from an upstream Mova stage should be evaluated with predicted inputs as well as gold, so pipeline errors are measured rather than hidden.
  • I4 (learn): mapping existing dialog or intent corpora onto one standard act inventory could provide additional silver training and evaluation data for the speech-act component.
  • I5 (answer): readers that respond turn by turn should be tested without look-ahead, so reported accuracy matches real interactive use.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 401 worksSpeech act theory · 160 works

Cite

Cite as Eugenio Ribeiro, Ricardo Ribeiro and David Martins de Matos. 2015. The Influence of Context on Dialogue Act Recognition. arXiv:1506.00839.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1506.00839 · arXiv 1506.00839

@misc{ribeiro2015influence,
  title = {{The Influence of Context on Dialogue Act Recognition}},
  author = {Eugenio Ribeiro, Ricardo Ribeiro and David Martins {de Matos}},
  year = {2015},
  doi = {10.48550/arxiv.1506.00839},
  eprint = {1506.00839},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1506.00839}
}

References

46 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aha, David W., Dennis Kibler, and Marc K. Albert. 1991. Instance-Based Learning Algorithms. Machine Learning, 6(1):37–66. ↗
  2. Bangalore, Srinivas and Aravind K. Joshi. 1999. Supertagging: An Approach to Almost Parsing. Computational Linguistics, 25(2):237–265. Volume xx, Number xx
  3. Bard, E. G., C. Sotillo, A. H. Anderson, and M. M. Taylor. 1995. The DCIEM Map Task Corpus: Spontaneous Dialogue under Sleep Deprivation and Drug Treatment. In In Proceedings of the ESCA-NATO Tutorial and Workshop on Speech under Stress, pages 25–28. ↗
  4. Baum, L. E. and T. Petrie. 1966. Statistical Inference for Probabilistic Functions of Finite State Markov Chains. Annals of Mathematical Statistics, 37:1554–1563. ↗
  5. Benedí, José-Miguel, Eduardo Lleida, Amparo Varona, María-José Castro, Isabel Galiano, Raquel Justo, Iñigo Löpez De Letona, and Antonio Miguel. 2006. Design and Acquisition of a Telephone Spontaneous Speech Dialogue Corpus in Spanish: Dihana. In 5th LREC, pages 1636–1639. find
  6. Breiman, L. 1984. Classification and Regression Trees. Wadsworth statistics/probability series. Wadsworth International Group. find
  7. Brill, Eric. 1995. Transformation-Based Error-Driven Learning and Natural Language Processing: A Case Study in Part-of-Speech Tagging. Computational Linguistics, 21:543–565.
  8. Bunt, Harry, Jan Alexandersson, Jae-Woong Choe, Alex Chengyu Fang, Koiti Hasida, Volha Petukhova, Andrei Popescu-Belis, and David Traum. 2012. ISO 24617-2: A Semantically-Based Standard for Dialogue Annotation. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12).
  9. Bunt, Harry, Volha Petukhova, Andrei Malchanau, Kars Wijnhoven, and Alex Fang. 2016. The DialogBank. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016).
  10. Carletta, Jean, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, Guillaume Lathoud, Mike Lincoln, Agnes Lisowska, Iain McCowan, Wilfried Post, Dennis Reidsma, and Pierre Wellner. 2006. The AMI Meeting Corpus: A Pre-announcement. In Proceedings of the Second International Conference on Machine Learning for Multimodal Interaction, MLMI’05, pages 28–39, Berlin, Heidelberg. Springer-Verlag. find
  11. Carroll, John M. and Michael K. Tanenhaus. 1978. Functional Clauses and Sentence Segmentation. Journal of Speech, Language, and Hearing Research, 21(4):793–808. ↗
  12. Cortes, Corinna and Vladimir Vapnik. 1995. Support-Vector Networks. Machine Learning, 20(3):273–297. ↗
  13. Costantini, Erica, Susanne Burger, and Fabio Pianesi. 2002. NESPOLE!’s Multilingual and Multimodal Corpus. In LREC. European Language Resources Association. find
  14. Cover, T. and P. Hart. 1967. Nearest Neighbor Pattern Classification. IEEE Transactions on Information Theory, 13(1):21–27. Book Reviews ↗
  15. Dielmann, Alfred and Steve Renals. 2007. DBN Based Joint Dialogue Act Recognition of Multiparty Meetings. In IEEE International Conference on Acoustics Speech and Signal Processing, ICASSP’07, pages 133–136. find
  16. Fan, Rong-En, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research, 9:1871–1874. ↗
  17. Fang, A., J. Cao, H. Bunt, and X. Liu. 2012. The Annotation of the Switchboard Corpus with the New ISO Standard for Dialogue Act Analysis. In Proceedings of ISA-8, pages 13–18. find
  18. Friedman, Nir, Dan Geiger, and Moises Goldszmidt. 1997. Bayesian Network Classifiers. Machine Learning, 29(2-3):131–163. ↗
  19. Gambäck, Björn, Fredrik Olsson, and Oscar Täckström. 2011. Active Learning for Dialogue Act Classification. In INTERSPEECH, pages 1329–1332. ISCA. find
  20. Ghahramani, Zoubin. 1998. Learning Dynamic Bayesian Networks. In Adaptive Processing of Sequences and Data Structures, pages 168–197. Springer-Verlag. find
  21. Godfrey, J., E. Holliman, and J. McDaniel. 1992. SWITCHBOARD: Telephone Speech Corpus for Research and Development. In IEEE International Conference on Speech, and Signal Processing, ICASSP’92, volume 1, pages 517–520. find
  22. Hall, Mark, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA Data Mining Software: An Update. SIGKDD Exploration Newsletter, 11(1):10–18. ↗
  23. Jurafsky, D., E. Shriberg, and D. Biasca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coder manual. Technical Report Draft 13, University of Colorado, Institute of Cognitive Science. find
  24. Kay, M., J.M. Gawron, and P. Norvig. 1994. Verbmobil, A Translation System for Face-to-Face Dialog. CSLI, Stanford, CA. find
  25. Keizer, S. 2002. Dialogue Act Classification: Experiments with the SCHISMA Corpus. Technical report, Computational Linguistics University of Twente, October. find
  26. Keizer, Simon, Rieks op den Akker, and Anton Nijholt. 2002. Dialogue Act Recognition with Bayesian Networks for Dutch Dialogues. In Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue - Volume 2, SIGDIAL ’02, pages 88–94, Stroudsburg, PA, USA. Association for Computational Linguistics.
  27. Lafferty, John D., Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, pages 282–289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  28. Levin, Lori, Chad Langley, Alon Lavie, Donna Gates, Dorcas Wallace, and Kay Peterson. 2003. Domain Specific Speech Acts for Spoken Language Translation. In Proceedings of the 4th SIGdial Workshop on Discourse and Dialogue. find
  29. McCulloch, Warren S. and Walter Pitts. 1988. Neurocomputing: Foundations of research. MIT Press, Cambridge, MA, USA, chapter A Logical Calculus of the Ideas Immanent in Nervous Activity, pages 15–27. find
  30. Metropolis, Nicholas and S. Ulam. 1949. The Monte Carlo Method. Journal of the American Statistical Association, 44(247):335–341. ↗
  31. Platt, John. 1998. Fast Training of Support Vector Machines using Sequential Minimal Optimization. In Advances in Kernel Methods - Support Vector Learning. MIT Press. ↗
  32. Quinlan, J. Ross. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc.
  33. Raux, Antoine, Dan Bohus, Brian Langner, Alan W Black, and Maxine Eskenazi. 2006. Doing Research on a Deployed Spoken Dialogue System: One Year of LetâĂŹs Go! Experience. In Proceedings of INTERSPEECH, 2006, pages 65–68. find
  34. Ribeiro, Eugénio, Ricardo Ribeiro, and David Martins de Matos. 2016. Mapping the Dialog Act Annotations of the LEGO Corpus into the Communicative Functions of ISO 24617-2. ↗
  35. Rosenblatt, F. 1962. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Volume xx, Number xx Books, Washington. ↗
  36. Rotaru, M. 2002. Dialog Act Tagging Using Memory-Based Learning. Technical report, University of Pittsburgh. find
  37. Samuel, Ken, Sandra Carberry, and K. Vijay-shanker. 1998. Dialogue Act Tagging with Transformation-Based Learning. In Proceedings of the 36th Annual Meeting of the ACL and 17th Conference on Computational Linguistics, pages 1150–1156. ↗
  38. Schmitt, Alexander, Stefan Ultes, and Wolfgang Minker. 2012. A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let’s Go Bus Information System. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Mehmet Ugur Dogan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Eight International Conference on Language Resources and Evaluation, LREC’12, Istanbul, Turkey. European Language Resources Association (ELRA). find
  39. Searle, John R. 1969. Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, Cambridge, London. ↗
  40. Shriberg, Elizabeth, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The ICSI Meeting Recorder Dialog Act (MRDA) Corpus. In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue, pages 97–100. ↗
  41. Sridhar, Vivek Kumar Rangarajan, Srinivas Bangalore, and Shrikanth Narayanan. 2009. Combining Lexical, Syntactic and Prosodic Cues for Improved Online Dialog Act Tagging. Computer Speech and Language, 23(4):407–422. ↗
  42. Stolcke, Andreas, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics, 26(3):339–373. ↗
  43. Stone, Matthew. 2002. Communicative Intentions and Conversational Processes in Human-Human and Human-Computer Dialogue. In World Situated Language Use: Psycholinguistic, Linguistic, and Computational Perspectives on Bridging the Product and Action Traditions, pages 39–70. find
  44. Webb, Nick and Michael Ferguson. 2010. Automatic Extraction of Cue Phrases for Cross-corpus Dialogue Act Classification. In COLING, pages 1310–1317.
  45. Wilcoxon, Frank. 1945. Individual Comparisons by Ranking Methods. Biometrics Bulletin, 1(6):80–83. ↗
  46. Wright, Helen. 1998. Automatic Utterance Type Detection Using Suprasegmental Features. In ICSLP. ISCA. Book Reviews find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ribeiro-2015-influence-context-dialogue-act-recognition/