Elamur Research⌕
☰
home / papers / 2011 / A General, Abstract Model of Incremental Dialogue Processing

A General, Abstract Model of Incremental Dialogue Processing

Summary

Dialogue systems that react before a user finishes speaking can produce backchannels like "uh-huh", fast turn-taking, collaborative utterance completion and self-corrections. However, designers had no shared vocabulary for describing how such incremental systems are built. Schlangen and Skantze address this with a conceptual framework rather than a new system or dataset. It defines abstract data types, operations on them and possible constraints on processing modules.

The model describes:

  • the topology of the module network,
  • the direction and "bandwidth" of information flow,
  • the granularity of incremental units (IUs),
  • how a module's input increments relate to its output increments,
  • how hypotheses are revised or committed.

With these notions, one can specify anything from a plain non-incremental pipeline to a fully incremental, asynchronous, parallel and predictive architecture. The authors apply the model to existing systems. They also discuss its use as the basis for inter-module communication, including a compact encoding of updates through positional nodes.

The paper reports no empirical scores. Its contribution is a common descriptive language that makes systems comparable and makes design trade-offs explicit. The authors deliberately avoid recommending one "best" architecture.

Key points

  • Core concept: the incremental unit (IU), the minimal chunk a system processes. IUs can be linguistic (for example, words) or time-based (for example, every 500ms), and they are related to the larger units they belong to.
  • Four dimensions of variation: modularity/topology (pipeline, pipeline with a parallel path, star), granularity, input/output relations, and revisability of hypotheses.
  • Two update schemes: explicit add, revoke and commit messages, or a positional-node network tracked by the pair NC (newest committed) and RE (right edge). The second scheme keeps messages constant in size, lets IUs stay immutable, and ties consumer states to producer states. This is illustrated with an ASR right buffer and a number-word parser.
  • Origin: a revised and substantially extended version of Schlangen and Skantze (2009), published in Dialogue & Discourse 2(1), pp. 83–111, and funded by the DFG Emmy Noether Programme.
  • Future work: covering more existing systems, modelling Levelt's speech generation model, modality fusion, and links to information-state grounding (Larsson and Traum, 2000) via self-corrections.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

86 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 84 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5087/dad.2011.105

@inproceedings{schlangen2011general,
  title = {{A General, Abstract Model of Incremental Dialogue Processing}},
  author = {Schlangen, David and Skantze, Gabriel},
  booktitle = {Dialogue \& Discourse},
  year = {2011},
  doi = {10.5087/dad.2011.105},
  url = {https://doi.org/10.5087/dad.2011.105}
}

References

38 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gregory Aist, James Allen, Ellen Campana, Lucian Galescu, Carlos A. Gomez Gallo, Scott Stoness, Mary Swift, and Michael K. Tanenhaus. Software architectures for incremental understanding of human speech. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), Pittsburgh… find
  2. Gregory Aist, James Allen, Ellen Campana, Carlos Gomez Gallo, Scott Stoness, Mary Swift, and Michael K. Tanenhaus. Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods. In Proceedings of Decalog 2007, the 11th International Worksho… find
  3. James Allen, George Ferguson, and Amanda Stent. An architecture for more realistic conversational systems. In Proceedings of the conference on intelligent user interfaces, Santa Fe, USA, June 2001. find
  4. Gerry Altmann and Mark Steedman. Interaction with context during human sentence processing. Cognition, 30:191–238, 1988. find
  5. Timo Baumann, Michaela Atterer, and David Schlangen. Assessing and improving the performance of speech recognition for incremental systems. In Proceedings of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT) 2009 Conference, Boulder… find
  6. Johan Boye, Beth Ann Hockey, and Manny Rayner. Asynchronous dialogue management: Two case-studies. In Proceedings of the 4th Workshop on Semantics and Pragmatics of Dialogue (Götalog2000), pages 51–55, Gothenburg, Sweden, 2000. find
  7. Timothy Brick and Matthias Scheutz. Incremental natural language processing for HRI. In Proceedings of the Second ACM IEEE International Conference on Human-Robot Interaction, pages 263–270, Washington, DC, USA, 2007. find
  8. Timothy Brick, Paul Schermerhorn, and Matthias Scheutz. Speech and action: Integration of action and language for mobile robots. In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS ’07), San Diego, CA, USA, October/November 2007. find
  9. Okko Buß and David Schlangen. Modelling sub-utterance phenomena in spoken dialogue systems. In Proceedings of the 14th International Workshop on the Semantics and Pragmatics of Dialogue (Pozdial 2010), pages 33–41, Poznan, Poland, June 2010. find
  10. Okko Buß, Timo Baumann, and David Schlangen. Collaborating on utterances with a spoken dialogue system using an isu-based approach to incremental dialogue management. In Proceedings of the SIGdial 2010 Conference, pages 233–236, Tokyo, Japan, September 2010. find
  11. Adam Cheyer and David Martin. The open agent architecture. Journal of Autonomous Agents and Multi-Agent Systems, 4(1):143–148, March 2001. OAA. find
  12. Thomas Dean and Mark Boddy. An analysis of time-dependent planning. In Proceedings of AAAI-88, pages 49–54. AAAI, 1988. find
  13. David DeVault and Matthew Stone. Domain inference in incremental interpretation. In Proceedings of ICOS 4: Workshop on Inference in Computational Semantics, Nancy, France, September 2003. INRIA Lorraine. find
  14. David DeVault, Kenji Sagae, and David Traum. Can I finish? learning when to respond to incremental interpretation results in interactive dialogue. In Proceedings of the 10th Annual SIGDIAL Meeting on Discourse and Dialogue (SIGDIAL’09), London, UK, September 2009. find
  15. Jens Edlund, Joakim Gustafson, Mattias Heldner, and Anna Hjalmarsson. Towards human-like spoken dialogue systems. Speech Communication, 50:630–645, 2008. find
  16. Jonathan Ginzburg, Raquel Fernández, and David Schlangen. Unifying self- and other-repair. In Proceeding of DECALOG, the 11th International Workshop on the Semantics and Pragmatics of Dialogue (SemDial07), Trento, Italy, June 2007. find
  17. Anne Kilger and Wolfgang Finkler. Incremental generation for real-time applications. Technical Report RR-95-11, DFKI, Saarbrücken, Germany, 1995. find
  18. Staffan Larsson and David Traum. Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering, pages 323–340, 2000. find
  19. Willem J.M. Levelt. Speaking. MIT Press, Cambridge, USA, 1989. find
  20. Maryellen C. MacDonald. Probabilistic constraints and syntactic ambiguity resolution. Language and Cognitive Processes, 9(2):157–201, 1994. find
  21. William D. Marslen-Wilson. Linguistic structure and speech shadowing at very short latencies. Nature, 244:522–523, August 1973. find
  22. Joakim Nivre. Incrementality in deterministic dependency parsing. pages 50–57, Barcelona, Spain, July 2004. find our card
  23. David Schlangen. What we can learn from dialogue systems that don’t work: On dialogue systems as cognitive models. In Proceedings of DiaHolmia, the 13th International Workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL 2009), pages 51–58, Stockholm, Sweden, June 2009. find
  24. David Schlangen and Gabriel Skantze. A general, abstract model of incremental dialogue processing. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2009), pages 710–718, Athens, Greece, March 2009. find
  25. David Schlangen, Timo Baumann, and Michaela Atterer. Incremental reference resolution: The task, metrics for evaluation, and a bayesian filtering model that is sensitive to disfluencies. In Proceedings of SIGdial 2009, the 10th Annual SIGDIAL Meeting on Discourse and Dialogue, London, UK, September… find
  26. David Schlangen, Timo Baumann, Hendrik Buschmeier, Okko Buß, Stefan Kopp, Gabriel Skantze, and Ramin Yaghoubzadeh. Middleware for incremental processing in conversational agents. In Proceedings of the SIGdial 2010 Conference, pages 51–54, Tokyo, Japan, September 2010. find
  27. William Schuler, Stephen Wu, and Lane Schwartz. A framework for fast incremental interpretation during speech decoding. Computational Linguistics, 35(3), 2009. find
  28. Gabriel Skantze and Anna Hjalmarsson. Towards incremental speech generation in dialogue systems. In Proceedings of the SIGdial 2010 Conference, pages 1–8, Tokyo, Japan, September 2010. find
  29. Gabriel Skantze and David Schlangen. Incremental dialogue processing in a micro-domain. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2009), pages 745–753, Athens, Greece, March 2009. find
  30. Scott C. Stoness, James Allen, Greg Aist, and Mary Swift. Using real-world reference to improve spoken language understanding. In Proceedings of Workshop on Spoken Language Understanding at AAAI05, Pittsburgh, PA, USA, 2005. find
  31. Patrick Sturt and Vincenzo Lombardo. Processing coordinated structures: Incrementality and connectedness. Cognitive Science, 29:291–305, 2005. find
  32. Michael K. Tanenhaus, Michael J. Spivey-Knowlton, Kathleen M. Eberhard, and Julie C. Sedivy. Intergration of visual and linguistic information in spoken language comprehension. Science, 268, 1995. find
  33. D. Traum and P. Heeman. Utterance units in spoken dialogue. In E. Maier, M. Mast, and S. LuperFoy, editors, Dialogue Processing in Spoken Language Systems, Lecture Notes in Artificial Intelligence. Springer-Verlag, 1997. find
  34. Jos J.A. van Berkum, Arnout W. Koornneef, Marte Otten, and Mante S. Nieuwland. Establishing reference in language comprehension: An electrophysiological perspective. Brain Research, 1146:158–171, 2007. find
  35. Peter van Roy and Seif Haridi. Concepts, Techniques, and Models of Computer Programming. MIT Press, Cambridge, Massachusetts, USA, 2004. find
  36. Jason Williams and Steve Young. Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21(2):231–422, 2007. find
  37. Mats Wirén. Studies in Incremental Natural Language Analysis. PhD thesis, Linköping University, Linköping, Sweden, 1992. find
  38. S.J. Young, N.H. Russell, and J.H.S. Thornton. Token passing: a conceptual model for connected speech recognition systems. Technical report CUED/FINFENG/TR 38, Cambridge University Engineering Department, 1989. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/schlangen-2011-general-abstract-model-incremental-dialogue/