Elamur Research⌕
☰
home / papers / 2018 / Prosodic entrainment in dialog acts

Prosodic entrainment in dialog acts

Paper · 2018 · Uwe D. Reichel, Katalin Mady and Jennifer Cole

Summary

The paper asks whether prosodic entrainment, the tendency of dialog partners to converge in how they sound, varies with the function of an utterance. Earlier work usually compared turns or fixed time windows rather than dialog acts. The authors used a dialog-act annotated subset of the Illinois Game Corpus: American English pairs playing a Tangram same-or-different task, first cooperatively and then competitively. For each dialog act they measured entrainment on intonation features from a parametric superpositional stylization, as well as on energy and rhythm features. They then tested four hypotheses about authority, supportiveness, frequency and local predictability.

The results did not confirm the hypotheses about low-authority acts or unpredictable acts. Frequency helped only weakly, and only in cooperative play. The clearest finding is that "authority" splits into two kinds. Knowledge-based authority (explaining, clarifying) goes with entrainment in cooperative games. Executive authority (instructing) goes with disentrainment. Because entrainment patterns shift with both dialog act and game condition, the authors argue that entrainment is partly under active speaker control, not purely automatic.

Key points

  • Data: 16 dialogs (10 cooperative, 6 competitive) from 11 speaker pairs in the Illinois Game Corpus.
    • Mean dialog length is about 7 min 40 s.
    • The analysis covers 4011 dialog act segments.
    • Two annotators labeled the acts from transcripts only and resolved disagreements by discussion.
  • Dialog acts: the tag set describes conversational moves, including AC, AL, CH, CL, EX and IN. Prosody was characterized with stylization-based feature sets such as GF0 and ACC (pitch accents), alongside energy and rhythm measures.
  • Hypothesis outcomes:
    • H1 (more entrainment in low-authority acts) was rejected.
    • H2 (supportiveness) showed a more complex interaction with authority.
    • H3 (frequency) was partly confirmed, for cooperative dialogs only.
    • H4 (disentrainment in locally unpredictable acts) was rejected.
  • Cooperative games: EX and CL entrained, while IN strongly disentrained. In competitive games IN showed more entrainment.
  • EX and CL across conditions: cooperative dialogs had one disentraining feature set (GF0 for EX), while competitive dialogs had six. ACC disentrained for both acts in competition, possibly to make new information harder to process.

Interesting ideas

  • I1. Splitting authority into "authority from knowing" and "authority from commanding" exposed opposite behaviors that a single authority label had hidden.
  • I2. Measuring an effect per functional unit (dialog act) instead of per turn or time window revealed differences that would have averaged out.
  • I3. Running the same measurement under two contrasting conditions (cooperative vs competitive) lets the authors separate automatic behavior from deliberate behavior.
  • I4. The related-work section notes that an unsigned synchrony measure, which counts only the amount of convergence or divergence, predicted engagement better than a signed one.
  • I5. Using a parametric stylization lets the authors compare whole dynamic contours rather than simple means or maxima.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): speech-act frames in prag could carry an authority-type attribute (informative vs directive), so later reasoning about power or politeness does not merge the two.
  • I2 (learn): evaluation scores and witness-parser agreement could be broken down by clause or speech-act type, to find where errors concentrate instead of reporting one global number.
  • I3 (learn): a contrast condition could serve as a negative control, for example checking that a repair rule fires differently on text where it should not apply.
  • I4 (learn): when comparing witness parsers, the size of their divergence could be tracked separately from its direction, as a signal of uncertain sentences.
  • I5 (back): generated text could be checked against a structural profile of the source (clause shapes, information-status marking) rather than surface averages, to test whether the compressor preserves style.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Hungarian Academy of SciencesBudapest · Hungary4 works · 643 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 works

🧭 Topics 4Research subjects and languages the work is about.English · 1,373 worksDialogue systems · 401 worksSpeech & spoken language · 233 worksSpeech act theory · 160 works

Cite

Cite as Uwe D. Reichel, Katalin Mady and Jennifer Cole. 2018. Prosodic entrainment in dialog acts. arXiv:1810.12646.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1810.12646 · arXiv 1810.12646

@misc{reichel2018prosodic,
  title = {{Prosodic entrainment in dialog acts}},
  author = {Uwe D. Reichel, Katalin Mady and Jennifer Cole},
  year = {2018},
  doi = {10.48550/arxiv.1810.12646},
  eprint = {1810.12646},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1810.12646}
}

References

70 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Babel, M. and D. Bulatov: The role of fundamental frequency in phonetic accommodation. Language and Speech, 55:231–248, 2012. find
  2. Bates, D., M. Mächler, B. Bolker and S. Walker: Fitting Linear Mixed-Effects Models Using lme4 . Journal of Statistical Software, 67(1):1–48, 2015.
  3. Beňuš, Š.: Conversational Entrainment in the Use of Discourse Markers. In Bassis, S., A. Esposito and F. Morabito (eds.): Recent Advances of Neural Networks Models and Applications, Smart inovations, systems, and technologies, vol. 26, pp. 345–352. Springer, 2014. find
  4. Beňuš, Š.: Social aspects of entrainment in spoken interaction. Cognitive Computation, 6(4), 2014. find
  5. Beňuš, Š., R. Levitan and J. Hirschberg: Entrainment in spontaneous speech: The case of filled pauses in Supreme Court hearings. In Proc. 3rd IEEE conference on cognitive infocommunications, pp. 793–797, Košice, Slovakia, 2012. find
  6. Bocheński, J.: Was ist Autorität? Einführung in die Logik der Autorität. Herder, Freibug, Germany, 1974. find
  7. Boersma, P. and D. Weenink: PRAAT, a system for doing phonetics by computer . Techn. Rep., Institute of Phonetic Sciences of the University of Amsterdam, 1999. 132–182. find
  8. Branigan, H., M. Pickering, J. McLean and A. Cleland: Participant role and syntactic alignment in dialogue. Cognition, 104:163–197, 2007. find
  9. Brennan, S. and H. Clark: Conceptual pacts and lexical choice in conversation. J Exp Psychol Learn Mem Cogn, 22(6):1482–93, 1996. find
  10. Carletta, J., A. Isard, S. Isard, J. Kowtko, G. Doherty-Sneddon and A. Anderson: HCRC Dialogue Structure Coding Manual (HCRC/TR-82). Human Communication Research Centre, University of Edinburgh, Edinburgh, Scotland, 1996. find
  11. Carletta, J., A. Isard, S. Isard, J. Kowtko, G. Doherty-Sneddon and A. Anderson: The reliability of a dialogue structure coding scheme. Computational Linguistics, 23(1):13–31, 1997.
  12. Chartrand, T. and J. Bargh: The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76(6):893–910, 1999. find
  13. Clark, H. and E. Schaefer: Contributing to Discourse. Cognitive Science, pp. 259–294, 1989. find
  14. Cleland, A. and M. Pickering: The Use of Lexical and Syntactic Information in Language Production: Evidence from the Priming of Noun-phrase Structure. Journal of Memory and Language, 49:214–230, 2003. find
  15. Cole, J. and U. Reichel: Prosodic entrainment – the cognitive encoding of prosody and its relation to discourse function. Keynote at Framing speech satellite workshop of the Speech Prosody conference, Boston, 2016. find
  16. Danescu-Niculescu-Mizil, D., L. Lee, B. Pang and J. Kleinberg: Echoes of power: Language effects and power differences in social interaction. In Proc. 21st international conference on World Wide Web, pp. 699–708, Lyon, France, 2012. find
  17. De George, R.: The Nature and Limits of Authority. University Press of Kansas, Lawrence, 1985. find
  18. De Looze, C., S. Scherer, B. Vaughan and N. Campbell: Investigating automatic measurements of prosodic accommodation and its dynamics in social interaction. Speech Communication, 58:11–34, 2014. find
  19. Fox, J. and S. Weisberg: An R Companion to Applied Regression. Sage, Thousand Oaks CA, 2 ed., 2011. find
  20. Fuchs, S. and U. Reichel: On the relation between pointing gestures and speech production in German counting out rhymes: Evidence from motion capture data and speech acoustics. In Proc. P&P , pp. 51–54, Munich, Germany, 2016. find
  21. Giles, H. and N. Coupland: Language: Contexts and Consequences. Brooks/Cole, Pacific Grove, CA, 1991. find
  22. Giles, H., A. Mulac, J. Bradac and P. Johnson: Speech accommodation theory: The first decade and beyond . In McLaughlin, M. (ed.): Communication Yearbook , vol. 10, pp. 13–48. Sage, Newbury Park, CA, 1987. find
  23. Giles, H. and T. Ogay: Communication Accommodation Theory. In Whaley, B. and W. Samter (eds.): Explaining Communication: Contemporary Theories and Exemplars, pp. 293–310. Lawrence Erlbaum, Mahwah, NJ, 2007. find
  24. Gravano, A., v. Beňuš, R. Levitan and J. Hirschberg: Three ToBI-based measures of prosodic entrainment and their correlations with speaker engagement. In Proc. IEEE Spoken Language Technology Workshop, pp. 578–582, South Lake Tahoe, NV, 2014. find
  25. Gregory, S., K. Dagan and S. Webster: Evaluating the relation of vocal accommodation in conversation partners’ fundamental frequencies to perceptions of communication quality. J. Nonverbal Behavior, 21:23–43, 1997. find
  26. Gregory, S. and S. Webster: A nonverbal signal in voices of interview partners effectively predicts communication accommodation and social status perceptions. J. Pers. Soc. Psychol., 70:1231–1240, 1996. find
  27. Grice, H.: Logic and Conversation. In Cole, P. and J. Morgan (eds.): Speech acts, vol. 3 of Syntax and semantics, pp. 41–58. Academic Press, New York, 1975. find
  28. Gries, S.: Syntactic Priming: A Corpus-based Approach. Journal of Psycholinguistic Research, 2005. find
  29. Heinrich, C. and F. Schiel: The influence of alcoholic intoxication on the short-time energy function of speech. J. Acoust. Soc. Am., 135(5):2942–2951, 2014. find
  30. Hirschberg, J.: Speaking more like you: Entrainment in conversational speech.. In Proc. Interspeech, pp. 27–31, Florence, Italy, 2011. find
  31. Johnson, K.: Speech perception without speaker normalization: An exemplar model . In Johnson, K. and J. Mullenix (eds.): Talker Variability in Speech Processing, pp. 145–166. Academic Pres, San Diego, 1997. find
  32. Kisler, T., U. Reichel and F. Schiel: Multilingual processing of speech via web services. Computer, Speech, and Language, 45(C), 2017. find
  33. Kousidis, S., D. Dorran, C. McDonnell and E. Coyle: Times series analysis of acoustic feature convergence in human dialogues. In Proc. Interspeech, pp. 1692–1695, 2008. find
  34. Kraljic, T., S. Brennan and A. Samuel: Accommodating variation: Dialects, idiolects, and speech processing. Cognition, 107(1):54–81, 2008. find
  35. Ladd, R.: Intonational Phonology. Cambridge University Press, 2 ed., 2008. find
  36. Lee, C., M. Black, A. Katsamanis, A. Lammert, B. Baucom, A. Christensen, P. Georgiou and S. Narayanan: Quantification of Prosodic Entrainment in Affective Spontaneous Spoken Interactions of Married Couples. In Proc. Interspeech, pp. 793–796, Makuhari, Chiba, Japan, 2010. find
  37. Levitan, R., A. Gravano, L. Willson, Š. Beňuš, J. Hirschberg and A. Nenkova: Acoustic-prosodic entrainment and social behavior . In NAACL HLT ’12 Proc. of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 11–19, Montréal, Canada, 2012. find
  38. Levitan, R. and J. Hirschberg: Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions. In Proc. Interspeech, pp. 3081–3084, Florence, Italy, 2011. find
  39. Lewandowski, N.: Talent in nonnative phonetic convergence. PhD thesis, Institute for Natural Language Processing (IMS), University of Stuttgart, 2012. find
  40. Lewandowski, N. and D. Duran: Exemplar-theoretic modeling of phonetic convergence in dialogs. In Abtractbook Abstraction, Diversity, and Speech Dynamics, p. 35, Herrsching am Ammersee, 2017. find
  41. Mittelhammer, K. and U. Reichel: Characterization and prediction of dialogue acts using prosodic features. In Jokisch, O. (ed.): Elektronische Sprachverarbeitung 2016 , vol. 81 of Studientexte zur Sprachkommunikation, pp. 160–167. TUDpress, Dresden, Germany, 2016. find
  42. Mizukami, M., K. Yoshino, G. Neubig, D. Traum and S. Nakamura: Analyzing the Effect of Entrainment on Dialogue Acts. In Proc. of the SIGDIAL 2016 , pp. 310–318, Los Angeles, USA, 2016.
  43. Nenkova, A., A. Gravano and J. Hirschberg: High Frequency Word Entrainment in Spoken Dialogue. In Proc. of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies, pp. 169–172, 2008. find
  44. Nosofsky, R.: Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115:39–57, 1986. find
  45. Ohala, J.: The frequency code underlies the sound symbolic use of voice pitch. In Sound symbolism, pp. 325–347. Cambridge University Press, Cambridge, 1994. find
  46. Pardo, J.: On phonetic convergence during conversational interaction. J. Acoust. Soc. Am., 119:2382–2393, 2006. find
  47. Pardo, J.: Measuring phonetic convergence in speech production. Frontiers in Psychology, 4:Article 559, 2013. find
  48. Perez, J., R. Galvez and A. Gravano: Disentrainment may be a positive thing: A novel measure of unsigned acoustic-prosodic synchrony, and its relation to speaker engagement. In Proc. of Interspeech, pp. 1270–1274, San Francisco, 2016. find
  49. Pfitzinger, H., S. Burger and S. Heid: Syllable Detection in Read and Spontaneous Speech. In Proc. ICSLP , vol. 2, pp. 1261–1264, Philadelphia, 1996. find
  50. Pickering, M. and S. Garrod: Toward a Mechanistic Psychology of Dialogue. Behavioral and Brain Sciences, 27:169–225, 2004. find
  51. Pickering, M. J. and S. Garrod: An integrated theory of language production and comprehension. Behavioral and Brain Sciences, 36(4):329–347, 2013. find
  52. Pierrehumbert, J. and J. Hirschberg: The Meaning of Intonational Contours in the Interpretation of Discourse. In Cohen, P., J. Morgan and M. Pollack (eds.): Intentions in Communication, pp. 271–311. MIT Press, Cambridge, 1990. find
  53. Putnam, W. and R. Street: The conception and perception of noncontent speech performance: Implications for speech accommodation theory. Language, 46:97–114, 1984. find
  54. Reichel, U.: PermA and Balloon: Tools for string alignment and text processing. In Proc. Interspeech 2012 , p. paper no. 346, Portland, Oregon, 2012. find
  55. Reichel, U.: Linking bottom-up intonation stylization to discourse structure. Computer, Speech, and Language, 28:1340–1365, 2014. find
  56. Reichel, U.: CoPaSul Manual – Contour-based parametric and superpositional intonation stylization. RIL, MTA, Budapest, Hungary, 2016. https://arxiv.org/abs/1612.04765. ↗
  57. Reichel, U.: Unsupervised extraction of prosodic structure. In Trouvain, J., I. Steiner and B. Möbius (eds.): Elektronische Sprachverarbeitung 2017 , vol. 86 of Studientexte zur Sprachkommunikation, pp. 262–269. TUDpress, Dresden, Germany, 2017. find
  58. Reichel, U.: CoPaSul software. GitHub Repository, 2018. https://github.com/reichelu/copasul. ↗
  59. Reichel, U. and J. Cole: Entrainment analysis of categorical intonation representations. In Proc. P&P , pp. 165–168, Munich, Germany, 2016. find
  60. Reichel, U. and K. Mády: Comparing parameterizations of pitch register and its discontinuities at prosodic boundaries for Hungarian. In Proc. Interspeech 2014 , pp. 111–115, Singapore, 2014. find
  61. Reichel, U., N. Pörner, D. Nowack and J. Cole: Analysis and classification of cooperative and competitive dialogs. In Proc. Interspeech, p. paper 3056, Dresden, Germany, 2015. find
  62. Reitter, D. and J. Moore: Alignment and task success in spoken dialogue. Journal of Memory and Language, 76:29–46, 2014. find
  63. Rietveld, T. and P. Vermillion: Cues for Perceived Pitch Register . Phonetica, 60:261–272, 2003. find
  64. Savitzky, A. and M. Golay: Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Analytical Chemistry, 36(8):1627–1639, 1964. find
  65. Schegloff, E.: Sequence Organization in Interaction. Cambridge University Press, Cambridge, 2006. find
  66. Schweitzer, A. and N. Lewandowski: Social Factors in Convergence of F1 and F2 in Spontaneous Speech. In Proc. 10th International Seminar on Speech Production, pp. 391–394, Cologne, 2014. find
  67. Shriberg, E., R. Bates, A. Stolcke, P. Taylor, D. Jurafsky, K. Ries, N. Coccaro, R. Martin, M. Meteer and C. V. Ess-Dykema: Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech? . Language and Speech 41(3-4), pp. 439–487, 1998. find
  68. Sperber, D. and D. Wilson: Relevance: Communication and Cognition. Blackwell, Oxford, 1986. find
  69. Sperber, D. and D. Wilson: Relevance Theory. In Ward, G. and L. Horn (eds.): Handbook of Pragmatics, pp. 607–632. Blackwell, Oxford, 2004. find
  70. Street, R.: Speech convergence and social evaluation in fact-finding interviews. Human Communication Research, pp. 139–169, 1984. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/reichel-2018-prosodic-entrainment-dialog-acts/