Elamur Research⌕
☰
home / papers / 2026 / Gaze Behaviour & Conversation Unfolding in the HCRC Map Task Corpus

Gaze Behaviour & Conversation Unfolding in the HCRC Map Task Corpus

Summary

The paper examines how eye-gaze relates to the way a dialogue progresses linguistically, an area where the literature on gaze at turn boundaries offers no consistent pattern. Using the HCRC Map Task Corpus, the authors analyse the 31 dialogues where Giver and Follower could see each other. They look at each participant's gaze (down at the map, up at the partner, or off elsewhere) at the beginning and end of turns. To align gaze and turn tiers on a shared time axis, they apply the event-based Beginning-Middle-End (BME) annotation method, which supports counting co-occurring events in contingency tables. Gaze is then tested with χ²-tests against three turn properties: grouped dialogue acts, lexical entropy change, and turn-to-turn repetition. Speakers who look at their partner at the end of a turn tend to be expressing complexity or trouble. Gaze kept on the map is more typical of smooth, disagreement-free stretches. Addressees who look up or away at turn onset show signs of adapting their vocabulary. The work offers a reusable cross-modal alignment approach and links gaze to cognitive effort in task dialogue.

Key points

  • Data: HCRC Map Task Corpus (128 dialogues, 64 speakers, mostly from Glasgow). Of the 46 gaze-annotated dialogues, 31 allowed eye contact. After anomalous gaze annotations were removed, these yielded 10,761 paired gazes and 5,889 turns (originally 10,894 and 5,943).
  • Alignment: the BME (Beginning-Middle-End) method orders the onsets and offsets of annotations across tiers. Duration is measured by counting intervening events ("Middles") as well as by absolute time.
  • Dialogue acts were merged into five supra-categories (Instruction, Confirmation, Explanation, Doubt, Rejection) so the χ²-tests would be valid.
  • Lexical measures:
    • transcripts were normalised (contractions split, text lowercased);
    • tokens were tagged with the spaCy small English model and split into open-class (NOUN, VERB, ADJ, PROPN) and closed-class words;
    • entropy change at turn boundaries was classed as increase, decrease, or stable, alongside turn-to-turn repetition.
  • Main findings:
    • end-of-turn gaze at the partner aligns with difficulty and complexity;
    • gaze held down at the map marks obstacle-free exchanges;
    • listeners' gaze up or off at turn start relates to lexical adaptation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/33muut9zdprh

@inproceedings{murat2026gaze,
  title = {{Gaze Behaviour \& Conversation Unfolding in the HCRC Map Task Corpus}},
  author = {Murat, Anaïs Claire and Vogel, Carl},
  booktitle = {Proceedings of the 22nd Joint ACL - ISO Workshop on Interoperable Semantic Annotation and Representation (ISA-22) @ LREC 2026},
  year = {2026},
  doi = {10.63317/33muut9zdprh},
  url = {https://doi.org/10.63317/33muut9zdprh}
}

References

51 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne H Anderson, Ellen Gurman Bard, Cathy Sotillo, Alison Newlands, and Gwyneth Doherty-Sneddon. 1997. Limited visual control of the intelligibility of speech in face-to-face dialogue. Perception & Psychophysics, 59(4):580–592. find
  2. Jens Edlund, Simon Alexandersson, Jonas Beskow, Lisa Gustavsson, Mattias Heldner, Anna Hjalmarsson, Petter Kallionen, and Ellen Marklund. 2012. 3rd party observer gaze as a continuous measure of dialogue flow. In Proceedings of the Eight International Conference on Language Resources and Evaluation… find
  3. Michael Argyle and Mark Cook. 1976. Gaze and mutual gaze. Cambridge Univ. Press, Cambridge. find
  4. Simon Garrod and Martin J. Pickering. 2007. Alignment in dialogue. In M. Gareth Gaskell, editor, The Oxford Handbook of Psycholinguistics, pages 442–452. Oxford University Press. find
  5. Janet Beavin Bavelas, Linda Coates, and Trudy Johnson. 2002. Listener Responses as a Collaborative Process: The Role of Gaze. Journal of Communication, 52(3):566–580. find
  6. Dmitriy Genzel and Eugene Charniak. 2002. Entropy rate constancy in text. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 199–206. find
  7. Geoffrey W. Beattie. 1978. Floor apportionment and gaze in conversational dyads. British Journal of Social and Clinical Psychology, 17(1):7–15. find
  8. Milan Gnjatović, Jovica Tasevski, Branislav Borovac, and Nemanja Maček. 2018. An Entropy-Based Approach to Automatic Detection of Critical Changes in Human-Machine Interaction. In 2018 9th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 000175–000178. ISSN: 2380-735… find
  9. Elizabeth A. Boyle, Anne H. Anderson, and Alison Newlands. 1994. The Effects of Visibility on Dialogue and Performance in a Cooperative Problem Solving Task. Language and Speech, 37(1):1–20. find
  10. Susan E. Brennan, Xin Chen, Christopher A. Dickinson, Mark B. Neider, and Gregory J. Zelinsky. 2008. Coordinating cognition: the costs and benefits of shared gaze during collaborative search. Cognition, 106(3):1465–1477. find
  11. Geert Brône, Bert Oben, Annelies Jehoul, Jelena Vranjes, and Kurt Feyaerts. 2017. Eye gaze and viewpoint in multimodal interaction management. Cognitive Linguistics, 28(3):449–483. Publisher: De Gruyter Mouton. find
  12. Jean Carletta, Amy Isard, Stephen Isard, Jacqueline Kowtko, Alison Newlands, Gwyneth Doherty-Sneddon, and Anne Anderson. 1996. HCRC dialogue structure coding manual. Technical report, University of Edinburgh. HCRC Technical Report 82. find
  13. Mark S. Cary. 1978. The Role of Gaze in the Initiation of Conversation. Social Psychology, 41(3):269. find
  14. Ziedune Degutyte and Arlene Astell. 2021. The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings. Frontiers in Psychology, 12. find
  15. Gwyneth Doherty-Sneddon. 1995. The development of conversational and communication skills. Ph.D. thesis, University of Glasgow. find
  16. Charles Goodwin. 1980. Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning. Sociological Inquiry, 50(3-4):272– 302. find
  17. Joy E. Hanna, Susan E. Brennan, and Kelly J. Savi-etta. 2020. Eye Gaze and Head Orientation Cues in Face-to-Face Referential Communication. Discourse Processes, 57(3):201–223. _eprint: https://doi.org/10.1080/0163853X.2019.1675467. — [doi:10.1080/0163853x.2019.1675467](https://doi.org/10.1080/0163853x.2019.1675467) ↗
  18. Patrick G. T. Healey, Matthew Purver, and Christine Howes. 2014. Divergence in dialogue. PloS one, 9(6):e98598. Https://doi.org/10.1371/journal.pone.0098598. — [doi:10.1371/journal.pone.0098598](https://doi.org/10.1371/journal.pone.0098598) find
  19. Simon Ho, Tom Foulsham, and Alan Kingstone. 2015. Speaking and Listening with the Eyes: Gaze Signaling during Dyadic Interactions. PLOS ONE, 10(8):e0136905. Publisher: Public Library of Science. find
  20. Laszlo Hunyadi, Tamás Váradi, Gy Kovács, István Szekrényes, Hermina Kiss, and Karolina Takács. 2018. Human-human, human-machine communication: on the HuComTech multimodal corpus. Selected papers from the CLARIN Annual Conference 2018. Linköping Electronic Conference Proceedings, 159:56–65. find
  21. Kristiina Jokinen, Kazuaki Harada, Masafumi Nishida, and Seiichi Yamamoto. 2010. Turn-Alignment Using Eye-Gaze and Speech in Conversational Interaction. Interspeech, pages 2018– 2021. find
  22. Frank Keller. 2004. The entropy rate principle as a predictor of processing effort: An evaluation against eye-tracking data. In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 317–324. find
  23. Adam Kendon. 1967. Some functions of gaze-direction in social interaction. Acta Psychologica, 26:22–63. find
  24. Adam Kendon. 1978. Looking in conversation and the regulation of turns at talk: A comment on the papers of G Beattie and D. R. Rutter et al . British Journal of Social and Clinical Psychology, 17(1):23–24. find
  25. Kobin H. Kendrick and Judith Holler. 2017. Gaze Direction Signals Response Preference in Conversation. Research on Language and Social Interaction, 50(1):12–32. find
  26. Maria Koutsombogera and Carl Vogel. 2019. Observing Collaboration in Small-Group Interaction. Multimodal Technologies and Interaction, 3(3):45. find
  27. Magnus S. Magnusson. 2000. Discovering hidden time patterns in behavior: T-patterns and their detection. Behavior Research Methods, Instruments, & Computers, 32(1):93–110. find
  28. Magnus S. Magnusson. 2020. T-Pattern Detection and Analysis (TPA) With THEMETM: A Mixed Methods Approach. Frontiers in Psychology, 10. find
  29. Alexander Mehler, Andy Lücking, and Peter Menke. 2011. Assessing Lexical Alignment in Spontaneous Direction Dialogue Data by Means of a Lexicon Network Model. In Alexander F. Gelbukh, editor, Computational Linguistics and Intelligent Text Processing, volume 6608, pages 368–379. Springer Berlin Heide… find
  30. Gregor Mehlmann, Markus Häring, Kathrin Janowski, Tobias Baur, Patrick Gebhard, and Elisabeth André. 2014. Exploring a Model of Gaze for Grounding in Multimodal HRI. In Proceedings of the 16th International Conference on Multimodal Interaction, pages 247–254, Istanbul Turkey. ACM. find
  31. Anaïs Claire Murat, Maria Koutsombogera, and Carl Vogel. 2026. Event chronography in multimodal data: the BME method for quantitative analyses. In Proceedings of the 2026 Language Resources and Evaluation Conference. To appear. find
  32. Anaïs Claire Murat, Maria Koutsombogera, and Carl Vogel. 2022. Mutual Gaze and Linguistic Repetition in a Multimodal Corpus. In Proceedings of the Language Resources and Evaluation Conference, pages 2771–2780, Marseille, France. European Language Resources Association (ELRA). find
  33. Anaïs Claire Murat and Carl Vogel. 2025. Online meetings: When repetition signals engagement. In 2025 IEEE 16th International Conference on Cognitive Infocommunications (CogInfoCom), pages 37–42, Vienna, Austria. IEEE. find
  34. Mark B. Neider, Xin Chen, Christopher A. Dickinson, Susan E. Brennan, and Gregory J. Zelinsky. 2010. Coordinating spatial referencing using shared gaze. Psychonomic Bulletin & Review, 17(5):718–724. find
  35. Hannele Nicholson, Ellen Gurman Bard, Robin Lickley, Anne H. Anderson, and Catriona Havard (3) Yiya Chen. 2005. Disfluency and behaviour in dialogue: evidence from eye-gaze. In Disfluency in Spontaneous Speech (DiSS 2005), pages 133–138. find
  36. Catharine Oertel, Marcin Włodarczak, Jens Edlund, Petra Wagner, and Joakim Gustafson. 2012. Gaze patterns in turn-taking. In 13th Annual Conference of the International Speech Communication Association 2012 (INTERSPEECH 2012), pages 2246–2249. find
  37. Ulrich J. Pfeiffer, Kai Vogeley, and Leonhard Schilbach. 2013. From gaze cueing to dual eye-tracking: Novel approaches to investigate the neural correlates of gaze in social interaction. Neuroscience & Biobehavioral Reviews, 37(10):2516–2528. find
  38. Martin J. Pickering and Simon Garrod. 2004. Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27(02). find
  39. Dalila Mekhaldi. 2006. A study on multimodal document alignment: bridging the gap between textual documents and spoken language. Ph.D. thesis, Faculty of Science, University of Fribourg (Switzerland). find
  40. David Reitter and Johanna D Moore. 2007. Predicting success in dialogue. In Proceedings of the 45th Annual Meeting of the Association of Comp utational Linguistics, pages 808–815. Association for Computational Linguistics. find
  41. Ines Montani, Matthew Honnibal, Adriane Boyd, Sofie Van Landeghem, and Henning Peters. 2023. explosion/spaCy: v3.7.2: Fixes for APIs and requirements. find
  42. Justine Reverdy, Maria Koutsombogera, and Carl Vogel. 2020. Linguistic Repetition in Three-Party Conversations. In Anna Esposito, Marcos Faundez-Zanuy, Francesco Carlo Morabito, and Eros Pasero, editors, Neural Approaches to Dynamics of Signal Exchanges, volume 151, pages 359–370. Springer Singapore… find
  43. Justine Reverdy and Carl Vogel. 2017. Measuring Synchrony in Task-Based Dialogues. In Interspeech 2017, pages 1701–1705, Stockholm, Sweden. ISCA. find
  44. Federico Rossano. 2010. Questioning and responding in Italian. Journal of Pragmatics, 42(10):2756–2771. find
  45. Simona Sbranna, Michelina Savino, Florence Baills, and Martine Grice. 2025. Gaze behaviour and vocal feedback in task-based dyadic conversations with and without eye contact. Frontiers in Communication, 10. Publisher: Frontiers. find
  46. Malin Spaniol, Alicia Janz, Simon Wehrle, Kai Vogeley, and Martine Grice. 2023. Multimodal signalling: The interplay of oral and visual feedback in conversation. In Proceedings of the 20th International Congress of Phonetic Sciences. ICPhS. find
  47. Jürgen Streeck. 2014. Mutual gaze and recognition: Revisiting Kendon’s “Gaze direction in two-person conversation”. In Mandana Seyfeddinipur and Marianne Gullberg, editors, From Gesture in Conversation to Visible Action as Utterance: Essays in honor of Adam Kendon, pages 35–56. John Benjamins Publis… find
  48. Yu Wang, Yang Xu, Gabriel Skantze, and Hendrik Buschmeier. 2024. How much does non-verbal communication conform to entropy rate constancy?: A case study on listener gaze in interaction. In Findings of the Association for Computational Linguistics: ACL 2024, pages 3533– 3545. find
  49. Peter Wittenburg, Hennie Brugman, Albert Russel, Alex Klassmann, and Han Sloetjes. 2006. ELAN: a professional framework for multimodality research. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), Genoa, Italy. European Language Resources Associati… find our card
  50. Elisabeth Zima, Clarissa Weiß, and Geert Brône. 2019. Gaze and overlap resolution in triadic interactions. Journal of Pragmatics, 140:49–69. 10. find
  51. Language Resource References Thompson, Henry S. and Anderson, Anne and Bard, Ellen Gurman and Doherty-Sneddon, Gwyneth and Newlands, Alison and Sotillo, Cathy. 1993. The HCRC Map Task corpus: natural dialogue for speech recognition. Association for Computational Linguistics, ISLRN 777-455-577-608-1. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/murat-2026-gaze-behaviour-conversation-unfolding-hcrc/