Elamur Research⌕
☰
home / papers / 2026 / Who Plays Which Role When? Communication Role Dynamics for Peer Recognition and Team Performance Prediction

Who Plays Which Role When? Communication Role Dynamics for Peer Recognition and Team Performance Prediction

Paper · 2026 · Yifan Song, Wenxuan Wendy Shi, Brian Bailey et al.

Summary

Computational studies of team roles often rely on ad hoc personas or data-driven clusters, and rarely follow real teams over long periods. This paper adapts a role taxonomy from education research (Nestsiarovich and Pons, 2020) to Slack chat from a semester-long user interface design project in a computer science course. Two expert annotators assigned eight constructive roles to each student in each deliverable window, treating the task as multi-label. The authors then tested whether LLMs can reproduce these expert labels so that annotation can scale. Using the role labels, they traced how roles change over the project. Different roles dominate different phases, and individuals take on a wider mix of roles as the work grows more complex. Role-based features predicted peer recognition better than lexical, conversational and zero-shot LLM baselines. On the public DeliData deliberation corpus, the same roles helped exceed previously reported results for predicting team performance gain. The work offers interpretable, theory-based signals for analysing and supporting collaboration.

Key points

  • Corpus: 6,307 Slack messages (93,474 words) from 55 consenting students in 18 teams, covering eight deliverables (D1–D8) and grouped into 424 student-deliverable instances. Raw messages are not released for privacy reasons.
  • Roles: The original taxonomy has ten roles. The eight retained are connector, gatekeeper, representative, arbitrator, facilitator, explorer, initiator and information provider. Passive collector and outsider are excluded because low activity on Slack cannot reliably reveal them.
  • Role dynamics: Connector rises during user research (D3). Explorer, information provider and facilitator peak during the first functional prototype (D6). Gatekeeper is highest in the final evaluation phase (D8).
  • Peer recognition: Role features improve prediction by about 5–10% over the baselines, and the best LLM annotator agrees well with the expert labels.
  • DeliData: Combining team-context roles with conversational statistics scores 0.74, compared with 0.70 for the best prior feature combination and 0.68 for a zero-shot LLM.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States184 works · 69,372 citations

Also on the byline: California State Polytechnic University.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 worksSocial Psychology · 306 worksComputer Science Applications · 102 works

Cite

Cite as Yifan Song, Wenxuan Wendy Shi, Brian Bailey and Tal August. 2026. Who Plays Which Role When? Communication Role Dynamics for Peer Recognition and Team Performance Prediction. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1930

@inproceedings{song2026who,
  title = {{Who Plays Which Role When? Communication Role Dynamics for Peer Recognition and Team Performance Prediction}},
  author = {Yifan Song, Wenxuan Wendy Shi, Brian Bailey and Tal August},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1930},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1930}
}

References

35 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kelsey Allen, Giuseppe Carenini, and Raymond Ng. 2014. Detecting disagreement in conversations using pseudo-monologic rhetorical structure. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1169– 1180, Doha, Qatar. Association for Computational…
  2. A. Aranzabal, E. Epelde, and M. Artetxe. 2022. Team formation on the basis of belbin’s roles to enhance students’ performance in project based learning. Education for Chemical Engineers, 38:22–37. find
  3. Tal August, Lauren Kim, Katharina Reinecke, and Noah A. Smith. 2020. Writing strategies for science communication: Data and computational analysis. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5327–5344, Online. Association for Computationa… find
  4. Satanjeev Banerjee and Alexander Rudnicky. 2006. You are what you say: Using meeting participants’ speech to detect their roles and expertise. In Proceedings of the Analyzing Conversations in Text and Speech, pages 23–30, New York City, New York. Association for Computational Linguistics. find
  5. Kenneth D. Benne and Paul Sheats. 1948. Functional roles of group members. Journal of Social Issues, 4(2):41–49. Place: United Kingdom Publisher: Blackwell Publishing. find
  6. Hancheng Cao, Vivian Yang, Victor Chen, Yu Jin Lee, Lydia Stone, N’godjigui Junior Diarrassouba, Mark E. Whiting, and Michael S. Bernstein. 2021. My team will go on: Differentiating high and low viability teams through team interaction. Proc. ACM Hum.-Comput. Interact., 4(CSCW3). find
  7. Wen Dong, Bruno Lepri, Fabio Pianesi, and Alex Pentland. 2013. Modeling functional roles dynamics in small group interactions. IEEE Transactions on Multimedia, 15(1):83–95. find
  8. Raji Ghawi, Siegfried Müller, and Jürgen Pfeffer. 2021. Improving team performance prediction in mmogs with temporal communication networks. Social Network Analysis and Mining, 11. find
  9. Shanyun He, Xinyue Shi, Tae-Hee Choi, and Junqing Zhai. 2023. How do students’roles in collaborative learning affect collaborative problem-solving competency? a systematic review of research. Thinking Skills and Creativity, 50:101423. find
  10. Christopher Hundhausen, Phill Conrad, Olusola Adesope, Ahsun Tariq, Samir Sbai, and Andrew Lu. 2023. Investigating reflection in undergraduate software development teams: An analysis of online chat transcripts. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, SI… find
  11. Farnaz Jahanbakhsh, Wai-Tat Fu, Karrie Karahalios, Darko Marinov, and Brian Bailey. 2017. You want me to work with who? stakeholder perceptions of automated team formation in project-based courses. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, page 3201–32… find
  12. Chenyan Jia, Michelle S. Lam, Minh Chau Mai, Jeffrey T. Hancock, and Michael S. Bernstein. 2024. Embedding democratic values into social media ais via societal objective functions. Proc. ACM Hum.-Comput. Interact., 8(CSCW1). find
  13. Georgi Karadzhov, Tom Stafford, and Andreas Vlachos. 2023. Delidata: A dataset for deliberation in multi-party problem solving. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW2):1–25. find
  14. Klaus Krippendorff. 2004. Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30(3):411– 433. find
  15. Yihuai Lan, Zhiqiang Hu, Lei Wang, Yang Wang, Deheng Ye, Peilin Zhao, Ee-Peng Lim, Hui Xiong, and Hao Wang. 2024. LLM-based agent society investigation: Collaboration and confrontation in avalon gameplay. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, page… find
  16. Haoran Li, Ziyi Su, Yun Xue, Zhiliang Tian, Yiping Song, and Minlie Huang. 2025a. Advancing collaborative debates with role differentiation through multi-agent reinforcement learning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),… find
  17. Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab, and Maarten Sap. 2025b. BIG5-CHAT: Shaping LLM personalities through training on human-grounded data. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 20434– 20471, Vie…
  18. Diane Litman, Susannah Paletz, Zahra Rahimi, Stefani Allegretti, and Caitlin Rice. 2016. The teams corpus and entrainment in multi-party spoken dialogues. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1421–1431, Austin, Texas. Association for Comput… find
  19. Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. 2024. Large language models are superpositions of all characters: Attaining arbitrary role-play via self-alignment. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7828–7840,…
  20. Keith Maki, Michael Yoder, Yohan Jo, and Carolyn Rosé. 2017. Roles and success in Wikipedia talk pages: Identifying latent patterns of behavior. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1026–1035, Taipei, Taiwan. Asian… find
  21. John Mathieu, John Hollenbeck, Daan Knippenberg, and Daniel Ilgen. 2017. A century of work teams in the journal of applied psychology. Journal of Applied Psychology, 102:452–467. find
  22. John E. Mathieu, Scott I. Tannenbaum, Michael R. Kukenberger, Jamie S. Donsbach, and George M. Alliger. 2015. Team role experience and orientation: A measure and tests of construct validity. Group & Organization Management, 40(1):6–34. find
  23. R. Meredith Belbin. 2011. Management Teams: Why They Succeed or Fail (3rd ed.). Human Resource Management International Digest, 19(3). find
  24. Kristina Nestsiarovich and Dirk J. Pons. 2020. Team role adoption and distribution in engineering project meetings. Behavioral Sciences, 10. find
  25. Thien Hai Nguyen, Kiyoaki Shirai, and Julien Velcin. 2016. Sentiment analysis on social media for stock movement prediction. Expert Syst. Appl., 42(24):9603–9611. find
  26. Vlad Niculae and Cristian Danescu-Niculescu-Mizil. 2016. Conversational markers of constructive discussions. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568–578, San Diego, California. Associ… find
  27. Mazni Omar, Sharifah-Lailee Syed-Abdullah, and Naimah Mohd Hussin. 2011. Developing a team performance prediction model: A rough sets approach. In Informatics Engineering and Information Science, pages 691–705, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  28. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computat…
  29. David Reitter and Johanna D. Moore. 2007. Predicting Success in Dialogue. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 808–815, Prague, Czech Republic. Association for Computational Linguistics. find
  30. Antonette Shibani, Elizabeth Koh, Vivian Lai, and Kyong Jin Shim. 2017. Assessing the language of chat for teamwork dialogue. Journal of Educational Technology Society, 20(2):224–237. find
  31. Yifan Song, Ritika Vithani, Wenxuan Wendy Shi, and Brian P. Bailey. 2026. From data to action: Empowering students to assess and improve teamwork with cross-tool log data. In Proceedings of the 57th ACM Technical Symposium on Computer Science Education V.1, page 992–998, New York, NY, USA. Associati… find
  32. P. C. Wason. 1968. Reasoning about a rule. Quarterly Journal of Experimental Psychology, 20(3):273–281. find
  33. Anna Wegmann, Tijs A. Van Den Broek, and Dong Nguyen. 2024. What’s mine becomes yours: Defining, annotating and detecting context-dependent paraphrases in news interview dialogs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 882–912, Miami, Florida,… find
  34. Diyi Yang, Miaomiao Wen, and Carolyn Rosé. 2015. Weakly supervised role identification in teamwork interactions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers… find
  35. Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. Conversations Gone Awry: Detecting Early Signs of Conversational Failure. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (V…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-who-plays-which-role-when/