Elamur Research⌕
☰
home / papers / 2020 / The Teacher-Student Chatroom Corpus

The Teacher-Student Chatroom Corpus

Paper · 2020 · Andrew Caines, Helen Yannakoudakis, Helena Edmondson et al.

Summary

Data for building language-teaching dialogue systems is scarce. Existing conversation corpora either lack real classroom roles or, like CIMA, use crowdworkers who only pretend to be tutors. To fill this gap, Caines et al. built the Teacher-Student Chatroom Corpus (TSCC). It contains genuine one-hour, one-to-one English lessons held in a text chatroom. To protect participants' privacy, the authors avoided commercial messengers and hosted lessons on their own minimal chat app, written in Shiny for R. Two qualified teachers taught eight learners and were free to plan each lesson as they saw fit. An experienced teacher and examiner then annotated every transcript. The annotation covers anonymisation, minimal grammatical corrections of student turns, CEFR proficiency labels and discourse or teaching sequence tags. Early analyses show that learner errors in chat look more like errors in written exam essays than in speech. They also show that lesson phases follow recognisable patterns. The resource supports discourse research and could eventually inform chatbots that assist human teachers rather than replace them.

Key points

  • TSCC has 102 lessons, 2 teachers and 8 students, with about 13.5K turns and 133K word tokens. It is free for research under CC BY-NC-SA 4.0.
  • Learners were 12–40 years old (mean 23) and attended 2–32 lessons each (mean 12), so longitudinal study is possible. Among the learners' native languages, Japanese and Ukrainian each account for two speakers, with Thai, Spanish, Italian and Mandarin Chinese also represented.
  • Corrected student turns were typed with ERRANT and compared with the FCE essay corpus and the CROWDED speech corpus. TSCC patterns resemble FCE: mostly 'replacement' edits, few 'unnecessary' ones, and unusually few punctuation errors.
  • Topic management mostly happens early in a lesson, while eliciting and scaffolding cluster mid-lesson. C1 learners get fewer exercises and elicitations than B1/B2 learners but make more enquiries.
  • Planned extensions include multiple correction annotations, more participants, and deeper analysis of how feedback relates to student turns.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom287 works · 125,174 citations🎓King's College LondonLondon · United Kingdom50 works · 1,852 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksPsychology · 1,673 worksDevelopmental and Educational Psychology · 533 works

Cite

Cite as Andrew Caines, Helen Yannakoudakis, Helena Edmondson, Helen Allen, Pascual Pérez-Paredes, Bill Byrne and Paula Buttery. 2020. The Teacher-Student Chatroom Corpus. Linköping Electronic Conference Proceedings 175.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3384/ecp2017510 · arXiv 2011.07109

@inproceedings{caines2020teacher,
  title = {{The Teacher-Student Chatroom Corpus}},
  author = {Andrew Caines, Helen Yannakoudakis, Helena Edmondson, Helen Allen, Pascual Pérez-Paredes, Bill Byrne and Paula Buttery},
  booktitle = {Linköping Electronic Conference Proceedings 175},
  year = {2020},
  doi = {10.3384/ecp2017510},
  eprint = {2011.07109},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3384/ecp2017510}
}

References

30 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sherrie Atwood, William Turnbull, and Jeremy I. M. Carpendale. 2010. The construction of knowledge in classroom talk. Journal of the Learning Sciences, 19(3):358–402. find
  2. Douglas Biber, Susan Conrad, Randi Reppen, Pat Byrd, Marie Helt, Victoria Clark, Viviana Cortes, Eniko Csomay, and Alfredo Urzua. 2004. Representing language use in the university: Analysis of the TOEFL 2000 spoken and written academic language corpus. Princeton, NJ: Educational Testing Service. find
  3. BNC Consortium. 2001. The British National Corpus, version 2 (BNC World). find
  4. Christopher Bryant, Mariano Felice, Øistein Andersen, and Ted Briscoe. 2019. The BEA-2019 Shared Task on Grammatical Error Correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications (BEA).
  5. Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
  6. Christopher Bryant and Hwee Tou Ng. 2015. How far are we from fully automatic high quality grammatical error correction? In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Lon… find
  7. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Langua…
  8. Andrew Caines, Christian Bentz, Dimitrios Alikaniotis, Fridah Katushemererwe, and Paula Buttery. 2016. The glottolog data explorer: Mapping the world’s languages. In Proceedings of the LREC 2016 Workshop ‘VisLR II: Visualization as Added Value in the Development, Use and Evaluation of Language Resou… find
  9. Andrew Caines and Paula Buttery. 2017. The effect of task and topic on language use in learner corpora. In Lynne Flowerdew and Vaclav Brezina, editors, Written and spoken learner corpora and their use in different contexts. London: Bloomsbury. find
  10. Ronald Carter and Michael McCarthy. 1997. Exploring Spoken English. Cambridge: Cambridge University Press. find
  11. Winston Chang, Joe Cheng, JJ Allaire, Yihui Xie, and Jonathan McPherson. 2020. shiny: Web Application Framework for R. R package version 1.4.0.2. find
  12. Eniko Csomay. 2012. Lexical Bundles in Discourse Structure: A Corpus-Based Study of Classroom Discourse. Applied Linguistics, 34(3):369–388. find
  13. Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011. Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs. In Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics.
  14. Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani-Tur. 2020. MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines. In Proceedings of The 12th Language Res… find
  15. Jane Evison. 2013. Turn openings in academic talk: where goals and roles intersect. Classroom Discourse, 4(1):3–26. find
  16. Lynda Griffin and James Roy. 2019. A great resource that should be utilised more, but also a place of anxiety: student perspectives on using an online discussion forum. Open Learning: The Journal of Open, Distance and e-Learning, pages 1–16. find
  17. Sviatlana Höhn. 2017. A data-driven model of explanations for a chatbot that helps to practice conversation in a foreign language. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue. find
  18. Ian Hutchby and Robin Wooffitt. 1988. Conversation analysis. Cambridge: Polity Press. find
  19. Robbie Love, Claire Dembry, Andrew Hardie, Vaclav Brezina, and Tony McEnery. 2017. The Spoken BNC2014: designing and building a spoken corpus of everyday conversations. International Journal of Corpus Linguistics, 22:319–344. find
  20. Anne O’Keeffe and Steve Walsh. 2012. Applying corpus linguistics and conversation analysis in the investigation of small group teaching in higher education. Corpus Linguistics and Linguistic Theory, 8:159–181. find
  21. Alan Ritter, Colin Cherry, and Bill Dolan. 2010. Unsupervised modeling of Twitter conversations. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
  22. Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, and Jason Weston. 2020. Recipes for building an open-domain chatbot. arXiv, 2004.13637.
  23. Carolyn P. Rosé, Diane Litman, Dumisizwe Bhembe, Kate Forbes, Scott Silliman, Ramesh Srivastava, and Kurt VanLehn. 2003. A comparison of tutor and student behavior in speech versus text based tutoring. In Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural La… find
  24. Harvey Sacks, Emanuel Schegloff, and Gail Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language, 50:696– 735. find
  25. Nicolas Schrading, Cecilia Ovesdotter Alm, Ray Ptucha, and Christopher Homan. 2015. An analysis of domestic abuse discourse on Reddit. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. find
  26. Paul Seedhouse. 2004. Conversation analysis methodology. Language Learning, 54:1–54. find
  27. Rita Simpson, Susan Briggs, John Ovens, and John Swales. 2002. Michigan Corpus of Academic Spoken English. Ann Arbor: University of Michigan. find
  28. Katherine Stasaski, Kimberly Kao, and Marti A. Hearst. 2020. CIMA: A large open access dialogue dataset for tutoring. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications.
  29. Etsuko Toyoda and Richard Harrison. 2002. Categorization of text chat communication between learners and native speakers of Japanese. Language Learning & Technology, 6:82–99. find
  30. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/caines-2020-teacher-student-chatroom-corpus/