Elamur Research⌕
☰
home / papers / 2024 / Speak & Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback

Speak & Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback

Paper · 2024 · Kate Knill, Diane Nicholls, Mark J.F. Gales et al.

Summary

Research on automated speaking assessment and feedback for learners of English has been held back by a shortage of public, well-annotated L2 speech. This paper describes the Speak & Improve Corpus 2025, which was built to fill that gap. Its recordings come from open, spontaneous answers to practice tests on the free Speak & Improve web platform. The corpus has about 315 hours of learner audio, each test part carrying a holistic proficiency score. Around 55 hours also come with detailed manual transcriptions that keep disfluencies, mark phrase boundaries, and include grammatically corrected versions. Speakers span many first languages and range from CEFR A2 to C1. Annotation followed a staged process: quality and proficiency scoring, then transcription produced by correcting ASR output, then error correction. The authors present it as the broadest public L2 English learner corpus to date and the first to pair audio with grammatical error corrections. This makes it usable for spoken language assessment, spoken GEC and feedback, ASR for learner speech, and disfluency detection. It underpins the Speak & Improve Challenge 2025.

Key points

  • Size: about 315 hours of scored L2 English audio, with roughly 55 hours manually transcribed and error-corrected. The data is split into Train, Dev and Eval, and release is non-commercial via the ELiT website.
  • Source: Speak & Improve version 1 had about 1.7M users between December 2018 and September 2024, who submitted more than 18.4M answers. The test follows the Linguaskill Speaking format.
  • The test has five parts: Interview, Read Aloud, an opinion Long Turn, a presentation describing a graphic, and a Communication Activity. Read-aloud answers are excluded, as are the first two interview questions because they risk exposing personal information.
  • Phase 1 scoring works as follows:
    • Each utterance gets an audio quality score from 3 to 5; anything below 3 is dropped.
    • Each part gets a holistic score from 1 to 6.
    • Only parts judged A2 or above go on to further annotation.
  • Timeline: challenge participants get a pre-release from December 2024 to March 2025, and full public release is planned for April 2025. Compared with ICNALE, CLES, LearnerVoice, speechocean762, L2-ARCTIC, ETLT, NICT-JLE and KISTEC, the corpus offers wider L1 and proficiency coverage and includes audio.

Interesting ideas

  • I1. Transcripts are kept verbatim, with hesitations, false starts, repairs and code-switching, and a separate corrected layer sits alongside them, so messy input and its intended form stay aligned.
  • I2. A cheap holistic scoring pass runs first and filters out low-quality or off-task items before costly detailed annotation, which concentrates annotation effort where it pays off.
  • I3. Gold transcriptions were made by having humans fix ASR output instead of typing from scratch, a post-editing approach that speeds up annotation.
  • I4. Annotators marked phrase boundaries inside long unpunctuated utterances, giving spoken text explicit segmentation units.
  • I5. Test sets use only complete responses, while partially finished tests are still allowed in training data, so evaluation stays clean without discarding usable noisy data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Mova could parse a cleaned layer while keeping a link back to the raw tokens. Disfluencies and repairs would then be explainable edits rather than silent deletions.
  • I2 (learn): A quick quality or agreement score could triage candidate silver sentences before they reach expensive LLM teacher tiers.
  • I3 (learn): Annotators or a teacher model could correct Mova's own UD trees and frames instead of building them fresh, and the corrections could seed self-induced repair rules.
  • I4 (tokens): A boundary marker of this kind would help Mova split unpunctuated or speech-like input into parseable units before tagging.
  • I5 (learn): Mova's evaluation sets could be kept strictly to fully verified items, and partial or noisy material could still feed training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom243 works · 121,396 citations🏢Cambridge University PressCambridge · United Kingdom5 works · 355 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSpeech & spoken language · 480 works

Cite

Cite as Kate Knill, Diane Nicholls, Mark J.F. Gales, Mengjie Qian and Pawel Stroinski. 2024. Speak & Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback. arXiv:2412.11986.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2412.11986 · arXiv 2412.11986

@misc{knill2024speak,
  title = {{Speak \& Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback}},
  author = {Kate Knill, Diane Nicholls, Mark J.F. Gales, Mengjie Qian and Pawel Stroinski},
  year = {2024},
  doi = {10.48550/arxiv.2412.11986},
  eprint = {2412.11986},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2412.11986}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. D. Nicholls, K. M. Knill, M. J. F. Gales, A. Ragni, and P. Ricketts, “Speak & Improve: L2 English Speaking Practice Tool,” in INTERSPEECH 2023, 2023, pp. 3669–3670. find
  2. K. M. Knill, D. Nicholls, M. Gales, P. Stroinski, and A. Watkinson, “Annotation of L2 English Speech for Developing and Evaluating End-to-End Spoken Grammatical Error Correction,” in 9th Workshop on Speech and Language Technology in Education (SLaTE), 2023, pp. 146–150. find
  3. C. of Europe. Council for Cultural Co-operation. Education Committee. Modern Languages Division, Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge University Press, 2001. find
  4. M. Qian, K. Knill1, S. Banno, S. Tang, P. Karanasou, and M. J. G. andDiane Nicholls, “Speak & Improve Challenge 2025: Tasks and Baseline Systems,” 2024. [Online]. Available: http://arxiv.org/abs/2412.11985 ↗
  5. S. Coulange, M.-H. Fries, M. Masperi, and S. Rossato, “A corpus of spontaneous L2 English speech for real-situation speaking assessment,” in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024, pp. 293–297.
  6. H. Kim, J. Myung, S. Kim, S. Lee, D. Kang, and J. Kim, “LearnerVoice: A Dataset of Non-Native English Learners’ Spontaneous Speech,” in Interspeech 2024, 2024, pp. 2325–2329.
  7. J. Zhang, Z. Zhang, Y. Wang, Z. Yan, Q. Song, Y. Huang, K. Li, D. Povey, and Y. Wang, “speechocean762: An Open-Source Non-Native English Speech Corpus for Pronunciation Assessment,” in Interspeech 2021, 2021, pp. 3710–3714.
  8. G. Zhao, S. Sonsaat, A. Silpachai, I. Lucic, E. Chukharev-Hudilainen, J. Levis, and R. Gutierrez-Osuna, “L2-ARCTIC: A Non-native English Speech Corpus,” in Interspeech 2018, 2018, pp. 2783–2787. find
  9. R. Gretter, M. Matassoni, D. Falavigna, A. Misra, C. Leong, K. Knill, and L. Wang, “ETLT 2021: Shared Task on Automatic Speech Recognition for Non-Native Children’s Speech,” in Interspeech 2021, 2021, pp. 3845–3849. find
  10. R. Gretter, M. Matassoni, S. Bannò, and F. Daniele, “TLT-school: a Corpus of Non Native Children Speech,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, 2020, pp. 378–385. ↗
  11. E. Izumi, K. Uchimoto, and H. Isahara, “The NICT JLE Corpus: Exploiting the language learners’ speech database for research and education,” International Journal of the Computer, the Internet and Management, vol. 12, no. 2, pp. 119–125, 2004. find
  12. K. Kanzawa, Y. Kobayashi, J. Lee, H. Mitsunaga, M. Mori, , and Y. Tanaka, “The KIT Speaking Test Corpus (KISTEC),” https://kitstcorpus.jp/, 2022. ↗
  13. R. Dale and A. Kilgarriff, “Helping our own: The HOO 2011 pilot shared task,” in Proceedings of the 13th European Workshop on Natural Language Generation, 2011, pp. 242–249.
  14. H. T. Ng, S. M. Wu, Y. Wu, C. Hadiwinoto, and J. Tetreault, “The CoNLL-2013 Shared Task on Grammatical Error Correction,” in Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task. Sofia, Bulgaria: Association for Computational Linguistics, 2013, pp. 1–12. [Online]. Available: https://aclanthology.org/W13-3601 ↗
  15. H. T. Ng, S. M. Wu, T. Briscoe, C. Hadiwinoto, R. H. Susanto, and C. Bryant, “The CoNLL-2014 shared task on grammatical error correction,” in Proceedings of the eighteenth conference on computational natural language learning: shared task, 2014, pp. 1–14. find
  16. C. Bryant, M. Felice, Ø. E. Andersen, and T. Briscoe, “The BEA-2019 shared task on grammatical error correction,” in Proceedings of the fourteenth workshop on innovative use of NLP for building educational applications, 2019, pp. 52–75.
  17. “Shared task on Multilingual Grammatical Error shared-task-multilingual-grammatical-error-correction-2025, 2024. find
  18. B. Hamner, J. Morgan, lynnvandev, M. Shermis, and T. V. Ark, “The Hewlett Foundation: Automated Essay Scoring,” https://kaggle.com/competitions/asap-aes, 2012, kaggle. ↗
  19. C. Baur, C. Chua, J. Gerlach, M. Rayner, M. Russell, H. Strik, and X. Wei, “Overview of the 2017 Spoken CALL Shared Task,” in 7th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2017), 2017, pp. 71–78. find
  20. C. Baur, A. Caines, C. Chua, J. Gerlach, M. Qian, M. Rayner, M. Russell, H. Strik, and X. Wei, “Overview of the 2018 Spoken CALL Shared Task,” in Interspeech 2018, 2018, pp. 2354–2358. Routledge, 2023. Correction ,” https://www.aclweb.org/portal/content/ ↗
  21. ——, “Overview of the 2019 Spoken CALL Shared Task,” in 8th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2019), 2019, pp. 1–5. find
  22. J. Xu, M. Brenchley, E. Jones, A. Pinnington, T. Benjamin, K. Knill, G. Seal-Coon, M. Robinson, and A. Geranpayeh, “Linguaskill Building a validity argument for the Speaking test,” Linguaskill Research Reports, UCLES, Tech. Rep, 2020. ↗
  23. S. J. Young, G. Evermann, M. J. F. Gales, T. Hain, D. Kershaw, X. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. C. Woodland, The HTK Book (for HTK Version 3.4.1). University of Cambridge, 2009. [Online]. Available: http://htk.eng.cam.ac.uk ↗
  24. S. Balloccu, P. Schmidtová, M. Lango, and O. Dusek, “Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Y. Graham and M. Purver, Eds. St. Julian’s, Malta: Association for Computational Linguistics, Mar. 2024, pp. 67–93. [Online]. Available: https://aclanthology.org/2024.eacl-long.5 ↗
  25. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State-of-the-Art Natural Language Processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Q. Liu and D. Schlangen, Eds. Online: Association for Computational Linguistics, Oct. 2020, pp. 38–45. [Online]. Available: https://aclanthology.org/2020.emnlp-demos.6 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/knill-2024-speak-improve-corpus-2025-l2/