Elamur Research⌕
☰
home / papers / 2025 / Tracing L1 Interference in English Learner Writing: A Longitudinal Corpus with Error Annotations

Tracing L1 Interference in English Learner Writing: A Longitudinal Corpus with Error Annotations

Paper · 2025 · Poorvi Acharya, J. Elizabeth Liebl, Dhiman Goswami et al.

Summary

Learner corpora seldom explain why a non-native writer makes an error. Existing taxonomies mostly record where an error sits and what edit fixes it, and few resources follow the same students over time. The authors build LENS, a longitudinal English learner corpus. It contains 687 texts from international Master's students in a U.S. academic-writing course (2022–2024), covering 15 L1s. Most analyses use the Arabic, Chinese and Vietnamese cohorts, the only groups large enough to study.

The paper does two things with the corpus. First, it uses LLMs with human checking to annotate errors as L1 interference. The model must name the linguistic level involved and justify its label with a concrete property of the writer's native language. Second, it benchmarks Native Language Identification (NLI) with SVM ensembles, BERT-family encoders and LLMs. Fine-tuned GPT-4o performs best by a wide margin, while zero-shot LLMs fall behind classical baselines.

The resource makes it possible to study how transfer effects change during an academic term. It is released under CC BY-NC-SA 4.0.

Key points

  • Corpus scope: LENS has 687 texts from 15 L1s, about 175K words in total. Writers were advanced learners (IELTS 7) producing short answers, long essays and group reflections.

  • Analyzed subset: 104,396 tokens from three cohorts:

    • Arabic: 35 learners, 345 docs
    • Chinese: 18 learners, 133 docs
    • Vietnamese: 4 learners, 47 docs
    • Per-learner submission counts and week spans are reported as a measure of longitudinal depth.

  • Claimed novelty: LENS is presented as the first corpus to combine four features: fine-grained error labels, explicit L1-transfer annotations, per-learner metadata, and repeated submissions over time.

  • NLI results (accuracy / F1):

    SettingModelAccuracyF1
    Fine-tuned LLMGPT-4o0.970.96
    Fine-tuned LLMLLaMa 3.1 70B0.870.84
    EncoderRoBERTa0.790.75
    ClassicalSVM Ensemble0.750.73
    Zero-shot LLMGPT-4o0.660.66
    Zero-shot LLMFLAN-T50.320.37

  • Anonymization: An English NER module replaced person, organization and location tokens with placeholders, followed by a manual check. Only age, gender and L1 were kept as metadata.

Interesting ideas

  • I1. The LLM annotator must return a structured verdict (is this transfer, at which linguistic level, and which feature of the native language explains it), and humans verify a subset; demanding a justification makes LLM labels auditable rather than opaque.
  • I2. Errors are classified by their cause, not only by their location or the repair operation, which turns an error log into a diagnostic signal.
  • I3. Strong classical baselines (SVM over n-grams, fine-tuned encoders) outperform off-the-shelf zero-shot LLMs, so a cheap baseline is essential before trusting a large model.
  • I4. The dataset is described per group (documents, tokens, median length, long vs short task ratio, time span) because cohorts differ in composition and could otherwise confound results.
  • I5. Named entities are swapped for placeholders while the rest of the text stays intact, preserving structure without identities.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM teacher tiers could be required to attach a level tag and a checkable linguistic reason to every silver label, so disagreeing witnesses can be adjudicated on stated grounds.
  • I2 (learn): self-induced repair rules could record a hypothesized cause (tokenization, attachment, tagging) alongside the edit, making clusters of failures easier to fix at the right stage.
  • I3 (learn): every evaluation could include the perceptron parser and simple n-gram models as controls, so gains from gparse or LLM tiers are measured against a real floor.
  • I4 (learn): treebank and test-set reports could be broken down by genre and length, so accuracy differences are not mistaken for model effects.
  • I5 (world): running fables or word problems with names replaced by placeholders would test whether entity tracking and holdings rely on structure rather than on recognizing specific names.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksDevelopmental and Educational Psychology · 523 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,379 works

Cite

Cite as Poorvi Acharya, J. Elizabeth Liebl, Dhiman Goswami, Kai North, Marcos Zampieri and Antonios Anastasopoulos. 2025. Tracing L1 Interference in English Learner Writing: A Longitudinal Corpus with Error Annotations. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.766

@inproceedings{acharya2025tracing,
  title = {{Tracing L1 Interference in English Learner Writing: A Longitudinal Corpus with Error Annotations}},
  author = {Poorvi Acharya, J. Elizabeth Liebl, Dhiman Goswami, Kai North, Marcos Zampieri and Antonios Anastasopoulos},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.766},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.766}
}

References

59 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. ↗
  2. Luca Benedetto, Giovanni Aradelli, Antonia Donvito, Alberto Lucchetti, Andrea Cappelli, and Paula Buttery. 2024. Using LLMs to simulate students’ responses to exam questions. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11351–11368, Miami, Florida, USA. Association for Computational Linguistics. find
  3. Daniel Blanchard, Joel Tetreault, Derrick Higgins, Aoife Cahill, and Martin Chodorow. 2013. Toefl11: A corpus of non-native english. ETS Research Report Series, 2013(2):i–15. ↗
  4. Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52–75, Florence, Italy. Association for Computational Linguistics.
  5. Centre for English Corpus Linguistics (CECL). 2024. Learner corpora around the world. https://uclouvain.be/en/research-institutes/ilc/ cecl/learner-corpora-around-the-world. html. Accessed: 2025-09-17. ↗
  6. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25:1–53. find
  7. Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017. Multiword expression processing: A survey. Computational Linguistics, 43(4):837–892. ↗
  8. Cyril Goutte and Serge Léger. 2017. Exploring optimal voting in native language identification. In Proceedings of BEA. find
  9. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner English: The NUS corpus of learner English. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 22–31, Atlanta, Georgia. Association for Computational Linguistics.
  10. Arturo E. Hernandez, Juliana Ronderos, Jean Philippe Bodet, Hannah Claussenius-Kalman, My V. H. Nguyen, and Ferenc Bunta. 2021. German in childhood and latin in adolescence: On the bidialectal nature of lexical access in english. Humanities and Social Sciences Communications, 8(1):162. ↗
  11. J. De Gregorio, R. Toral, and D. Sánchez. 2024. Exploring language relations through syntactic distances and geographic proximity. EPJ Data Science, 13:61. ↗
  12. Pieter de Haan. 2000. Tagging non-native english with the tosca-icle tagger. Corpus Linguistics and Linguistic Theory, pages 69–79. ↗
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL. ↗
  14. Ana Díaz-Negrillo and Jesús Fernández-Domínguez. 2006. Error Tagging Systems for Learner Corpora. Revista española de lingüística aplicada, ISSN 0213-2028, Vol. 19, 2006, pags. 83-102, 19. find
  15. Rod Ellis. 2015. Understanding second language acquisition 2nd edition. Oxford university press. find
  16. Tao Fang, Shu Yang, Kaixin Lan, Derek F. Wong, Jinpeng Hu, Lidia S. Chao, and Yue Zhang. 2023. Is chatgpt a highly fluent grammatical error correction system? a comprehensive evaluation. Preprint, arXiv:2304.01746. ↗
  17. Susan M Gass, Jennifer Behney, and Luke Plonsky. 2020. Second language acquisition: An introductory course. Routledge. find
  18. Binyam Gebrekidan Gebre, Marcos Zampieri, Peter Wittenburg, and Tom Heskes. 2013. Improving native language identification with tf-idf weighting. In Proceedings of BEA. find
  19. Jeroen Geertzen, Theodora Alexopoulou, and Anna Korhonen. 2014. Efcamdat: The ef-cambridge open language database. In Selected Proceedings of the 2012 Second Language Research Forum, pages 240–254, Somerville, MA. Cascadilla Proceedings Project. find
  20. Dhiman Goswami, Sharanya Thilagan, Kai North, Shervin Malmasi, and Marcos Zampieri. 2024. Native language identification in texts: A survey. In Proceedings of NAACL. find
  21. Dhiman Goswami, Marcos Zampieri, Kai North, Shervin Malmasi, and Antonios Anastasopoulos. 2025. Multilingual native language identification with large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop), pages 193–199. find
  22. András Imrényi. 2013. The syntax of Hungarian auxiliaries: A dependency grammar account. In Proceedings of the Second International Conference on Dependency Linguistics (DepLing 2013), pages 118– 127, Prague, Czech Republic. Charles University in Prague, Matfyzpress, Prague, Czech Republic. find
  23. Shin’ichiro Ishikawa. 2023. The ICNALE Guide: An Introduction to a Learner Corpus Study on Asian Learners’ L2 English. Routledge, London. find
  24. Scott Jarvis and Scott A Crossley. 2012. Approaching Language Transfer Through Text Classification: Explorations in the Detectionbased Approach. Multilingual Matters. find
  25. Masamune Kobayashi, Masato Mita, and Mamoru Komachi. 2024. Large language models are state-ofthe-art evaluator for grammatical error correction. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 68–77, Mexico City, Mexico. Association for Computational Linguistics. find
  26. Stephen Krashen. 1981. Second language acquisition. Second Language Learning. find
  27. G Radha Krishna, R Krishnan, and VK Mittal. 2019. An automated system for regional nativity identification of indian speakers from english speech. In Proceedings of IEEE INDICON. find
  28. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. ↗
  29. Mengsay Loem, Masahiro Kaneko, Sho Takase, and Naoaki Okazaki. 2023. Exploring effectiveness of GPT-3 in grammatical error correction: A study on performance and controllability in prompt-based methods. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 205–219, Toronto, Canada. Association for Computational Linguistics. find
  30. Ehsan Lotfi, Ilia Markov, and Walter Daelemans. 2020. A deep generative approach to native language identification. In Proceedings of COLING. find
  31. Anke Lüdeling and Hagen Hirschmann. 2015. Error annotation systems, page 135–158. Cambridge Handbooks in Language and Linguistics. Cambridge University Press. find
  32. Shervin Malmasi and Mark Dras. 2017. Multilingual native language identification. Natural Language Engineering. find
  33. Timothy Osborne, Michael Putnam, and Thomas Groß. 2012. Catenae: Introducing a novel unit of syntactic analysis. Syntax, 15(4):354–396. ↗
  34. Shervin Malmasi, Keelan Evanini, Aoife Cahill, Joel Tetreault, Robert Pugh, Christopher Hamill, Diane Napolitano, and Yao Qian. 2017. A report on the 2017 native language identification shared task. In Proceedings of BEA. find
  35. Erin Pacquetet. 2024. The Effect of Linguistic Properties on Typing Behaviors and Production Processes. Ph.D. thesis, ProQuest LLC. ERIC Number: ED657074. find
  36. Shervin Malmasi et al. 2016. Native language identification: explorations and applications. Ph.D. thesis, Macquarie University. ↗
  37. Beata Megyesi, Lena Granstedt, Sofia Johansson, Julia Prentice, Daniel Rosén, Carl-Johan Schenström, and Elena Volodina. 2018. Learner corpus anonymization in the age of gdpr: Insights from the creation of a learner corpus of swedish. In Proceedings of the 7th Workshop on NLP for Computer Assisted Language Learning at SLTC 2018 (NLP4CALL 2018), pages 47–56. find
  38. Fanny Meunier. 2016. Introduction to the longdale project. In Erik Castello, Katherine Ackerley, and Francesca Coccetta, editors, Studies in Learner Corpus Linguistics. Research and Applications for Foreign Language Teaching and Assessment, pages 123– 126. Peter Lang, Berlin. find
  39. Atsushi Mizumoto and Masaki Eguchi. 2023. Exploring the potential of using an ai language model for automated essay scoring. Research Methods in Applied Linguistics, 2(2):100050. ↗
  40. Liat Nativ, Yuval Nov, Noam Ordan, Shuly Wintner, and Anat Prior. 2024. Do more proficient writers use fewer cognates in l2? a computational approach. Bilingualism: Language and Cognition, 27(1):84–94. ↗
  41. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online. Association for Computational Linguistics. ↗
  42. Jack C. Richards. 1971. A non-contrastive approach to error analysis1. ELT Journal, XXV(3):204–219. ↗
  43. Bandar Mohammad Saeed Al-Sobhi. 2019. The nitty-gritty of language learners’ errors – contrastive analysis, error analysis and interlanguage. International Journal of Education and Literacy Studies, 7(3):49. ↗
  44. Burr Settles, Chris Brust, Erin Gustafson, Masato Hagiwara, and Nitin Madnani. 2018. Second language acquisition modeling. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 56–65, New Orleans, Louisiana. Association for Computational Linguistics. find
  45. Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Kevin Gimpel, and Mohit Iyyer. 2024. GEE! grammar error explanation with large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 754–781, Mexico City, Mexico. Association for Computational Linguistics. find
  46. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1–14, Baltimore, Maryland. Association for Computational Linguistics. find
  47. Bernardo Stearns, Nicolas Ballier, Thomas Gaillat, Andrew Simpkin, and John P. McCrae. 2024. Evaluating the generalisation of an artificial learner. In Proceedings of the 13th Workshop on Natural Language Processing for Computer Assisted Language Learning, pages 199–208, Rennes, France. LiU Electronic Press. find
  48. Yee Man Ng and Ilia Markov. 2025. Leveraging open-source large language models for native language identification. In Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 20–28, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  49. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. ↗
  50. Diane Nicholls. 2003. The cambridge learner corpus: Error coding and analysis for lexicography and elt. In Proceedings of the Corpus Linguistics 2003 conference, volume 16, page 572–581. Cambridge University Press Cambridge. find
  51. Ahmet Yavuz Uluslu and Gerold Schneider. 2022. Scaling native language identification with transformer adapters. In Proceedings of the 5th International Conference on Natural Language and Speech Processing (ICNLSP 2022), pages 298–302, Trento, Italy. Association for Computational Linguistics. find
  52. Terence Odlin. 1989. Language Transfer. Cambridge Applied Linguistics. Cambridge University Press. find
  53. Lourdes Ortega. 2014. Understanding second language acquisition. Routledge. ↗
  54. Sowmya Vajjala and Taraka Rama. 2018. Experiments with universal CEFR classification. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 147–153, New Orleans, Louisiana. Association for Computational Linguistics. find
  55. Georgios Velentzas, Andrew Caines, Rita Borgo, Erin Pacquetet, Clive Hamilton, Taylor Arnold, Diane Nicholls, Paula Buttery, Thomas Gaillat, Nicolas Ballier, and Helen Yannakoudakis. 2024. Logging keystrokes in writing by English learners. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10725– 10746, Torino, Italia. ELRA and ICCL.
  56. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 180–189, Portland, Oregon, USA. Association for Computational Linguistics.
  57. Marcos Zampieri, Alina Maria Ciobanu, and Liviu P. Dinu. 2017. Native language identification on text and speech. In Proceedings of BEA. find
  58. Wei Zhang and Alexandre Salle. 2023. Native language identification with large language models. arXiv preprint arXiv:2312.07819. ↗
  59. Gustavo Zomer and Ana Frankenberg-Garcia. 2021. Beyond grammatical error correction: Improving L1-influenced research writing in English using pre-trained encoder-decoder models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2534–2540, Punta Cana, Dominican Republic. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/acharya-2025-tracing-l1-interference-english-learner/