Elamur Research⌕
☰
home / papers / 2025 / Using NLI to Identify Potential Collocation Transfer in L2 English

Using NLI to Identify Potential Collocation Transfer in L2 English

Paper · 2025 · Haiyin Yang, Zoey Liu and Stefanie Wulff

Summary

The paper asks whether native language identification (NLI) can serve as a discovery tool for L1 collocation transfer in learner English. Classic transfer studies start from hypothesised structures and need native data from both languages. NLI instead lets a classifier highlight the word combinations that best separate L1 groups. The authors used 15 L1 groups from the International Corpus of Learner English (ICLE). They extracted four collocation types using spaCy dependency parses: adverb-verb, verb-noun, adjective-noun, and three-word bundles headed by a verb. Features were filtered by usage share, one-way ANOVA, and a prompt-entropy score that removes topic-driven items. A ridge classifier was then trained under 10-fold cross-validation.

The top-weighted features overlapped with earlier findings:

  • French learners' three-word bundles matched those reported by Paquot (2013).
  • Chinese learners used a narrower range of intensifiers, as Wu and Tissari (2021) found.

Hierarchical clustering compared with the confusion matrix showed that weak results for some L1s come from small samples and collocation profiles shared with other groups. The approach offers a broad, interpretable screen for transfer candidates, including for L1/L2 pairs with little prior research.

Key points

  • Data: ICLE essays from 15 L1s, roughly 5,600 texts averaging about 600 words. L1s with under 2% of the data were dropped.
    • Group sizes are unbalanced: mean 379, SD 171.
    • Chinese is the largest group (980 essays, about 16%) and Finnish the smallest (230, under 4%).
  • Collocation extraction: spaCy dependency relations ensure word pairs are syntactically linked rather than merely adjacent. For three-word bundles, the verb must be an ancestor of the other two words.
  • Topic control: each feature gets an entropy score over the essay prompts it appears in, with a cutoff of 0.25. Low entropy means the feature is tied to a few prompts and is discarded.
  • Model and diagnosis: a ridge classifier, whose high-coefficient features were compared with prior transfer studies. Classification was strong for Chinese, Japanese, Tswana and Italian, and weak for Czech and Finnish.
  • Intensifier finding and limitations: among top adverb-verb features, French had 6 intensifier types and Italian 5, while Tswana and Japanese had only 3.
    • About 45% of training features never occur in test data.
    • Proficiency is not balanced across L1 groups.
    • spaCy's accuracy on L2 English is unknown (around 90% labeled dependencies on standard text).

Interesting ideas

  • I1. Collocations are defined through dependency links (head-child relations, plus a clause-bounded ancestor check for triples) rather than surface adjacency, which removes spurious pairs such as an adverb that belongs to a different verb.
  • I2. The spread of a feature across source prompts is scored with entropy, and features concentrated in a few topics are discarded as confounds.
  • I3. An interpretable linear model is used as a candidate generator, and its top features are then checked against independent prior findings.
  • I4. Hierarchical clustering of class profiles is set beside the confusion matrix to explain errors as genuine similarity between classes rather than model failure.
  • I5. The authors note that the parser's accuracy on non-native text is unknown, even though every feature depends on it.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Verb-object and adjective-noun preference tables in the compiled knowledge layer could be harvested from UD trees with the same constraints, so each entry traces back to a real syntactic relation.
  • I2 (learn): Silver-data patterns or self-induced repair rules could be scored by entropy across source documents or genres, rejecting rules that only fire in one text family.
  • I3 (learn): Weights from a perceptron or a rule learner could be exported as ranked hypotheses and validated against external resources (e.g., VerbNet classes), with mismatches treated as items for review.
  • I4 (tree): Clustering dependency labels by their context profiles and comparing the clusters with the parser's label-confusion matrix could separate truly ambiguous label pairs from fixable errors.
  • I5 (tree): Add a learner-English evaluation slice for the perceptron parser and gparse, so that downstream frames built from L2 input carry a known error rate.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of FloridaGainesville · United States33 works · 364 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

🧭 Topics 4Research subjects and languages the work is about.English · 2,263 worksReasoning · 1,426 worksIdioms & multiword expressions · 217 worksGrammatical error correction & learner language · 156 works

Cite

Cite as Haiyin Yang, Zoey Liu and Stefanie Wulff. 2025. Using NLI to Identify Potential Collocation Transfer in L2 English. Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.bea-1.49

@inproceedings{yang2025using,
  title = {{Using NLI to Identify Potential Collocation Transfer in L2 English}},
  author = {Haiyin Yang, Zoey Liu and Stefanie Wulff},
  booktitle = {Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)},
  year = {2025},
  doi = {10.18653/v1/2025.bea-1.49},
  url = {https://doi.org/10.18653/v1/2025.bea-1.49}
}

References

30 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yves Bestgen and Sylviane Granger. 2011. Categorising spelling errors to assess l2 writing. International journal of continuing engineering education and lifelong learning, 21(2/3):235–252. ↗
  2. Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. 2013. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122. ↗
  3. Hakan Cangır and Philip Durrant. 2021. Cross-linguistic collocational networks in the l1 turkish–l2 english mental lexicon. Lingua, 258:103057. ↗
  4. A. Cowie. 2006. Phraseology, pages 579–585. Elsevier, Oxford. find
  5. Philip Durrant and Alice Doherty. 2010. Are high-frequency collocations psychologically real? investigating the thesis of collocational priming. Corpus linguistics and linguistic theory, 6(2):125–155. ↗
  6. Dhiman Goswami, Sharanya Thilagan, Kai North, Shervin Malmasi, and Marcos Zampieri. 2024. Native language identification in texts: A survey. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3149–3160, Mexico City, Mexico. Association for Computational Linguistics. find
  7. S. Granger. 1998. The computer learner corpus: A versatile new source of data for SLA research, pages 3– 18. Addison Wesley Longman, London New York. find
  8. S. Granger, M. Dupont, F. Meunier, H. Naets, and M. Paquot. 2020. The International Corpus of Learner English. Version 3. Louvainla-Neuve: Presses universitaires de Louvain. find
  9. Martin Hilpert. 2008. New evidence against the modularity of grammar: Constructions, collocations, and speech perception. Cognitive linguistics, 19(3):491– 511. ↗
  10. Michael Hoey. 2005. Lexical priming : a new theory of words and language. Routledge, London, UK. ↗
  11. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spacy: Industrial-strength natural language processing in python. find
  12. Radu Tudor Ionescu and Marius Popescu. 2017. Can string kernels pass the test of time in Native Language Identification? In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 224–234, Copenhagen, Denmark. Association for Computational Linguistics. find
  13. S. Jarvis. 2000. Methodological rigor in the study of transfer : Identifying l1 influence in the interlanguage lexicon. Language learning, 50(2):245–309. find
  14. Scott Jarvis, Yves Bestgen, and Steve Pepper. 2013. Maximizing classification accuracy in native language identification. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 111–118, Atlanta, Georgia. Association for Computational Linguistics. find
  15. Scott Jarvis and Scott A. Crossley. 2018. The Detection-Based Approach: An Overview, pages 1–33. Multilingual Matters, Bristol, Blue Ridge Summit. find
  16. Vsevolod Kapatsinski and Joshua Radicke. 2009. Frequency and the emergence of prefabs: Evidence from monitoring. Formulaic language: Acquisition, loss, psychological reality, functional explanations, 2:499–520. ↗
  17. Moshe Koppel, Jonathan Schler, and Kfir Zigdon. 2005. Automatically determining an anonymous author’s native language. In International Conference on Intelligence and Security Informatics, pages 209–217. Springer. find
  18. Stephen D. Krashen. 1981. Second language acquisition and second language learning, 1st edition. Language teaching methodology series. Pergamon Press, Oxford ;. find
  19. Batia Laufer and Tina Waldman. 2011. Verb-noun collocations in second language writing: A corpus analysis of learners’ english. Language learning, 61(2):647– 672. ↗
  20. Zoey Liu, Tiwalayo Eisape, Emily Prud’hommeaux, and Joshua K Hartshorne. 2022. Data-driven crosslinguistic syntactic transfer in second language learning. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 44. find
  21. Ehsan Lotfi, Ilia Markov, and Walter Daelemans. 2020. A Deep Generative Approach to Native Language Identification. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1778–1783, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  22. Shervin Malmasi and Mark Dras. 2015. Large-scale native language identification with cross-corpus evaluation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1403–1409, Denver, Colorado. Association for Computational Linguistics. find
  23. Ilia Markov, Vivi Nastase, and Carlo Strapparava. 2020. Exploiting native language interference for native language identification. Natural Language Engineering, page 1–31. ↗
  24. Nadja Nesselhauf. 2003. The use of collocations by advanced learners of english and some implications for teaching. Applied linguistics, 24(2):223–242. ↗
  25. Magali Paquot. 2013. Lexical bundles and l1 transfer effects. International journal of corpus linguistics, 18(3):391–417. ↗
  26. J. M. Sinclair. 1991. Collocation. Oxford University Press, Oxford. find
  27. Anna Siyanova and Norbert Schmitt. 2008. L2 learner production and processing of collocation: A multi-study perspective. Canadian modern language review, 64(3):429–458. find
  28. Jr Ward, Joe H. 1963. Hierarchical grouping to optimize an objective function. Journal of the American statistical association, 58(301):236–244. ↗
  29. Brent Wolter and Henrik Gyllstad. 2011. Collocational links in the l2 mental lexicon and the influence of l1 intralexical knowledge. Applied linguistics, 32(4):430–449. ↗
  30. Junyu Wu and Heli Tissari. 2021. Intensifier-verb collocations in academic english by chinese learners compared to native-speaker students. Chinese journal of applied linguistics, 44(4):470–487. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2025-nli-identify-potential-collocation-transfer/