📄 What makes a word hard to learn? Modeling L1 influence on English vocabulary difficulty

Summary

The paper asks why some English words are harder to learn than others, and how much that depends on the learner's native language. The authors use the Knowledge-based Vocabulary Lists (KVL), crowd-sourced form-recall tests from the closed track of the BEA 2026 shared task. Each item gives a word in the learner's L1, a context sentence and the first English letter; the learner types the English word. Item difficulty is estimated from the responses with a Rasch model. For Spanish, German and Chinese L1 speakers, separate CatBoost regressors are trained on 24 features in four groups: familiarity, meaning, surface form and L1 transfer. Shapley values then show how much each group contributes. Familiarity, mainly frequency and exposure, dominates for all three languages. For Spanish and German, orthographic overlap with the L1 word gives a second route to easiness. Chinese learners lack this route, and their difficulty depends on familiarity plus surface features. The result is interpretable, L1-specific difficulty estimates for planning vocabulary curricula.

Key points

• KVL draws on responses from more than 100,000 English learners. It provides 6,091 train, 677 dev and 748 test items for each L1 (Spanish, German, Chinese), and higher Rasch scores mean easier words.
• The features are 24 in total: 11 for familiarity (e.g., log frequency and contextual diversity from SUBTLEX-UK, age of acquisition), plus meaning and surface groups, and a single transfer feature. That feature is the cosine similarity of character n-gram TF-IDF vectors for the English word and its L1 translation, and it is always zero for Chinese.
• The word cable shows the L1 effect: it scores 2.3 for Spanish (identical spelling), 0.9 for German (cognate Kabel) and 0.0 for Chinese (电缆).
• Meaning features add little, probably because the task tests spelling recall rather than understanding. Spanish- and German-trained models transfer well to each other.
• An interactive demo extends predictions to roughly 4,000–7,000 extra SUBTLEX-UK words per language.

Interesting ideas

• I1. Grouping features and summing their Shapley values per group lets you compare how different populations' models reason, not just how accurate they are; this turns attribution into a profile you can contrast.
• I2. A Rasch model estimates item difficulty and solver ability jointly from binary right/wrong outcomes, so the difficulty score is not confounded by who happened to attempt each item.
• I3. A very crude orthographic similarity (character n-gram TF-IDF cosine) proved a strong signal wherever scripts are shared, so a cheap proxy can carry much of a linguistic effect before investing in phonology.
• I4. Training on one population and testing on another is used as evidence that the two share an underlying mechanism, with typology predicting where transfer will work.
• I5. The authors stress that the test format decides which knowledge gets measured: meaning features look unimportant only because the task probes spelling.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Group parser and frame-extractor features (lexical, morphological, positional), then compare group attributions across genres such as fables and school tests to see where each witness relies on different cues.
• I2 (answer): Fit a Rasch model over many reader variants and test questions to get calibrated item difficulty, which helps separate genuinely hard questions from weak readers.
• I3 (back): When translating through the meta-world, score source and target word pairs this way to flag likely cognates, and to mark high-similarity pairs as false-friend risks for extra checking.
• I4 (tree): Train the perceptron parser or gparse on one treebank dialect and evaluate on another; strong transfer would suggest that conversion rules capture shared structure, while failures point to dialect-specific annotation.
• I5 (learn): Design evaluations so that a component's measured importance is not an artifact of the probe, for example by pairing each test with a format that requires the skill and a negative control that does not.