Elamur Research⌕
☰
home / papers / 2018 / EFLLex: A Graded Lexical Resource for Learners of English as a Foreign Language

EFLLex: A Graded Lexical Resource for Learners of English as a Foreign Language

Paper · 2018 · Luise Dürlich and Thomas François

Summary

Curriculum designers and teachers have to decide which English words to teach at each stage. Native-speaker frequency lists such as the BNC do not match what learners actually read and say nothing about proficiency levels. Expert lists such as the English Vocabulary Profile assign each word to a single level. EFLLex, the English member of the CEFRLex family (after French and Swedish), takes a different view: it treats vocabulary learning as continuous and gives each word a frequency distribution across CEFR levels.

The authors built a corpus of 1,971 reading-comprehension texts from textbooks, graded readers and online materials, each levelled by its publisher. Five POS taggers were compared on a manually adjudicated sample, and NLP4J was chosen to lemmatize and tag the whole corpus. Per-level frequencies were then computed with a robust estimator, and the output was manually post-edited.

The result describes 15,280 English words and was compared with MRC, BNC and EVP. It shows where a word already appears below its nominal level and ranks words within a level. This makes it useful for language teaching and computer-assisted language learning.

Key points

  • The corpus has 1,971 texts and 486,178 words, drawn from 17 textbooks, 33 graded readers and 7 online sources. It covers levels A1–C1 only, since C2 material was scarce, and B2 and C1 are the largest subcorpora.
  • The tagger candidates were TreeTagger, a FreeLing HMM (TnT-style) tagger, the Stanford tagger, SVMTool and NLP4J. NLP4J scored best with 95.88% fully correct tokens.
  • Two experts rated the tagger output with five error categories:
    • correct;
    • wrong POS;
    • wrong lemma;
    • both wrong;
    • segmentation error.
  • Agreement between the experts was weak (Cohen's Kappa from 0.378 to 0.627), so they built an agreed reference annotation before scoring.
  • Frequencies are counted per lemma+POS rather than per inflected form, so that verbs are not undercounted. Per-level counts reveal contrasts, for example write occurs 934 times at A1 while explore occurs only 20 times.

Interesting ideas

  • I1. Representing a word's difficulty as a distribution over levels, rather than a single label, keeps information about early exposure and allows ranking within a level.
  • I2. Choosing a tagger by error type (POS vs lemma vs segmentation), not overall accuracy, shows which failure would actually harm the resource.
  • I3. When annotator agreement is low, experts reconcile their judgments into one adjudicated reference before scoring the systems.
  • I4. Counting at the lemma+POS level avoids splitting the mass of highly inflected words and separates homographs.
  • I5. Building a word list from texts actually aimed at learners, instead of native corpora, matches the target audience's real exposure.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Compiled lexical knowledge could store graded or distributional attributes, such as register or level spread, instead of one hard class, so that downstream reasoning can explain borderline cases.
  • I2 (tokens): The en crate's evaluation could report lemma, tag and segmentation errors separately, so regressions in lemmatization (e.g. her lemmatized as she) are not hidden inside one accuracy number.
  • I3 (learn): When witness parsers or LLM teacher tiers disagree, a small adjudicated gold slice could serve as the reference for scoring silver data, instead of trusting raw majority votes.
  • I4 (answer): Vocabulary statistics used by school-course readers could be keyed on lemma+POS, so judgments about difficulty or familiarity are not skewed by inflection.
  • I5 (back): Generation for learners or children could prefer words that are frequent at the reader's level in such graded material, which keeps output understandable and gives an explainable lexical choice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓UCLouvainLouvain-la-Neuve · Belgium11 works · 3,665 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksDevelopmental and Educational Psychology · 260 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Luise Dürlich and Thomas François. 2018. EFLLex: A Graded Lexical Resource for Learners of English as a Foreign Language. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{durlich2018efllex,
  title = {{EFLLex: A Graded Lexical Resource for Learners of English as a Foreign Language}},
  author = {Luise Dürlich and Thomas François},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/43e9y9se8gyj},
  url = {https://doi.org/10.63317/43e9y9se8gyj}
}

References

29 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alderson, J. (2007). The CEFR and the need for more research. The Modern Language Journal, 91(4):659–663. ↗
  2. Baayen, R., Piepenbrock, R., and van Rijn, H. (1993). The CELEX lexical data base on CD-ROM. find
  3. Bahns, J. and Eldaw, M. (1993). Should We Teach EFL Students Collocations? System, 21(1):101–14. ↗
  4. Brants, T. (2000). TnT – a statistical part-of-speech tagger. In Proceedings of the 6th Conference on Applied Natural Language Processing, pages 224–231. ↗
  5. Brysbaert, M. and New, B. (2009). Moving beyond kučera and francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for american english. Behavior research methods, 41(4):977–990. ↗
  6. Capel, A. (2010). A1-B2 vocabulary: Insights and issues arising from the English Profile Wordlists project. English Profile Journal, 1(1):1–11. ↗
  7. Capel, A. (2012). Completing the English Vocabulary Profile: C1 and C2 vocabulary. English Profile Journal, 3:1– 14. ↗
  8. Carreras, X., Chao, I., Padró, L., and Padró, M. (2004). Freeling: An open-source suite of language analyzers. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC’04).
  9. Carroll, J., Davies, P., and Richman, B. (1971). The American Heritage word frequency book. American Heritage Publishing CO., New York. find
  10. Choi, J. D. (2016). Dynamic feature induction: The last gist to the state-of-the-art. In Proceedings of NAACL. find
  11. Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Press Syndicate of the University of Cambridge. find
  12. François, T., Gala, N., Watrin, P., and Fairon, C. (2014). FLELex: a graded lexical resource for French foreign learners. In Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014). find
  13. François, T., Volodina, E., Ildikó, P., and Tack, A. (2016). SVALex: a CEFR-graded lexical resource for Swedish foreign and second language learners. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), pages 213–219. find
  14. Francis, W. N. and Kučera, H. (1982). Frequency Analysis of English Usage. Houghton Mifflin. find
  15. Giménez, J. and Màrquez, L. (2004). SVMTool: A general POS tagger generator based on support vector machines. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal.
  16. Kučera, H. and Francis, W. N. (1967). Computational analysis of present-day American English. Dartmouth Publishing Group. find
  17. Laufer, B. and Ravenhorst-Kalovski, G. (2010). Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a foreign language, 22(1):15–30. ↗
  18. Laufer, B. (1992). How much lexis is necessary for reading comprehension? In Vocabulary and applied linguistics, pages 126–132. Springer. ↗
  19. Leech, G., Rayson, P., and Wilson, A. (2001). Word frequencies in written and spoken english: based on the british national corpus. find
  20. Marello, C. (2012). Word lists in Reference Level Descriptions of CEFR (Common European Framework of Reference for Languages). In Proceedings of the XV Euralex International Congress, pages 328–335. find
  21. North, B. (2005). The CEFR levels and descriptor scales. In Multilingualism and assessment: Achieving transparency, assuring quality, sustaining diversity. Proceedings of the ALTE Berlin Conference, pages 21–66. find
  22. Paquot, M. and Granger, S. (2012). Formulaic language in learner corpora. Annual Review of Applied Linguistics, 32:130–149. ↗
  23. Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. In Proceedings of the International Conference on New Methods in Language Processing (NeMLaP), pages 44–49. find
  24. Tack, A., François, T., Ligozat, A.-L., and Fairon, C. (2016). Evaluating lexical simplification and vocabulary knowledge for learners of french: possibilities of using the flelex resource. In Proceedings of the Tenth conference on International Language Resources and Evaluation (LREC’16), pages 230–236. find
  25. Thorndike, E. and Lorge, I. (1944). The teacher’s wordbook of 30,000 words. find
  26. Thorndike, E. (1921). Word knowledge in the elementary school. The Teachers College Record, 22(4):334–370. ↗
  27. Toutanova, K., Klein, D., Manning, C. D., and Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics.
  28. Wilson, M. (1988). MRC Psycholinguistic Database: Machine Usable Dictionary, Version 2.00. In Behaviour Research Methods, Instruments & Computers, volume 20, pages 6–10. Springer US. ↗
  29. Zeno, S., Ivens, S., Millard, R., and Duvvuri, R. (1995). The educator’s word frequency guide. Touchstone Applied Science Associates, Inc., New York. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/durlich-2018-efllex-graded-lexical-resource-learners/