Elamur Research⌕
☰
home / papers / 2021 / Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset

Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset

Paper · 2021 · Matthew Shardlow, Richard Evans and Marcos Zampieri

Summary

The paper argues that Complex Word Identification (CWI) has been framed badly. Most work treats it as a binary decision, usually decided by whether any annotator marked a word as hard. The authors propose a continuous task instead, Lexical Complexity Prediction (LCP).

They do three things:

  • They survey the CWI–2016 and CWI–2018 shared tasks.
  • They study which features correlate with complexity.
  • They examine annotation problems in CWI–2016, CWI–2018 and Maddela–2018.

From this analysis they derive a set of recommendations for building complexity datasets. They then extend CompLex 1.0 into CompLex 2.0, an English corpus covering three genres. In it, crowd workers rate each target word in context on a 5-point Likert scale, and the ratings are averaged into a single score. The resource covers single words and multi-word expressions, and it served as the official data for SemEval-2021 Task 1.

The experiments show three things:

  • Complexity can be predicted across genres.
  • Words rated harder are also rated less consistently.
  • The same token's score shifts between contexts.

The authors conclude that graded judgments describe complexity better than binary ones. CompLex 2.0 is the first public dataset of continuous, in-context complexity scores.

Key points

  • CWI–2016 (SemEval):
    • Training data had 2,237 target words in 200 sentences, each rated by 20 annotators; one "complex" vote was enough to label a word complex.
    • Test data had 88,221 words in 9,000 sentences, each judged by a single annotator.
    • 21 teams submitted 42 systems, and the best F1 was only 0.353 (PLUJAGH).
  • CWI–2018 (BEA workshop): it covered English, Spanish, German and French, and added a probabilistic track. However, its "probability" was simply the share of annotators who chose the binary label "complex".
  • CompLex rating scale: a 5-point Likert scale from Very Easy to Very Difficult, with the mean taken per instance. The authors acknowledge this assumes the scale points are evenly spaced.
  • Categorical comparison: the CWI-2016 and CWI-2018 datasets outperformed CompLex 2.0 (Table 13). The authors present this as a comparison of which features matter, not as a quality ranking.
  • Release and future work: CompLex 2.0 is released under CC-BY at github.com/MMU-TDMLab/CompLex. The authors suggest future work on sense-annotated complexity data.

Interesting ideas

  • I1. Replace "one annotator objected, so the label is positive" with averaged graded ratings, so that a single outlier cannot flip a label and the scores carry fine-grained signal.
  • I2. Read the spread of annotator ratings as a signal in its own right: items with high disagreement are genuinely subjective, not just noisy.
  • I3. Score each new dataset against a written table of design recommendations, so that resources can be compared feature by feature.
  • I4. Train on several genres together and test on held-out genres, to check that the model learns genre-independent properties rather than domain quirks.
  • I5. Model difficulty from frequency and polysemy jointly, with a predicted order from frequent monosemous words (easiest) to rare polysemous words (hardest).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): store silver-tree agreement among witness parsers as a graded score per arc, not a binary accept/reject, and weight training examples by that score.
  • I2 (answer): when a reader faces a test question, items whose underlying words or frames are "disputed" in the teacher/witness data could get lower answer confidence and an explicit explanation of why.
  • I3 (learn): keep a fixed checklist for every treebank and silver set (sources, genres, agreement, context kept, MWEs covered) and publish it alongside the evaluation with negative controls.
  • I4 (tree): evaluate the perceptron parser and gparse with leave-one-genre-out splits to see which parse errors come from the domain rather than from the grammar.
  • I5 (world): combine WordNet sense counts with corpus frequency in the compiled lexicon to give an explainable difficulty estimate per lemma, useful for flagging hard words in school-course texts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WolverhamptonWolverhampton · United Kingdom30 works · 6,272 citations🎓Rochester Institute of TechnologyRochester · United States10 works · 4,906 citations🎓Manchester Metropolitan UniversityManchester · United Kingdom2 works · 204 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksDevelopmental and Educational Psychology · 260 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Matthew Shardlow, Richard Evans and Marcos Zampieri. 2021. Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset. arXiv:2102.08773 · Lang Resources and Evaluation 56, 1153-1194 (2022).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/s10579-022-09588-2 · arXiv 2102.08773

@misc{shardlow2021predicting,
  title = {{Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset}},
  author = {Matthew Shardlow, Richard Evans and Marcos Zampieri},
  year = {2021},
  doi = {10.1007/s10579-022-09588-2},
  eprint = {2102.08773},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s10579-022-09588-2}
}

References

88 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AbuRa’ed, A., Saggion, H.: LaSTUS/TALN at Complex Word Identification (CWI) 2018 Shared Task. In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  2. Alfter, D., Pilán, I.: SB@GU at the Complex Word Identification 2018 Shared Task. In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  3. Aroyehun, S.T., Angel, J., Alvarez, D.A.P., Gelbukh, A.: Complex Word Identification: Convolutional Neural Network vs. Feature Engineering . In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  4. Bada, M., Eckert, M., Evans, D., Garcia, K., Shipley, K., Sitnikov, D., Baumgartner, W.A., Cohen, K.B., Verspoor, K., Blake, J.A., et al.: Concept annotation in the craft corpus. BMC bioinformatics 13(1), 161 (2012) find
  5. Bingel, J., Schluter, N., Martı́nez Alonso, H.: CoastalCPH at SemEval-2016 Task 11: The importance of designing your Neural Networks right. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1028–1033. Association for Computational Linguistics, San Diego, California (2016) find
  6. Biran, O., Brody, S., Elhadad, N.: Putting it simply: a context-aware approach to lexical simplification. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: shortpapers (ACL-2011), pp. 496–501. Portland, Oregon (2011)
  7. Bott, S., Rello, L., Drndarevic, B., Saggion, H.: Can Spanish be simpler? LexSiS: Lexical Simplification for Spanish. In: Proceedings of the 12th International Conference on Intelligent Text Processing and Computational Linguistics, Lecture Notes in Computer Science, pp. 8–15. Springer, Samos, Greece (2012) find
  8. Brants, T., Franz, A.: The google web 1t 5-gram corpus version 1.1. LDC2006T13 (2006) find
  9. Brooke, J., Uitdenbogerd, A., Baldwin, T.: Melbourne at SemEval 2016 Task 11: Classifying Type-level Word Complexity using Random Forests with Corpus and Word List Features. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 975–981. Association for Computational Linguistics, San Diego, California (2016) find
  10. Brysbaert, M., Buchmeier, M., Conrad, M., Jacobs, A.M., Bölte, J., Böhl, A.: The word frequency effect. Experimental psychology (2011) find
  11. Butnaru, A., Ionescu, R.T.: UnibucKernel: A kernel-based learning method for complex word identification . In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  12. Choubey, P., Pateria, S.: Garuda & Bhasha at SemEval-2016 Task 11: Complex Word Identification Using Aggregated Learning Models. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1006–1010. Association for Computational Linguistics, San Diego, California (2016) find
  13. Christodouloupoulos, C., Steedman, M.: A massively parallel corpus: the bible in 100 languages. Language Resources and Evaluation 49(2), 375–395 (2015). DOI 10.1007/s10579-014-9287-y ↗
  14. Connine, C., Mullennix, J., Shernoff, E., Yelen, J.: Word familiarity and frequency in visual and auditory word recognition. Journal of experimental Psychology: Learning, Memory, and Cognition 16(6), 1084–1096 (1990) find
  15. Davoodi, E., Kosseim, L.: CLaC at SemEval-2016 Task 11: Exploring linguistic and psycho-linguistic Features for Complex Word Identification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 982–985. Association for Computational Linguistics, San Diego, California (2016) find
  16. De Hertog, D., Tack, A.: Deep Learning Architecture for Complex Word Identification . In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  17. Deutsch, T., Jasbi, M., Shieber, S.: Linguistic features for readability assessment. In: Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 1–17. Association for Computational Linguistics, Seattle, WA, USA → Online (2020). DOI 10.18653/v1/2020.bea-1.1 ↗
  18. Devlin, S., Tait, J.: The use of a psycholinguistic database in the simplification of text for aphasic readers. Linguistic Databases pp. 161–173 (1998) find
  19. Dolby, J.L., Resnikoff, H.L., MacMurray, F.L.: A tape dictionary for linguistic experiments. In: Proceedings of the American Federation of information processing societies: Fall Joint Computer Conference, pp. 419 – 423. Spartan Books, Baltimore, MD (1963) find
  20. Dupoux, E., Mehler, J.: Monitoring the lexicon with normal and compressed speech: Frequency effects and the prelexical code. Journal of Memory & Language 29, 316–335 (1990) find
  21. Fellbaum, C.: Wordnet. In: R. Poli, M. Healy, A. Kameas (eds.) Theory and Applications of Ontology: Computer Applications, pp. 231–243. Springer Netherlands (2010) find
  22. Finnimore, P., Fritzsch, E., King, D., Sneyd, A., Ur Rehman, A., Alva-Manchego, F., Vlachos, A.: Strong baselines for complex word identification across multiple languages. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 970–977. Association for Computational Linguistics, Minneapolis, Minnesota (2019) find
  23. Gilhooly, K., Logie, R.: Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words. Behavior Research Methods 12(4), 396–427 (1980) find
  24. Gillin, N.: Sensible at SemEval-2016 Task 11: Neural Nonsense Mangled in Ensemble Mess. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 963–968. Association for Computational Linguistics, San Diego, California (2016) find
  25. Gooding, S., Kochmar, E.: CAMB at CWI Shared Task 2018: Complex Word Identification with Ensemble-Based Voting . In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  26. Gooding, S., Kochmar, E.: Complex word identification as a sequence labelling task. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1148–1153. Association for Computational Linguistics, Florence, Italy (2019) find
  27. Gooding, S., Kochmar, E., Sarkar, A., Blackwell, A.: Comparative judgments are more consistent than binary classification for labelling word complexity. In: Proceedings of the 13th Linguistic Annotation Workshop, pp. 208–214 (2019) find
  28. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11, 10–18 (2009). DOI http://doi.acm.org/10.1145/1656274.1656278 ↗
  29. Hartmann, N., dos Santos, L.B.: NILC at CWI 2018: Exploring Feature Engineering and Feature Learning . In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  30. Horn, C., Manduca, C., Kauchak, D.: Learning a lexical simplifier using Wikipedia. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 458–463. Association for Computational Linguistics, Baltimore, Maryland (2014)
  31. Kajiwara, T., Komachi, M.: Complex Word Identification Based on Frequency in a Learner Corpus . In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  32. Kauchak, D.: Improving text simplification language modeling using unsimplified text data. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1537–1546. Association for Computational Linguistics, Sofia, Bulgaria (2013)
  33. Kauchak, D.: Pomona at SemEval-2016 Task 11: Predicting Word Complexity Based on Corpus Frequency. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1047–1051. Association for Computational Linguistics, San Diego, California (2016) find
  34. Kinoshita, S.: Generation enhances semantic processing? the role of distinctiveness in the generation effect. Memory & Cognition 17, 563–571 (1989) find
  35. Koehn, P.: Europarl: A parallel corpus for statistical machine translation. In: Proceedings of MT Summit (2005) find
  36. Konkol, M.: UWB at SemEval-2016 Task 11: Exploring Features for Complex Word Identification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1038–1041. Association for Computational Linguistics, San Diego, California (2016) find
  37. Kuncheva, L.I., Bezdek, J.C., Duin, R.P.: Decision Templates for Multiple Classifier Fusion: An Experimental Comparison. Pattern Recognition 34(2), 299–314 (2001) find
  38. Kuru, O.: AI-KU at SemEval-2016 Task 11: Word Embeddings and Substring Features for Complex Word Identification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1042–1046. Association for Computational Linguistics, San Diego, California (2016) find
  39. Kučera, H., Francis, W.: Computational analysis of present-day American English. Brown University Press, Providence, RI, USA (1967) find
  40. Leeds, B.G.: Kindergarten children and the influence of letter shapes and meaningfulness of vocabulary as factors influencing word recognition. Technical Report ED136250, Department of Health, Education & Welfare, National Institute of Education (1976) find
  41. Leroy, G., Kauchak, D., Mouradi, O.: A user-study measuring the effects of lexical simplification and coherence enhancement on perceived and actual text difficulty. International journal of medical informatics 82(8), 717–730 (2013) find
  42. Levene, H.: Robust tests for equality of variances. In: I. Olkin (ed.) Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling. Stanford University Press, Palo Alto, CA (1960) find
  43. Maddela, M., Xu, W.: A word-complexity lexicon and a neural readability ranking model for lexical simplification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3749–3760. Association for Computational Linguistics, Brussels, Belgium (2018). DOI 10.18653/v1/D18-1410 ↗
  44. Malmasi, S., Dras, M., Zampieri, M.: LTG at SemEval-2016 Task 11: Complex Word Identification with Classifier Ensembles. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 996–1000. Association for Computational Linguistics, San Diego, California (2016) find
  45. Malmasi, S., Zampieri, M.: MAZA at SemEval-2016 Task 11: Detecting Lexical Complexity Using a Decision Stump Meta-Classifier. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 991–995. Association for Computational Linguistics, San Diego, California (2016) find
  46. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, pp. 55–60 (2014)
  47. Marslen-Wilson, W.: Activation, competition, and frequency in lexical acess. MIT Press, Cambridge, MA (1990) find
  48. Martı́nez Martı́nez, J.M., Tan, L.: USAAR at SemEval-2016 Task 11: Complex Word Identification with Sense Entropy and Sentence Perplexity. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 958–962. Association for Computational Linguistics, San Diego, California (2016) find
  49. Morrison, C., Ellis, A.W.: Real age of acquisition effects in word naming and lexical decision. British Journal of Psychology 91 Pt. 2(2), 167–180 (2000) find
  50. Mukherjee, N., Patra, B.G., Das, D., Bandyopadhyay, S.: JU NLP at SemEval-2016 Task 11: Identifying Complex Words in a Sentence. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 986–990. Association for Computational Linguistics, San Diego, California (2016) find
  51. Paetzold, G., Specia, L.: SemEval 2016 Task 11: Complex Word Identification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 560–569. Association for Computational Linguistics, San Diego, California (2016)
  52. Paetzold, G., Specia, L.: SV000gg at SemEval-2016 Task 11: Heavy Gauge Complex Word Identification with System Voting. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 969–974. Association for Computational Linguistics, San Diego, California (2016) find
  53. Paetzold, G.H.: Lexical simplification for non-native english speakers. Phd thesis, University of Sheffield (2016) find
  54. Paetzold, G.H., Alva-Manchego, F., Specia, L.: MASSAlign: Alignment and Annotation of Comparable Documents. In: The Companion Volume of the IJCNLP 2017 Proceedings: System Demonstrations, pp. 1–4 (2017) find
  55. Paetzold, G.H., Specia, L.: A survey on lexical simplification. Journal of Artificial Intelligence Research 60, 549–593 (2017) find
  56. Paivio, A.: Dual coding theory: Retrospect and current status. Canadian Journal of Psychology/Revue canadienne de psychologie 45(3), 255–287 (1991) find
  57. Paivio, A., Yuille, J.C., Madigan, S.A.: Concreteness, imagery, and meaningfulness values for 925 nouns. Journal of Experimental Psychology 76(1, Pt.2), 1–25 (1968) find
  58. Palakurthi, A., Mamidi, R.: IIIT at SemEval-2016 Task 11: Complex Word Identification using Nearest Centroid Classification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1017–1021. Association for Computational Linguistics, San Diego, California (2016) find
  59. Peirce, C.S.S.: Prolegomena to an apology for pragmaticism. The Monist 6, 492–546 (1906) find
  60. Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Proceedings of EMNLP (2014)
  61. Popović, M.: Complex Word Identification using Character n-grams. In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  62. Quijada, M., Medero, J.: HMC at SemEval-2016 Task 11: Identifying Complex Words Using Depth-limited Decision Trees. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1034–1037. Association for Computational Linguistics, San Diego, California (2016) find
  63. Ronzano, F., Abura’ed, A., Espinosa Anke, L., Saggion, H.: TALN at SemEval-2016 Task 11: Modelling Complex Words by Contextual, Lexical and Semantic Features. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1011–1016. Association for Computational Linguistics, San Diego, California (2016) find
  64. Sag, I.A., Baldwin, T., Bond, F., Copestake, A.A., Flickinger, D.: Multiword expressions: A pain in the neck for nlp. In: Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing, CICLing ’02, p. 1–15. Springer-Verlag, Berlin, Heidelberg (2002)
  65. Schneider, N., Onuffer, S., Kazour, N., Danchik, E., Mordowanec, M.T., Conrad, H., Smith, N.A.: Comprehensive annotation of multiword expressions in a social web corpus. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pp. 455–461. European Language Resources Association (ELRA), Reykjavik, Iceland (2014)
  66. Schwanenflugel, P.J.: Why are abstract concepts hard to understand? In: P.J. Schwanenflugel (ed.) The psychology of word meaning, pp. 223–250. Erlbaum, Mahwah, NJ (1991) find
  67. Schwanenflugel, P.J., Harnishfeger, K.K., Stowe, R.W.: Context availability and lexical decisions for abstract and concrete words. Journal of Memory and Language 27(5), 499–520 (1988) find
  68. Segui, J., Mehler, J., Frauenfelder, U., Morton, J.: The word frequency effect and lexical access. Neuropsychologia 20(6), 615–627 (1982) find
  69. SHAPIRO, S.S., WILK, M.B.: An analysis of variance test for normality (complete samples)†. Biometrika 52(3-4), 591–611 (1965). DOI 10.1093/biomet/52.3-4.591 ↗
  70. Shardlow, M.: A comparison of techniques to automatically identify complex words. In: 51st Annual Meeting of the Association for Computational Linguistics Proceedings of the Student Research Workshop, pp. 103–109. Association for Computational Linguistics, Sofia, Bulgaria (2013)
  71. Shardlow, M.: The CW corpus: A new resource for evaluating the identification of complex words. In: the Second Workshop on Predicting and Improving Text Readability for Target Reader Populations (PITR 2013). Association for Computational Linguistics, Sofia, Bulgaria (2013) find
  72. Shardlow, M., Cooper, M., Zampieri, M.: CompLex — a new corpus for lexical complexity prediction from Likert Scale data. In: Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI), pp. 57–62. European Language Resources Association, Marseille, France (2020) find
  73. Shardlow, M., Evans, R., Paetzold, G., Zampieri, M.: Semeval-2021 task 1: Lexical complexity prediction. In: Proceedings of the 14th International Workshop on Semantic Evaluation (SemEval-2021) (2021)
  74. S.P, S., Kumar M, A., K P, S.: AmritaCEN at SemEval-2016 Task 11: Complex Word Identification using Word Embedding. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1022–1027. Association for Computational Linguistics, San Diego, California (2016) find
  75. Steacy, L.M., Compton, D.L.: Examining the role of imageability and regularity in word reading accuracy and learning efficiency among first and second graders at risk for reading disabilities. Journal of Experimental Child Psychology 178, 226–250 (2019) find
  76. Strohmaier, D., Gooding, S., Taslimipoor, S., Kochmar, E.: SeCoDa: Sense complexity dataset. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 5962–5967. European Language Resources Association, Marseille, France (2020). URL https://aclanthology.org/2020.lrec-1.730 ↗
  77. Svartvik, J., Quirk, R. (eds.): Handbook of semantic word norms. Liver/Gleerups, Lund (1980) find
  78. Thorndike, E.L., Lorge, I.: The teacher’s word book of 30,000 words. Teacher’s College, Columbia University, New York, NY, USA (1944) find
  79. Toglia, M.P., Battig, W.F.: Handbook of semantic word norms. Erlbaum, Hillsdale, NJ, USA (1978) find
  80. Wani, N., Mathias, S., Gajjam, J.A., Bhattacharyya, P.: The Whole is Greater than the Sum of its Parts: Towards the Effectiveness of Voting Ensemble Classifiers for Complex Word Identification. In: Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, New Orleans, United States (2018) find
  81. Wilson, M.: MRC psycholinguistic database: Machine-usable dictionary, version 2.00. Behavior Research Methods, Instruments, & Computers 20, 6–10 (1988) find
  82. Wróbel, K.: PLUJAGH at SemEval-2016 Task 11: Simple System for Complex Word Identification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 953–957. Association for Computational Linguistics, San Diego, California (2016) find
  83. Yaneva, V., Orăsan, C., Evans, R., Rohanian, O.: Combining multiple corpora for readability assessment for people with cognitive disabilities. In: Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 121– 132. Association for Computational Linguistics, Copenhagen, Denmark (2017). DOI 10.18653/v1/W17-5013 ↗
  84. Yimam, S.M., Biemann, C., Malmasi, S., Paetzold, G., Specia, L., Štajner, S., Tack, A., Zampieri, M.: A report on the complex word identification shared task 2018. In: Proceedings of BEA (2018) find
  85. Yimam, S.M., Štajner, S., Riedl, M., Biemann, C.: Multilingual and Cross-Lingual Complex Word Identification. In: Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pp. 813–822. INCOMA Ltd., Varna, Bulgaria (2017) find
  86. Yonelinas, A.P., Otten, L.J., Shaw, K.N., Rugg, M.D.: Separating the brain regions involved in recollection and familiarity in recognition memory. Journal of Neuroscience 25(11), 3002–3008 (2005) find
  87. Zampieri, M., Malmasi, S., Paetzold, G., Specia, L.: Complex Word Identification: Challenges in Data Annotation and System Performance. In: Proceedings of the 4th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA 2017), pp. 59–63. Asian Federation of Natural Language Processing, Taipei, Taiwan (2017) find
  88. Zampieri, M., Tan, L., van Genabith, J.: MacSaar at SemEval-2016 Task 11: Zipfian and Character Features for ComplexWord Identification. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1001–1005. Association for Computational Linguistics, San Diego, California (2016) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shardlow-2021-predicting-lexical-complexity-english-texts/