Elamur Research⌕
☰
home / papers / 2025 / Modélisation de la lisibilité en français pour les personnes en situation d'illettrisme

Modélisation de la lisibilité en français pour les personnes en situation d'illettrisme

Summary

The paper fills a gap for French: until now, there has been no readability formula for adults with functional illiteracy (illettrisme). These are people who attended school but cannot confidently read and understand everyday texts. The authors from UCLouvain compiled a corpus of 461 short texts of 11 communicative types. Most of the texts were provided by trainers who work with this audience, and the rest were taken from the web. The texts were annotated using a specially designed four-level difficulty scale. The authors then systematically compared the main approaches: classical machine learning on linguistic features, fine-tuning CamemBERT, a hybrid of CamemBERT with linguistic variables, and generative LLMs in zero-shot and few-shot settings. The proposed formula is built on the hybrid architecture. The authors also examine whether model uncertainty is related to disagreements among annotators. The main performance metrics are absent from the provided excerpt because the middle of the article was omitted. The value of the work is that it provides the first open resource and model for this audience in French. They help select texts for reading instruction programs, and the approach itself can be transferred to other languages that lack such data.

Key points

  • Corpus: 461 texts ranging from 25 to 608 tokens in length. It covers 11 communication types: personal, professional, commercial, legal and religious texts, social media posts, fiction, non-fiction and didactic books, etc. The corpus is published on GitHub (iread4skills_readability_corpus_fr).
  • Scale: four levels: «Très Facile», «Facile», «Accessible», «+Complexe». They are tied to length of schooling: roughly up to the 6th year, up to the 9th, and beyond the 9th. After the first round, a 5-point Likert scale was added within each level. The result is a dual annotation: ordinal with 4 levels and discrete with 20 values.
  • Annotation: the texts were annotated by 15 paid professionals in the field of illettrisme, mostly trainers. The work was carried out in Qualtrics following an agreed guideline, and training sessions were held for the annotators.
  • Models: SVM, DT and RF on linguistic features, CamemBERT-v2, and a hybrid (CamemBERT + features, RF). Generative LLMs received a prompt asking them to classify the text according to CEFR (A1–C2). In the few-shot setting, they were given 4 examples, one per level, with an explanation.
  • Uncertainty and annotator disagreement: the entropy of model outputs was compared with the variance of annotators' ratings (via Krippendorff decomposition). Weak but significant correlations were obtained for SVM (Spearman −0.196), RF (−0.147) and the hybrid (−0.137). For CamemBERT-v2 and DT, the correlations are not significant.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{aissa2025modelisation,
  title = {{Modélisation de la lisibilité en français pour les personnes en situation d'illettrisme}},
  author = {Aissa, Wafa and Bañeras-Roux, Thibault and Vanzeveren, Elodie and Gao, Lingyun and Pintard, Alice and Wilkens, Rodrigo and François, Thomas},
  booktitle = {Actes des 32ème Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : articles scientifiques originaux},
  year = {2025},
  url = {https://aclanthology.org/2025.jeptalnrecital-taln.34/}
}

References

47 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A LUISIO S., S PECIA L., G ASPERIN C. & S CARTON C. (2010). Readability assessment for text simplification. In Fifth Workshop on Innovative Use of NLP for Building Educational Applications, p. 1–9, Los Angeles. find our card
  2. A NTOUN W., K ULUMBA F., T OUCHENT R., DE LA C LERGERIE É., S AGOT B. & S EDDAH D. (2024). Camembert 2.0 : A smarter french language model aged to perfection. arXiv preprint arXiv :2411.08868. — [arXiv:2411.08868](https://arxiv.org/abs/2411.08868) find
  3. A RBEL Y. A. (2024). The readability of contracts : Big data analysis. Journal of Empirical Legal Studies, 21(4), 927–978. find
  4. A RTSTEIN R. & P OESIO M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4), 555–596. DOI : 10.1162/coli.07-034-R2. find
  5. A ZPIAZU I. M. & P ERA M. S. (2019). Multiattentive recurrent neural network architecture for multilingual readability assessment. Transactions of the Association for Computational Linguistics, 7, 421–436. find
  6. B ERKMAN N. D., S HERIDAN S. L., D ONAHUE K. E., H ALPERN D. J. & C ROTTY K. (2011). Low health literacy and health outcomes : an updated systematic review. Annals of internal medicine, 155(2), 97–107. find
  7. B ESSE J.-M., L UIS M.-H., B OUCHUT A.-L. & M ARTINEZ F. (2009). La mesure des compétences en traitement de l’écrit chez des adultes en grande difficulté. Économie et statistique, 424(1), 31–48. find
  8. B LANDIN A., L ECORVÉ G., B ATTISTELLI D. & E TIENNE A. (2020). Recommandation d’âge pour des textes (age recommendation for texts). In Actes de la 6e conférence conjointe Journées d’Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre… find
  9. C HA M., G WON Y. & K UNG H. (2017). Language modeling by clustering with word embeddings for text readability assessment. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, p. 2003–2006 : ACM. C OLLINS -T HOMPSON K. (2014). Computational assessment of text readabi… find
  10. C ROSSLEY S., H EINTZ A., C HOI J. S., B ATCHELOR J., K ARIMI M. & M ALATINSZKY A. (2023). A large-scaled corpus for assessing text readability. Behavior Research Methods, 55(2), 491–507. find
  11. D ALE E. & C HALL J. (1948). A formula for predicting readability. Educational research bulletin, 27(1), 11–28. find
  12. D ALE E. & T YLER R. (1934). A study of the factors influencing the difficulty of reading materials for adults of limited reading ability. The Library Quarterly, 4, 384–412. find
  13. D ASCALU M. (2014). Readerbench (2)-individual assessment through reading strategies and textual complexity. In Analyzing Discourse and Text Complexity for Learning and Collaborating, p. 161–188. Springer. D ELL ’O RLETTA F., M ONTEMAGNI S. & V ENTURI G. (2011). Read-it : Assessing readability of it… find
  14. D EUTSCH T., J ASBI M. & S HIEBER S. (2020). Linguistic features for readability assessment. arXiv preprint arXiv :2006.00377. find
  15. D ING C. & P ENG H. (2003). Minimum redundancy feature selection from microarray gene expression data. In Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003, p. 523–528. DOI : 10.1109/CSB.2003.1227396. — [doi:10.1109/csb.2003.1227396](https://doi.org/10.1109/csb.2003.1227396) find
  16. D ITTRICH S., W EISS Z., S CHRÖTER H. & M EURERS D. (2019). Integrating large-scale web data and curated corpus data in a search engine supporting german literacy education. In Proceedings of the 8th workshop on NLP for computer assisted language learning, p. 41–56. find
  17. D U B AY W. H. (2004). The principles of readability. Online submission. find
  18. F ENG L., J ANSCHE M., H UENERFAUTH M. & E LHADAD N. (2010). A Comparison of Features for Automatic Readability Assessment. In COLING 2010 : Poster Volume, p. 276–284. find
  19. F ILIGHERA A., S TEUER T. & R ENSING C. (2019). Automatic text difficulty estimation using embeddings and neural networks. In European Conference on Technology Enhanced Learning, p. 335–348 : Springer. find
  20. F LESCH R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221–233. find
  21. F RANÇOIS T. (2011). Les apports du traitement automatique du langage à la lisibilité du français langue étrangère. Thèse de doctorat, Université Catholique de Louvain. Thesis Supervisors : Cédrick Fairon and Anne Catherine Simon. find
  22. F RANÇOIS T. & F AIRON C. (2012). An “AI readability” formula for French as a foreign language. In J. T SUJII , J. H ENDERSON & M. P A ŞCA , Éds., Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, p. 466–477, Je… find
  23. G RAY W. & L EARY B. (1935). What makes a book readable. Chicago : Illinois : University of Chicago Press. find
  24. G UNNING R. (1952). The technique of clear writing. New York : McGraw-Hill. find
  25. H ERNANDEZ N., O ULBAZ N. & F AINE T. (2022). Open corpora and toolkit for assessing text readability in french. In Proceedings of the 2nd Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI) within the 13th Language Resources and Evaluation Conference, p. 54–61. find
  26. J AMET H., M ANDERLIER M., S HRESTHA Y. R. & V LACHOS M. (2024). Evaluation and simplification of text difficulty using llms in the context of recommending texts in french to facilitate language learning. In Proceedings of the 18th ACM Conference on Recommender Systems, p. 987–992. find
  27. K RAMER O. & K RAMER O. (2016). Scikit-learn. Machine learning for evolution strategies, p. 45–53. find
  28. L AL B. S. (2015). The economic and social cost of illiteracy : an overview. International Journal of Advance Research and Innovative Ideas in Education, 1(5), 663–670. find
  29. L IU F. & L EE J. S. (2023). Hybrid models for sentence readability assessment. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), p. 448–454. find
  30. M ARTIN L., M ULLER B., S UÁREZ P. J. O., D UPONT Y., R OMARY L., DE L A C LERGERIE É. V., S EDDAH D. & S AGOT B. (2020). Camembert : a tasty french language model. In ACL 2020-58th Annual Meeting of the Association for Computational Linguistics. M ARTINC M., P OLLAK S. & R OBNIK -Š IKONJA M. (2021)… find our card
  31. M CINNES N. & H AGLUND B. J. (2011). Readability of online health information : implications for health literacy. Informatics for health and social care, 36(4), 173–189. find
  32. M ESSIAS E. (2003). Income inequality, illiteracy rate, and life expectancy in brazil. American Journal of Public Health, 93(8), 1294–1296. find
  33. N ADEEM F. & O STENDORF M. (2018). Estimating linguistic complexity for science texts. In Proceedings of the thirteenth workshop on innovative use of NLP for building educational applications, p. 45–55. find
  34. Q IN Q., H U W. & L IU B. (2020). Feature projection for improved text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, p. 8161–8171. find
  35. R IEKMANN W. & G ROTLÜSCHEN A. (2011). Konservative entscheidungen : Größenordnung des funktionalen analphabetismus in deutschland. REPORT-Zeitschrift für Weiterbildungsforschung, (3), 24–35. R USSELER J., M ENKHAUS K., A ULBERT -S IEPELMEYER A., G ERTH I. & B OLTZMANN M. (2012). " alpha plus" : An… find
  36. S CHWARM S. & O STENDORF M. (2005). Reading level assessment using support vector machines and statistical language models. Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p. 523–530. find
  37. S HANNON C. E. (2001). A mathematical theory of communication. ACM SIGMOBILE mobile computing and communications review, 5(1), 3–55. find
  38. S PECIA L., J AUHAR S. K. & M IHALCEA R. (2012). Semeval-2012 task 1 : English lexical simplification. In * SEM 2012 : The First Joint Conference on Lexical and Computational Semantics– Volume 1 : Proceedings of the main conference and the shared task, and Volume 2 : Proceedings of the Sixth Interna… find our card
  39. S TEWARD J. (2023). The economic & social cost of illiteracy : A snapshot of illiteracy in a global context. find
  40. T ACK A., F RANÇOIS T., R OEKHAUT S. & F AIRON C. (2017). Human and automated CEFR-based grading of short answers. In J. T ETREAULT , J. B URSTEIN , C. L EACOCK & H. Y ANNAKOUDAKIS , Éds., Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, p. 169–179, Co… find
  41. V AJJALA S. & M EURERS D. (2012). On improving the accuracy of readability classification using insights from second language acquisition. In Proceedings of the seventh workshop on building educational applications using NLP, p. 163–173. find
  42. V AN N GO D. & P ARMENTIER Y. (2023). Towards sentence-level text readability assessment for french. In Second Workshop on Text Simplification, Accessibility and Readability (TSAR@ RANLP2023). find
  43. W EISS Z., D ITTRICH S. & M EURERS D. (2018). A linguistically-informed search engine to identifiy reading material for functional illiteracy classes. In Proceedings of the 7th workshop on NLP for Computer Assisted Language Learning, p. 79–90. find
  44. W ILKENS R., A LFTER D., W ANG X., P INTARD A., T ACK A., Y ANCEY K. P. & F RANÇOIS T. (2022). Fabra : French aggregator-based readability assessment toolkit. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, p. 1217–1233. find
  45. W ILKENS R., W ATRIN P., C ARDON R., P INTARD A., G RIBOMONT I. & F RANÇOIS T. (2024). Exploring hybrid approaches to readability : experiments on the complementarity between linguistic features and transformers. In Y. G RAHAM & M. P URVER , Éds., Findings of the Association for Computational Lingui… find
  46. W ILSON M. (2009). Readability and patient education materials used for low-income populations. Clinical Nurse Specialist, 23(1), 33–40. find
  47. Y ANCEY K., P INTARD A. & F RANÇOIS T. (2021). Investigating readability of french as a foreign language with deep learning and cognitive and pedagogical features. Lingue e Linguaggio, 2021(2), 229–258. DOI : 10.1418/102814. — [doi:10.1418/102814](https://doi.org/10.1418/102814) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/aissa-2025-modelisation-de-la-lisibilite-en/