Elamur Research⌕
☰
home / papers / 2026 / A Human-in/on-the-Loop Framework for Accessible Text Generation

A Human-in/on-the-Loop Framework for Accessible Text Generation

Summary

Automatic simplification into Plain Language (PL) and Easy-to-Read (ER) formats is usually judged by metrics such as BLEU, SARI, SAMSA or BERTScore. These metrics reveal little about whether real readers, including people with intellectual disabilities, actually understand the output, or whether it complies with accessibility norms. The authors propose a conceptual framework that builds human involvement into LLM-based accessible text generation at two stages. Human-in-the-Loop (HiTL) input steers the model during generation, for example by accepting or rejecting proposed lexical substitutions. Human-on-the-Loop (HoTL) experts then review the outputs. Drawing on the group's earlier user studies and annotated lexical-simplification corpora, the authors turn this evidence into three tools: checklists aligned with standards, Event–Condition–Action rules that decide when expert review is required, and accessibility KPIs. Reviewer feedback is then used to adapt the model. The contribution is a process and design resource, not a new benchmark score. Its value lies in making accessible text production traceable, auditable and ethically accountable, which is especially important for health, legal and public-administration communication.

Key points

  • The workflow has five stages: HiTL guidance, automatic metric-based checks, ECA escalation triggers, HoTL expert review, and adaptation driven by structured feedback.
  • The framework is anchored in ISO 24495-1:2023 (PL) and the Spanish UNE 153101:2018 standard on Lectura Fácil (ER). It also references EN 301 549, WCAG and Directive (EU) 2016/2102.
  • The released artefacts (checklist schema, ECA trigger templates, KPI definitions) are available on Zenodo (HULAT Group, 2026).
  • An appendix works through a Spanish example about hunting and fishing rules in the Community of Madrid.
    • The first adaptation swapped the formal "trámites" for the colloquial "papeleos" and dropped the licence requirement.
    • After HoTL review, the revision restored both the administrative procedures and the licence.
    • The authors present this as a demonstration of the workflow, not as an evaluation result.
  • The available text reports no quantitative evaluation of the framework itself; its contribution is methodological.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2gtngp2nmx63

@inproceedings{moreno2026human,
  title = {{A Human-in/on-the-Loop Framework for Accessible Text Generation}},
  author = {Moreno, Lourdes and Martínez, Paloma},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2gtngp2nmx63},
  url = {https://doi.org/10.63317/2gtngp2nmx63}
}

References

44 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rodrigo Alarcon, Lourdes Moreno, and Paloma Martínez. 2023. Easier corpus: A lexical simplification resource for people with cognitive impairments. PLOS ONE, 18(4):e0283622. find
  2. Rodrigo Alarcon, Lourdes Moreno, Paloma Martínez, and José A. Macías. 2024. Easier system. evaluating a spanish lexical simplification proposal with people with cognitive impairments. International Journal of Human–Computer Interaction, 40(5):1195–1209. find
  3. Fernando Alva-Manchego, Carolina Scarton, and Lucia Specia. 2021. The (un)suitability of automatic evaluation metrics for text simplification. Computational Linguistics, 47(4):861–889. find our card
  4. Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI Magazine, 35(4):105–120. find
  5. Florian Böhm, Yang Gao, Christian M. Meyer, Ori Shapira, Ido Dagan, and Iryna Gurevych. 2019. Better rewards yield better summaries: Learning to summarise without references. arXiv preprint arXiv:1909.01214. find
  6. Luisa Carrer, Tatiana Anikina, Mounica Maddela, Fernando Alva-Manchego, and Carolina Scarton. 2024. Towards holistic human evaluation of automatic text simplification. In Proceedings of the 1st Workshop on Human Evaluation of NLP Systems (HumEval), pages 66–79. find
  7. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? arXiv preprint arXiv:2305.01937. find
  8. ETSI. 2021. En 301 549: Accessibility requirements for ict products and services. https://www.etsi.org/deliver/etsi_en/301500_301599/301549. ↗
  9. European Commission. 2019. Ethics guidelines for trustworthy ai. https://digital-strategy.ec.europa.eu/en/library/ ethics-guidelines-trustworthy-ai. ↗
  10. Y. Gao, K. Johnson, D. Froehlich, L. Carrer, and S. Ebling. 2025. Evaluating the effectiveness of direct preference optimization for personalizing german automatic text simplifications for persons with intellectual disabilities. arXiv preprint arXiv:2507.01479. find
  11. T. Goldsack, Z. Zhang, C. Tang, C. Scarton, and C. Lin. 2023. Enhancing biomedical lay summarisation with external knowledge graphs. arXiv preprint arXiv:2310.15702. find
  12. A. Hayakawa, S. Bott, and H. Saggion. 2025. Towards trustworthy lexical simplification: Exploring safety and efficiency with small llms. arXiv preprint arXiv:2509.25086. find
  13. HULAT Group. 2026. Artefacts for lrec 2026: A human-in/on-the-loop framework for accessible text generation. Zenodo record. find
  14. Paloma Martínez, Alberto Ramos, and Lourdes Moreno. 2024. Exploring large language models to generate easy to read content. Frontiers in IFLA. 2010. Directives for easy-to-read materials. Computer Science, 6:1394705. https://www.ifla.org/publications/directory-of-easy-to-read-publications/. ↗
  15. K. Mo and R. Hu. 2024. Expertease: A multi-agent framework for grade-specific document simpli-B. find
  16. İlgen and G. Hattab. 2025. Toward human-fication with large language models. Findings centered readability evaluation. arXiv preprint of the Association for Computational Linguistics: arXiv:2510.10801. — [arXiv:2510.10801](https://arxiv.org/abs/2510.10801) find
  17. Inclusion Europe. 2025. European standards Lourdes Moreno, Helen Petrie, Paloma Martínez, and guidelines for easy-to-read information. and Rodrigo Alarcon. 2024. Designing user inter-https://www.inclusion-europe.eu/faces for content simplification aimed at people easy-to-read-standards-guidelines/.… find
  18. ISO. 2023. Iso 24495-1:2023 plain language—part 1: Governing principles and guidelines. https: Lourdes Moreno, Jesus M. Sanchez-Gomez, //www.iso.org/standard/78907.html. find
  19. Marco Antonio Sanchez-Escudero, and Paloma Martínez. 2025. Hulat-uc3m @ clears2025 sub-Jonathan Lazar, Jinjuan Heidi Feng, and Harry task 1: Prompt-based simplification for plain lan-Hochheiser. 2017. Research Methods in Human-guage using spanish language models. In Pro-Computer Interaction. Morgan… find
  20. CEUR-WS.org, ISSN S. Moravac, A. Altinier, et al. 2025. Inclusive easy-1613-0073. to-read generation for individuals with cognitive Sergiu Nisioi, Sanja Štajner, Simone Paolo impairments. arXiv preprint arXiv:2510.00691. — [arXiv:2510.00691](https://arxiv.org/abs/2510.00691) find
  21. Ponzetto, and Liviu P. Dinu. 2017. Exploring Nianyu Li, Adebola Adeyiga, and Rami Bahsoon. neural text simplification models. In Proceedings 2020. Explanations for human-on-the-loop: A of the 55th Annual Meeting of the Association probabilistic model checking approach. In Pro-for Computational Lingu… find
  22. Symposium on Software Engineering for Adaptive and Self-Managing Systems, pages 181– 187. find
  23. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81. find
  24. M. Maddela, Y. Dou, D. Heineman, and W. Xu. 2022. Lens: A learnable evaluation metric for text simplification. arXiv preprint arXiv:2212.09739. find our card
  25. Mounica Maddela, Fernando Alva-Manchego, and Wei Xu. 2021. Controllable text simplification with explicit paraphrasing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3536–3553, Online. Associat… find our card
  26. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computation… find
  27. OECD. 2024. Skills Outlook 2024: Skills for Inclusive Transitions. OECD Publishing. find
  28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318. find our card
  29. PLAIN. 2021. Plainlanguage.gov. https://www.plainlanguage.gov. ↗
  30. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290. find
  31. Matthew Shardlow, Fernando Alva-Manchego, Riza Theresa Batista-Navarro, Stefan Bott, et al. 2024. An extensible massively multilingual lexical simplification pipeline dataset using the multils framework. In Proceedings of the 3rd Workshop on Tools and Resources for People with REAding DIfficulties (… find
  32. Ben Shneiderman. 2020. Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction, 36(6):495–504. find
  33. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems, volume 33, pages 3008–3021. find
  34. Elior Sulem, Omri Abend, and Ari Rappoport. 2018. Semantic structural evaluation for text simplification. arXiv preprint arXiv:1810.05022. find our card
  35. UNE. 2018. Une 153101:2018 ex. lectura fácil. pautas y recomendaciones para la elaboración de documentos. https://www.une.org/encuentra-tu-norma/busca-tu-norma/norma?c=N0060036. ↗
  36. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. find
  37. W3C. 2025. Wcag 2 overview. https://www.w3.org/WAI/standards-guidelines/wcag/. ↗
  38. W3C WAI. 2021. Making content usable for people with cognitive and learning disabilities (coga). https://www.w3.org/TR/coga-usable/. ↗
  39. W3C WAI. 2023. Web content accessibility guidelines (wcag) 2.2. https://www.w3.org/TR/WCAG22/. ↗
  40. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401–415. find our card
  41. Fabio Massimo Zanzotto. 2019. Human-in-the-loop artificial intelligence. Journal of Artificial Intelligence Research, 64:243–252. find
  42. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019. BERTScore: Evaluating text generation with BERT. arXiv preprint arXiv:1904.09675. find
  43. W. Zhao, M. Peyrard, F. Liu, Y. Gao, C. M. Meyer, and S. Eger. 2019. Moverscore: Text generation evaluating with contextualized embeddings and earth mover distance. arXiv preprint arXiv:1909.02622. find our card
  44. Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593. 10. Language Resource References A first set of reusable artefacts associated with… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/moreno-2026-human-loop-framework-accessible-text/