Elamur Research⌕
☰
home / papers / 2026 / LLM-Generated Stories for Students with Significant Cognitive Disabilities: Promise, Gaps, and Evaluation Framework

LLM-Generated Stories for Students with Significant Cognitive Disabilities: Promise, Gaps, and Evaluation Framework

Paper · 2026 · Pragati Maheshwary, Ananya Ganesh and Shamya Karumbaiah

Summary

Students with significant cognitive disabilities (SCD) take reading comprehension tests in alternate assessments such as Dynamic Learning Maps (DLM). These tests rely on short stories with simple vocabulary, brief sentences and a single clear theme. Experts write them through a slow, multi-stage review process that is hard to scale. The authors test whether GPT-4o can produce first drafts of such stories.

They took 8 retired, publicly available DLM Grade 11–12 ELA stories based on three classic books. Using 8-fold cross-validation with one-shot prompting, where one human story serves as the exemplar, they generated 120 stories. These were scored against the held-out human stories with a new reference-free framework covering three criteria: simplicity, fluency & coherence, and thematic adherence.

The generated texts usually reached the target simplicity. However, they connected sentences much less well than the human texts, and they tended to reuse similar themes across stories.

The authors conclude that LLMs can speed up the drafting stage. Expert revision is still required before such texts are used in high-stakes assessments.

Key points

  • Data: 8 human-written DLM stories adapted from three classics: A White Heron, My Antonia and The Great Gatsby. The abstract describes the test baseline as 7 expert stories, which is consistent with one story per fold serving as the exemplar.
  • Simplicity: About two-thirds of the 120 GPT-4o stories had readability scores, including Flesch-Kincaid Grade Level (FKGL), inside the human range.
  • Coherence: Only 5% of generated stories matched the human range for adjacent sentence similarity. This gap appeared in every fold.
  • Theme: Thematic adherence was moderate and diversity across stories was acceptable, though themes sometimes repeated.
  • Limitations:
    • Only one model and one-shot prompting were tested.
    • DLM staff did not review the framework or the stories.
    • The framework does not yet check testability, bias/sensitivity or people-first language.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Wisconsin–MadisonMadison · United States42 works · 48,486 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Pragati Maheshwary, Ananya Ganesh and Shamya Karumbaiah. 2026. LLM-Generated Stories for Students with Significant Cognitive Disabilities: Promise, Gaps, and Evaluation Framework. Proceedings of the Joint Workshop on Readability and Text Simplification (READIxTSAR) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3b9f4txwp2n2

@inproceedings{maheshwary2026llm,
  title = {{LLM-Generated Stories for Students with Significant Cognitive Disabilities: Promise, Gaps, and Evaluation Framework}},
  author = {Pragati Maheshwary, Ananya Ganesh and Shamya Karumbaiah},
  booktitle = {Proceedings of the Joint Workshop on Readability and Text Simplification (READIxTSAR) @ LREC 2026},
  year = {2026},
  doi = {10.63317/3b9f4txwp2n2},
  url = {https://doi.org/10.63317/3b9f4txwp2n2}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, and Lucia Specia. 2020. Asset: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations. In Proceedings of the 58th Annual Meeting of the Association for Computa…
  2. Catarina G. Belem, Parker Glenn, Alfy Samuel, Anoop Kumar, and Daben Liu. 2025. Readability reconsidered: A cross-dataset analysis of reference-free metrics. arXiv preprint arXiv:2510.15345. find
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901.
  4. Meagan Karvonen, A. K. Clark, C. Carlson, S. Wells Moreaux, and J. Burnes. 2021. Approaches to identification and instruction for students with significant cognitive disabilities who are english learners. Research and Practice for Persons with Severe Disabilities, 46(4):223–239. find
  5. J. Peter Kincaid, Robert P. Fishburne, Richard L. Rogers, and Brad S. Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Technical report, Naval Technical Training Command, Millington, TN, Resear… find
  6. Anthony Laverghetta Jr. and John Licato. 2023. Generating better items for cognitive assessments using large language models. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 414–428. Association for Computational Linguistics. find
  7. Elad Chamovitz and Omri Abend. 2022. Cognitive simplification operations improve text simplification. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 241–265. find
  8. Louis Martin, Angela Fan, Éric de la Clergerie, Antoine Bordes, and Benoît Sagot. 2022. MUSS: Multilingual unsupervised sentence simplification by mining paraphrases. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1651–1664, Marseille, France. European Language…
  9. A. K. Clark, A. Hirt, D. Whitcomb, W. J. Thompson, M. Wine, and M. Karvonen. 2025. Artificial intelligence in science and mathematics assessment for students with disabilities: Opportunities and challenges. Education Sciences, 15(2):233. find
  10. Brooke Nash, A. K. Clark, and Meagan Karvonen. 2016. First contact: A census of students taking the dynamic learning maps alternate assessment. Technical report, University of Kansas, Center for Educational Testing and Evaluation. find
  11. DLM Consortium. 2024. 2023–2024 technical manual—dlm alternate assessment system. Technical report, University of Kansas, Accessible Teaching, Learning, and Assessment Systems (ATLAS). find
  12. Karen Erickson and Lori A. Geist. 2016. The profiles of students with significant cognitive disabilities and complex communication needs. Augmentative and Alternative Communication, 32(3):187–197. find
  13. Yi Feng, Mingyang Song, Jiaqi Wang, Zhuang Chen, Guanqun Bi, Minlie Huang, Liping Jing, and Jian Yu. 2025. Ss-gen: a social story generation framework with large language models. AAAI’25/IAAI’25/EAAI’25. AAAI Press. find
  14. Meagan Karvonen and A. K. Clark. 2019. Students with the most significant cognitive disabilities who are also english learners. Research and Practice for Persons with Severe Disabilities, 44(2):71–86. find
  15. OpenAI. 2023. Gpt-4 technical report.
  16. G. Raffloer and Melanie C. Green. 2025. Of love & lasers: Perceptions of narratives by ai versus human authors. Computers in Human Behavior: Artificial Humans, 5:100168. find
  17. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.…
  18. B. Tan, N. Armoush, E. Mazzullo, O. Bulut, and M. Gierl. 2025. A review of automatic item generation techniques leveraging large language models. International Journal of Assessment Tools in Education, 12(2):317–340. find
  19. Martha L. Thurlow, S. S. Lazarus, E. D. Larson, D. A. Albus, K. K. Liu, and E. Kwong. 2017. find
  20. Alternate assessments for students with significant cognitive disabilities: Participation guidelines and definitions. Technical report, University of Minnesota, National Center on Educational Outcomes. User Prompt EXEMPLAR={exemplar_json} Now generate {n} independent stories inspired by “{target_boo… find
  21. Martha L. Thurlow, Y. Wu, Rachel F. Quenemoen, and E. Towles. 2016. Characteristics of students with significant cognitive disabilities. Technical report, National Center and State Collaborative. find
  22. Laura Vásquez-Rodríguez, Matthew Shardlow, Piotr Przybyła, and Sophia Ananiadou. 2023. Document-level text simplification with coherence evaluation. In Proceedings of the Second Workshop on Text Simplification, Accessibility and Readability, pages 85–101, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulga… find
  23. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401–415.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maheshwary-2026-llm-generated-stories-students-significant/