Elamur Research⌕
☰
home / papers / 2026 / Zero Shot Phonics: Evaluating Constraint-Adherent Phonics Story Generation in Large Language Models

Zero Shot Phonics: Evaluating Constraint-Adherent Phonics Story Generation in Large Language Models

Paper · 2026 · Maria Monica Manlises and Ethel Ong

Summary

Phonics stories help early readers learn to decode. They must repeat target grapheme-phoneme (GP) correspondences while staying short, simple and suitable for young children. This study tests whether large language models can meet all of these requirements at once without examples or fine-tuning. The authors built 16 prompt configurations, grouped into three prompt sets, and ran them zero-shot on six LLMs. This produced 8,688 outputs containing 39,096 short stories. Target GP pairs came from Pearson's Phonics Progression Chart. Each story was scored with a 12-metric framework covering four areas: phonological alignment, developmentally appropriate vocabulary, readability and narrative quality. The models reliably wrote easy, age-appropriate text. Their control over target sounds was only moderate, and narrative coherence varied. Both prompt wording and model choice mattered. Gains in phoneme reinforcement tended to cost story quality, and the reverse was also true. No model satisfied every constraint. The work frames phonics story generation as a demanding benchmark for controllable educational text generation. It suggests that methods beyond prompting will be needed, such as constrained decoding, structured generation or fine-tuning.

Key points

  • Models and data: six models available as of March 2025 were compared: Claude-3.7-Sonnet, GPT-4o, GPT-4o-Mini, Gemini-2.0 Flash, DeepSeek-V3 and Mistral-Large-2.1. All prompts, stories and evaluation code are released on Hugging Face as monicamanlises/zero-shot-phonics.
  • Prompt design:
    • Stories target four sentences; outputs of three to five sentences were kept.
    • Half the prompts ask for one story and half ask for eight.
    • To test whether wording matters, the prompts vary their label for the task, using either "phonics stories" or "decodable texts".
  • Phonological scoring: phonemes are written in ARPABET. Alignment is measured by matching GP pairs after G2P conversion, which ignores pronunciation variation.
  • Narrative scoring: narrative quality was judged by a single rubric-guided LLM evaluator. Readability relied on formula-based measures.
  • Limitations:
    • There was no expert review, classroom testing or comparison with human-written phonics stories.
    • Sampling was non-deterministic.
    • Only English was studied.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓De La Salle UniversityManila · Philippines2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksText generation · 202 works

Cite

Cite as Maria Monica Manlises and Ethel Ong. 2026. Zero Shot Phonics: Evaluating Constraint-Adherent Phonics Story Generation in Large Language Models. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bea-1.61

@inproceedings{manlises2026zero,
  title = {{Zero Shot Phonics: Evaluating Constraint-Adherent Phonics Story Generation in Large Language Models}},
  author = {Maria Monica Manlises and Ethel Ong},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.61},
  url = {https://doi.org/10.18653/v1/2026.bea-1.61}
}

References

26 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jawharah Alasmari, Mohammed Alzyoudi, Masheal Alshehri, Rana Alshammari, and Reyouf Aldakan. 2025. An automated predictive model for evaluating narrative cohesion in children’s stories: a computational linguistic approach considering gérard genette’s narrative structure theory. International Journal… find
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  3. Jennifer P. Cheatham and Jill H. Allor. 2012. The influence of decodability in early reading text on reading achievement: a review of the evidence. Reading and Writing, 25(9):2223–2246. find
  4. Department for Education and Skills. 2008. Letters and sounds: Principles and practice of high quality phonics. Technical report, UK Department for Education and Skills, Nottingham, UK. find
  5. Linnea C. Ehri. 2020. The science of learning to read words: A case for systematic phonics instruction. Reading Research Quarterly, 55(S1):S45–S60. find
  6. Charlie Hargood, David Millard, and Mark Weal. 2011. Measuring narrative cohesion: A five variables approach. In Narrative and Hypertext Workshop at Hypertext 11 (06/06/11). find
  7. Victor Kuperman, Hans Stadthagen-Gonzalez, and Marc Brysbaert. 2012. Age-of-acquisition ratings for 30,000 English words. Behavior Research Methods, 44(4):978–990. find
  8. Hugo Liu and Push Singh. 2002. Makebelieve: using commonsense knowledge to generate stories. In Eighteenth National Conference on Artificial Intelligence, page 957–958, USA. American Association for Artificial Intelligence. find
  9. Ronald William Marbun and Makoto Shishido. 2026. A readability-driven prompting framework for accurate grade-specific efl narrative creation. International Journal of Advanced Computer Science and Applications, 17(1). find
  10. James R. Meehan. 1977. Tale-spin, an interactive program that writes stories. In Proceedings of the 5th International Joint Conference on Artificial Intelligence - Volume 1, IJCAI’77, pages 91–98, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  11. Dennis Murphy Odo. 2024. The use of decodable texts in the teaching of reading in children without reading disabilities: a meta-analysis. Literacy, 58(3):267– 277. find
  12. National Reading Panel. 2000. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Technical Report NIH Publication No. 00-4754, National Institute of Child Health and Human De… find
  13. Pearson Education. n.d. Phonics progression chart. Accessed 2025. find
  14. Linkai Peng, Baorian Nuchged, and Yingming Gao. 2025. Spoken language intelligence of large language models for language learning. Preprint, arXiv:2308.14536. — [arXiv:2308.14536](https://arxiv.org/abs/2308.14536) find
  15. A All prompt sets incorporated the basic prompt structure in Figure 1 and had respective additional requirements listed in Figures 2, 3, and 4. System Prompt: You are a children’s story writer and children’s educator specializing in speech and language acquisition. You write decodable texts and phon… find
  16. Ashley E. Pennell, Rebecca Lee Payne Jordan, Kindel Turner Nash, Kerry Elson, and Woodrow Tra-then. 2024. A healthy diet for beginning readers: Decodable texts as part of a comprehensive literacy program. The Reading Teacher, 77(5):673–684. find
  17. Melissa Roemmele, Andrew S. Gordon, and Reid Swanson. 2017. Evaluating story generation systems using automated linguistic analyses. find
  18. Story Requirements: - (Prompt-specific additional requirements) - The story must contain exactly four sentences. - Each sentence must contain fewer than eleven syllables. - The story must reinforce the target phoneme(s). find
  19. Jim Rose. 2006. Independent review of the teaching of early reading. Technical report, UK Department for Education. find
  20. Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, and Xuezhe Ma. 2023. Evaluating large language models on controlled generation tasks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3155–3168, Singapore… find
  21. Ashima Suvarna, Harshita Khandelwal, and Nanyun Peng. 2024. PhonologyBench: Evaluating phonological skills of large language models. In Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 1–14, Bangkok, Thailand. Association for Computational Linguistics. find
  22. Yufei Tian, Tenghao Huang, Miri Liu, Derek Jiang, Alexander Spangher, Muhao Chen, Jonathan May, and Nanyun Peng. 2024. Are large language models capable of generating human-level narratives? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17659–17681,…
  23. S.R. Turner. 1992. MINSTREL, a Computer Model of Creativity and Storytelling. UCLA Computer Science Department. find
  24. Maria Valentini, Jennifer Weber, Jesus Salcido, Téa Wright, Eliana Colunga, and Katharina von der Wense. 2023. On the automatic generation and simplification of children‘s stories. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3588–3598, Singapore.… find
  25. Zhuohan Xie, Trevor Cohn, and Jey Han Lau. 2023. The next chapter: A study of large language models in storytelling. In Proceedings of the 16th International Natural Language Generation Conference, pages 323–351, Prague, Czechia. Association for Computational Linguistics. find
  26. Prompt Design Additional Requirements: - Each sentence must include at least 2 words containing the grapheme ’{grapheme}’. *Figure 2: Prompt Set 1: Single GP and Narrative Focus* Across all prompt sets, systematic variations, as seen in Table 2 were applied. The phrase “phonics story” was replaced w… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/manlises-2026-zero-shot-phonics-evaluating-constraint/