Elamur Research⌕
☰
home / papers / 2026 / Contextual Diversity Measure (CDM) for Controllable Story Generation in Large Language Models

Contextual Diversity Measure (CDM) for Controllable Story Generation in Large Language Models

Paper · 2026 · Richard Susilo, Hanna Suominen and Patrik Haslum

Summary

The paper targets a gap in controllable text generation: there is no metric for how varied a set of generated scenarios is when they all have to follow the same abstract meaning. Here that meaning is a fixed predicate-argument structure based on Semantic Role Labeling. For example, one ethical dilemma might be retold in medical, legal or engineering settings. The authors formalise this as instantiating abstract entities, predicates and roles with concrete phrases. They then propose the Contextual Diversity Measure (CDM). CDM embeds the filled phrases and computes a direction between instantiation centroids. It then splits each position's embedding change into a component along that direction and an orthogonal one.

They tested CDM on a scenario-based ethics dataset with a manually curated subset and an LLM-generated subset. Across four embedding models, CDM reliably ranked pairs of semantically distant domains as more diverse than pairs of nearby domains. Its separation margins were much larger than those of standard diversity and similarity metrics. This matters because CDM could serve both as an evaluation tool and as a training signal for diversity-aware generation.

Key points

  • CDM was run with GloVe, Word2Vec, MiniLM and FastText embeddings. All statistical tests were significant at p < 0.05.
    • Cohen's d was 0.292–0.508 on the curated subset.
    • Cohen's d was 0.677–1.195 on the LLM-generated subset.
  • Baselines were BERTScore, Distinct-1, Distinct-2, Self-BLEU and Sentence Similarity.
    • On the curated subset, CDM reached 100% accuracy with three embeddings. BERTScore also reached 100%.
    • On the LLM-generated subset, CDM's best accuracy was 91.9%, against 91.0% for the best baseline.
  • CDM's main advantage is discriminative power rather than accuracy.
    • CDM (Word2Vec) had a summed score difference of 161.047, versus 29.339 for Sentence Similarity, about 5.5× higher.
    • All CDM variants exceeded 121, while no baseline exceeded 30.
  • The best embedding choice differed between the two subsets.
  • The ethics data were annotated by Inger Mewburn (ANU) and originate from US NIH research-ethics training material.
    • Code and dataset are released on GitHub under the MIT License.
    • CDM has not yet been integrated into an actual generation pipeline.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Australian National UniversityCanberra · Australia23 works · 6,522 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksText generation · 202 works

Cite

Cite as Richard Susilo, Hanna Suominen and Patrik Haslum. 2026. Contextual Diversity Measure (CDM) for Controllable Story Generation in Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.98

@inproceedings{susilo2026contextual,
  title = {{Contextual Diversity Measure (CDM) for Controllable Story Generation in Large Language Models}},
  author = {Richard Susilo, Hanna Suominen and Patrik Haslum},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.98},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.98}
}

References

34 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nadezhda Almazova, Anna Rubtsova, Nora Kats, Yuri Eremin, and Natalia Smolskaia. 2021. Scenario-based instruction: The case of foreign language training at multidisciplinary university. Education Sciences, 11(5). find
  2. Shurui Bai, Donn Emmanuel Gonda, and Khe Foon Hew. 2024. Write-curate-verify: A case study of leveraging generative ai for scenario writing in scenario-based learning. IEEE Transactions on Learning Technologies, 17:1301–1312. find
  3. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 86–90, Montreal, Quebec, Canada. Association for Comput…
  4. Frederico Belcavello, Marcelo Viridiano, Alexandre Diniz da Costa, Ely Edison da Silva Matos, and Tiago Timponi Torrent. 2020. Frame-based annotation of multimodal corpora: Tracking (a)synchronies in meaning construction. In Proceedings of the International FrameNet Workshop 2020: Towards a Global,… find
  5. Federico Bianchi, Silvia Terragni, Dirk Hovy, Debora Nozza, and Elisabetta Fersini. 2021. Cross-lingual contextualized topic models with zero-shot learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1676–1683, O…
  6. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res., 3(Jan):993–1022.
  7. Xuan Cai, Xuesong Bai, Zhiyong Cui, Danmu Xie, Daocheng Fu, Haiyang Yu, and Yilong Ren. 2026. Text2scenario: Text-driven scenario generation for autonomous driving test. Automotive Innovation, 9(1):102–127. find
  8. Cheng Chang, Siqi Wang, Jiawei Zhang, Jingwei Ge, and Li Li. 2024. Llmscenario: Large language model driven scenario generation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 54(11):6581–6594. find
  9. Huimin Chen, Xiaoyuan Yi, Maosong Sun, Wenhao Li, Cheng Yang, and Zhipeng Guo. 2019. Sentiment-controllable chinese poetry generation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 4925–4931. International Joint Conferences on Artifici… find
  10. Dipanjan Das, Desai Chen, André F. T. Martins, Nathan Schneider, and Noah A. Smith. 2014. Frame-semantic parsing. Computational Linguistics, 40(1):9–56.
  11. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. Preprint, arXiv:1912.02164.
  12. Daniel Deutsch and Dan Roth. 2021. Understanding the extent to which content quality metrics measure the information quality of summaries. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 300–309, Online. Association for Computational Linguistics. find
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  14. Alexander R. Fabbri, Wojciech Kryściński, Bryan Mc-Cann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021. Summeval: Re-evaluating summarization evaluation. Preprint, arXiv:2007.12626. find
  15. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia. Association for Computational Linguistics.
  16. Le Fang, Tao Zeng, Chaochun Liu, Liefeng Bo, Wen Dong, and Changyou Chen. 2021. Outline to story: Fine-grained controllable story generation from cascaded events. Preprint, arXiv:2101.00822. — [arXiv:2101.00822](https://arxiv.org/abs/2101.00822) find
  17. Paullett Golden. 2018. Contextualized writing: Promoting audience-centered writing through scenario based learning. International Journal for the Scholarship of Teaching and Learning, 12(1):6. find
  18. Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. 2020. Content planning for neural story generation with aristotelian rescoring. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4319–4338, Online. Association fo… find
  19. Hongwen Guo, Mo Zhang, Paul Deane, and Randy Bennett. 2020. Effects of scenario-based assessment on students’ writing processes. Journal of Educational Data Mining, 12(1):19–45. find
  20. Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020. A probabilistic formulation of unsupervised text style transfer. Preprint, arXiv:2002.03912. find
  21. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12).
  22. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pag…
  23. Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, and Zhiyu Li. 2024. Controllable text generation for large language models: A survey. Preprint, arXiv:2408.12599.
  24. Nitika Mathur, Timothy Baldwin, and Trevor Cohn. 2020. Tangled up in BLEU: Reevaluating the evaluation of automatic machine translation evaluation metrics. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4984–4997, Online. Association for Computation…
  25. Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. 2018. Style transfer through back-translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 866–876, Melbourne, Australia. Association for Computat…
  26. Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, and Jason Wei. 2022. A recipe for arbitrary text style transfer with large language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 837–848, Du…
  27. Raphael Shu, Hideki Nakayama, and Kyunghyun Cho. 2019. Generating diverse translations with sentence codes. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1823–1827, Florence, Italy. Association for Computational Linguistics. find
  28. Junli Wang, Chenyang Zhang, Dongyu Zhang, Haibo Tong, Chungang Yan, and Changjun Jiang. 2025. A recent survey on controllable text generation: A causal perspective. Fundamental Research, 5(3):1194–1203. find
  29. Hanqing Zhang and Dawei Song. 2022. DisCup: Discriminator cooperative unlikelihood prompt-tuning for controllable text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3392–3406, Abu Dhabi, United Arab Emirates. Association for Computationa… find
  30. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models. ACM Comput. Surv., 56(3). find
  31. Mo Zhang, Peter W. van Rijn, Paul Deane, and Randy E. Bennett. 2019. Scenario-based assessments in writing: An experimental study. Educational Assessment, 24(2):73–90. find
  32. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. find
  33. Koulong Zheng, Zhiyu Shen, Zanhao Chen, Chang Che, and Huixia Zhu. 2024. Application of ai-empowered scenario-based simulation teaching mode in cardiovascular disease education. BMC Medical Education, 24(1):1003. find
  34. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, page 1097–1100, New York, NY, USA. Assoc…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/susilo-2026-contextual-diversity-measure-cdm-controllable/