Elamur Research⌕
☰
home / papers / 2021 / Automatic Story Generation: Challenges and Attempts

Automatic Story Generation: Challenges and Attempts

Summary

This survey addresses automatic story generation. Even modern transformer models struggle with this task: they have difficulty maintaining a coherent plot, consistent characters, and genre conventions. The authors, from Georgia Tech, do not propose a new model or dataset. Instead, they organize the existing literature differently from previous surveys: not by architecture type but by the field's key challenges. There are four such challenges: controllability of generation, use of commonsense knowledge, plausible character actions, and creativity of language. For each challenge, the authors examine specific systems, their approaches, strengths, and weaknesses. The main conclusions are as follows. The more tightly generation is controlled, the duller the stories become. Standard NLG metrics are poor at assessing narrative quality. Illogical and inconsistent plots remain an open problem. The paper is useful to newcomers as an initial map of the field. It also summarizes the state of the art at the threshold of the shift from seq2seq to large language models.

Key points

  • Controllability: the survey covers RL with reward shaping (Tambwekar et al., 2019), which has to be retrained for each new goal, and the Fusion model (Fan et al., 2018) on WRITING PROMPTS with a "premise → story" scheme. It also covers Plan-and-write (Yao et al., 2019), which uses a "title → storyline → text" scheme and inter- and intra-story repetition metrics. Plot Machines (Rashkin et al., 2020) combine an outline, memory, and discourse labels (introduction/body/conclusion). Generation by interpolation (Wang et al., 2020) generates candidates with GPT-2 and ranks them with RoBERTa by their consistency with the first and last sentences.
  • Trend: each of the reviewed works controls either the overall plan of the story or its ending. Overall, the field is shifting noticeably toward transformer models.
  • Commonsense: the most common approaches are fine-tuning on commonsense data, causality analysis, and incorporating knowledge graphs into encoders. All of them improve quality, but none guarantees a logical story.
  • Characters: an abstract representation of events as "predicate + arguments", with separate attention to preceding verbs, produced more coherent texts and more diverse verbs than the Fusion model. EC-CLF and EC-EM (Brahman & Chaturvedi, 2020) use RL to make GPT-2 follow the protagonist's emotional arc, which is evaluated with COMET. Interaction among multiple characters has barely been studied.
  • Future directions: metrics designed specifically for narratives (creativity, interestingness) are needed in place of BLEU, ROUGE, and perplexity. The authors also recommend drawing commonsense inferences at the level of the whole story rather than individual sentences or paragraphs, and seeking a balance between creativity and structural coherence.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.nuse-1.8 · arXiv 2102.12634

@inproceedings{alabdulkarim2021automatic,
  title = {{Automatic Story Generation: Challenges and Attempts}},
  author = {Alabdulkarim, Amal and Li, Siyan and Peng, Xiangyu},
  booktitle = {Proceedings of the Third Workshop on Narrative Understanding},
  year = {2021},
  doi = {10.18653/v1/2021.nuse-1.8},
  eprint = {2102.12634},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.nuse-1.8}
}

References

57 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  2. Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, and Mark O Riedl. 2020. Automated storytelling via causal, commonsense plot ordering. arXiv preprint arXiv:2009.00829. — [arXiv:2009.00829](https://arxiv.org/abs/2009.00829) find
  3. Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguistic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Langu… find our card
  4. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. pages 889–898. find
  5. Angela Fan, Mike Lewis, and Yann Dauphin. 2019. Strategies for structuring story generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2650– 2660. find
  6. John Austin. 2019. The book of endless history: Authorial use of gpt2 for interactive storytelling. In International Conference on Interactive Digital Storytelling, pages 429–432. Springer. find
  7. Saadia Gabriel, Chandra Bhagavatula, Vered Shwartz, Ronan Le Bras, Maxwell Forbes, and Yejin Choi. 2020. Paragraph-level commonsense transformers with recurrent memory. arXiv preprint arXiv:2010.01486. find
  8. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. Comet: Commonsense transformers for automatic knowledge graph construction. pages 4762– 4779. find our card
  9. Pablo Gervás. 2009. Computational approaches to storytelling and creativity. AI Magazine, 30(3):49–49. find
  10. Faeze Brahman and Snigdha Chaturvedi. 2020. Modeling protagonist emotions for emotion-aware storytelling. arXiv preprint arXiv:2010.06822. find
  11. Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, and Minlie Huang. 2020. A knowledge-enhanced pre-training model for commonsense story generation. Transactions of the Association for Computational Linguistics, 8:93–108. find
  12. Erik Cambria, Yangqiu Song, Haixun Wang, and Amir Hussain. 2011. Isanette: A common and common sense knowledge base for opinion mining. In 2011 IEEE 11th International Conference on Data Mining Workshops, pages 315–322. IEEE. find
  13. Jian Guan, Yansen Wang, and Minlie Huang. 2019. Story ending generation with incremental encoding and commonsense knowledge. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6473–6480. find
  14. Marc Cavazza and David Pizzi. 2006. Narratology for Interactive Storytelling: A Critical Introduction. In Technologies for Interactive Digital Storytelling and Entertainment, Lecture Notes in Computer Science, pages 72–83, Berlin, Heidelberg. Springer. find
  15. Chenglong Hou, Chensong Zhou, Kun Zhou, Jinan Sun, and Sisi Xuanyuan. 2019. A survey of deep learning applied to story generation. In Smart Computing and Communication, pages 1–10, Cham. Springer International Publishing. find
  16. Elizabeth Clark, Yangfeng Ji, and Noah A. Smith. 2018. Neural text generation in stories using entity representations as context. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers),… find
  17. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Cosmos qa: Machine reading comprehension with contextual commonsense reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langua… find our card
  18. Kevin Clark and Christopher D Manning. 2016. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2256–2262. find our card
  19. Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, and Noah A Smith. 2017. Dynamic entity representations in neural language models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1830–1839. find
  20. Joe Davison, Joshua Feldman, and Alexander M Rush. 2019. Commonsense knowledge mining from pre-trained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 117… find
  21. Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Advances in neural information processing systems, pages 3294–3302. find
  22. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and evaluation framework for deeper understanding of commonsense stories. arXiv preprint arXiv:1604.01696. find
  23. B. Kybartas and R. Bidarra. 2017. A survey on story generation techniques for authoring computational narratives. IEEE Transactions on Computational Intelligence and AI in Games, 9(3):239–253. find
  24. Wendy G Lehnert, Michael G Dyer, Peter N Johnson, CJ Yang, and Steve Harley. 1983. Boris—an experiment in in-depth understanding of narratives. Artificial intelligence, 20(1):15–62. find
  25. Nasrin Mostafazadeh, Aditya Kalyanpur, Lori Moon, David Buchanan, Lauren Berkowitz, Or Biran, and Jennifer Chu-Carroll. 2020. Glucose: Generalized and contextualized story explanations. arXiv preprint arXiv:2009.07758. find our card
  26. Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning. Citeseer. find
  27. Geoffrey Nunberg. 1987. Position paper on commonsense and formal semantics. In Theoretical Issues in Natural Language Processing 3. find
  28. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  29. Simon Ostermann, Ashutosh Modi, Michael Roth, Stefan Thater, and Manfred Pinkal. 2018. Mcscript: A novel dataset for assessing machine comprehension using script knowledge. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  30. Linbo Luo, Wentong Cai, Suiping Zhou, Michael Lees, and Haiyan Yin. 2015. A review of interactive narrative systems and technologies: a training perspective. SIMULATION, 91(2):126–147. Publisher: SAGE Publications Ltd STM. find
  31. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  32. Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley, and Garrison Cottrell. 2019. Improving neural story generation by targeted common sense grounding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Na… find
  33. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8). find
  34. Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao. 2020. Plotmachines: Outline-conditioned generation with dynamic plot state tracking. arXiv preprint arXiv:2004.14967. find
  35. Lara J Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O Riedl. 2017. Event representations for automated story generation with deep neural nets. arXiv preprint arXiv:1706.01331. find
  36. John Fairbanks Reeves. 1991. Computational morality: A process model of belief conflict and resolution for story understanding. find
  37. Mark O. Riedl. 2016. Computational Narrative Intelligence: A Human-Centered Goal for Artificial Intelligence. arXiv:1602.06484 [cs]. ArXiv: 1602.06484. — [arXiv:1602.06484](https://arxiv.org/abs/1602.06484) find
  38. Lara J. Martin, Brent Harrison, and Mark O. Riedl. 2016. Improvisational computational storytelling in open worlds. In Interactive Storytelling, pages 73– 84, Cham. Springer International Publishing. find
  39. Mark Owen Riedl and Vadim Bulitko. 2013. Interactive narrative: An intelligent systems approach. Ai Magazine, 34(1):67–67. find
  40. Gonzalo Méndez, Pablo Gervás, and Carlos León. 2016. On the use of character affinities for story plot generation. In Knowledge, Information and Creativity Support Systems, pages 211–225. Springer. find
  41. David L Roberts and Charles L Isbell. 2008. A Survey and Qualitative Analysis of Recent Advances in Drama Management. page 15. find
  42. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391. find
  43. Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019a. Atomic: An atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3027–3… find
  44. Todor Mihaylov and Anette Frank. 2018. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 821–832. find
  45. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019b. Social iqa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical R Michael Young, Stephen Ware, Brad Cassell, and Justus Robertson. 2013. Plans and Planning in Narrative Gene… find
  46. Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4453– 4463. find
  47. Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A Smith. 2017. The effect of different writing tasks on linguistic style: A case study of the roc story cloze task. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 15–2… find
  48. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Processing Systems, pages 9054–9065. find our card
  49. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: an open multilingual graph of general knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pages 4444–4451. find
  50. Wendy A. Suzuki, Mónica I. Feliú-Mójer, Uri Hasson, Rachel Yehuda, and Jean Mary Zarate. 2018. Dialogues: The Science and Power of Storytelling. The Journal of Neuroscience, 38(44):9468–9470. find
  51. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  52. Pradyumna Tambwekar, Murtaza Dhuliawala, Lara J Martin, Animesh Mehta, Brent Harrison, and Mark O Riedl. 2019. Controllable neural story plot generation via reward shaping. pages 5982–5988. find
  53. Trieu H Trinh and Quoc V Le. 2018. A simple method for commonsense reasoning. arXiv preprint arXiv:1806.02847. find
  54. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In International Conference on Learning Representations. find
  55. Su Wang, Greg Durrett, and Katrin Erk. 2020. Narrative interpolation for generating and understanding stories. arXiv preprint arXiv:2008.07466. — [arXiv:2008.07466](https://arxiv.org/abs/2008.07466) find
  56. An Yang, Quan Wang, Jing Liu, Kai Liu, Yajuan Lyu, Hua Wu, Qiaoqiao She, and Sujian Li. 2019. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2346–2… find
  57. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-Write: Towards Better Automatic Storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, 33:7378–7385. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/alabdulkarim-2021-automatic-story-generation-challenges-attempts/