Elamur Research⌕
☰
home / papers / 2026 / Computational Narrative Understanding for Expressive Text-to-Speech

Computational Narrative Understanding for Expressive Text-to-Speech

Paper · 2026 · Gaspard Michel, Elena V. Epure and Christophe Cerisara

Summary

Audiobook corpora are often seen as too flat for training expressive text-to-speech (TTS). The authors challenge this view. They note that fiction recordings alternate between neutral narration and livelier character dialogue, and existing segmentations blend the two. On LibriHeavy-small, pitch variability increases as a segment contains more quotations.

They build LibriQuote from English fiction recordings on LibriVox, matched with Project Gutenberg texts. Quotations are detected with BookNLP, the audio is transcribed by a Zipformer-Transducer ASR model, and text and audio are then aligned. Every quote also gets automatically extracted speech verbs and adverbs (e.g., "whispered softly") as pseudo-labels of intended delivery, and humans validated these labels.

Fine-tuning the flow-matching F5-TTS on the quotations improves both expressivity and intelligibility. For the autoregressive SparkTTS, fine-tuning mainly improves intelligibility, while training from scratch increases expressivity but lowers intelligibility. Benchmarking shows wide gaps between systems, and IndexTTS2 performs best. The resource supports expressive TTS research and Digital Humanities studies of audiobook prosody.

Key points

  • LibriQuote-train contains about 5.3K hours (5,359 h) of character quotations and 12.7K hours of narration from roughly 3,300 speakers. LibriQuote-test has 7.4 hours of quotations from 15 speakers.
  • The corpus covers about 3 million quotations from 2,991 public-domain novels. It ships with paragraph and quote alignments, so users can extract book context of any length.
  • In an analysis of 120,000 LibriHeavy-small segments, 75% contained narration only. Pitch standard deviation correlated with the number of quotes per segment (Spearman ρ = 0.218, p < 0.001).
  • Expressivity was assessed with an LALM-as-a-judge and a human listening study, and the two showed closely aligned trends. IndexTTS2 reached ground-truth expressivity but sometimes conveyed unintended emotions.
  • The dataset is released on Huggingface (gasmichel/LibriQuote) and GitHub (deezer/libriquote). The authors note it has no WER-based filtering yet.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Idiap Research InstituteMartigny-Combe · Switzerland32 works · 1,954 citations🔬Laboratoire Lorrain de Recherche en Informatique et ses ApplicationsNancy · France30 works · 3,899 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 480 works

Cite

Cite as Gaspard Michel, Elena V. Epure and Christophe Cerisara. 2026. Computational Narrative Understanding for Expressive Text-to-Speech. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.308

@inproceedings{michel2026computational,
  title = {{Computational Narrative Understanding for Expressive Text-to-Speech}},
  author = {Gaspard Michel, Elena V. Epure and Christophe Cerisara},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.308},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.308}
}

References

54 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Luis Daniel Gonzalez. 2021. How to become an audiobook narrator. find
  2. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, and 1 others. 2024. Seed-tts: A family of high-quality versatile speech generation models. arXiv preprint arXiv:2406.02430.
  3. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 421…
  4. Oleg Atamanenko, Anna Chalova, Joseph Coombes, Nikki Cope, Phillip Dang, Zhifeng Deng, Jimmy Du, Michael Ermolenko, Feifan Fan, Yufei Feng, Cheryl Fichter, Pavel Filimonov, Louis Fischer, Kylan Gibbs, Valeria Gusarova, Pavel Karpik, Andreas Assad Kottner, Ian Lee, Oliver Louie, and 13 others. 2025.… — [arXiv:2507.21138](https://arxiv.org/abs/2507.21138) find
  5. Zalán Borsos, Matt Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. find
  6. Brittany Goodwin. 2024. So You Want To Narrate Audiobooks. find
  7. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2025. A survey on llm-as-a-judge. Preprint, arXiv:2411.15594. find
  8. Alexander Gurung and Mirella Lapata. 2024. CHIRON: Rich character representations in long-form narratives. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8523–8547, Miami, Florida, USA. Association for Computational Linguistics. find
  9. Hans Ole Hatzel and Chris Biemann. 2024. Story embeddings — narrative-focused representations of fictional stories. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5931–5943, Miami, Florida, USA. Association for Computational Linguistics. find
  10. Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, and 1 others. 2024. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 8… find
  11. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  12. Shengpeng Ji, Tianle Liang, Yangzhuo Li, Jialong Zuo, Minghui Fang, Jinzheng He, Yifu Chen, Zhengqing Liu, Ziyue Jiang, Xize Cheng, Siqi Zheng, Jin Xu, Junyang Lin, and Zhou Zhao. 2025. Wavreward: Spoken dialogue models with generalist reward evaluators. Preprint, arXiv:2505.09558. — [arXiv:2505.09558](https://arxiv.org/abs/2505.09558) find
  13. VI Lcvenshtcin. 1966. Binary coors capable or ‘correcting deletions, insertions, and reversals. In Soviet physics-doklady, volume 10. find
  14. Ziyue Jiang, Jinglin Liu, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, and 1 others. 2023a. Megatts 2: Boosting prompting mechanisms for zero-shot speech synthesis. arXiv preprint arXiv:2307.07218. — [arXiv:2307.07218](https://arxiv.org/abs/2307.07218) find
  15. Ziyue Jiang, Jinglin Liu, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, and 1 others. 2023b. Megatts 2: Boosting prompting mechanisms for zero-shot speech synthesis. arXiv preprint arXiv:2307.07218. — [arXiv:2307.07218](https://arxiv.org/abs/2307.07218) find
  16. Jacob Kahn, Morgane Rivière, Weiyi Zheng, Evgeny Kharitonov, Qiantong Xu, Pierre-Emmanuel Mazaré, Julien Karadayi, Vitaliy Liptchinsky, Ronan Collobert, Christian Fuegen, Tatiana Likhomanenko, Gabriel Synnaeve, Armand Joulin, Abdelrahman Mohamed, and Emmanuel Dupoux. 2019. Libri-light: A benchmark f… — [arXiv:1912.07875](https://arxiv.org/abs/1912.07875) find
  17. Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, and Daniel Povey. 2024. Libriheavy: a 50,000 hours asr corpus with punctuation casing and context. Preprint, arXiv:2309.08105. — [arXiv:2309.08105](https://arxiv.org/abs/2309.08105) find
  18. Eugene Kharitonov, Damien Vincent, Zalán Borsos, Raphaël Marinier, Sertan Girgin, Olivier Pietquin, Matt Sharifi, Marco Tagliasacchi, and Neil Zeghidour. 2023. Speak, read and prompt: High-fidelity text-to-speech with minimal supervision. Transactions of the Association for Computational Linguistics…
  19. Diederik P. Kingma and Jimmy Ba. 2017. Adam: A method for stochastic optimization. Preprint, arXiv:1412.6980.
  20. Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. 2023a. High-fidelity audio compression with improved rvqgan. Preprint, arXiv:2306.06546. find
  21. Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. 2023b. High-fidelity audio compression with improved rvqgan. Advances in Neural Information Processing Systems, 36:27980–27993. find
  22. Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and 1 others. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems, 36:14005–14034. find
  23. Steven R Livingstone and Frank A Russo. 2018. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PloS one, 13(5):e0196391. find
  24. Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, ShiLiang Zhang, and Xie Chen. 2024. emotion2vec: Self-supervised pre-training for speech emotion representation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15747–15760, Bangkok, Thailand. Association for… find
  25. Ruskin Raj Manku, Yuzhi Tang, Xingjian Shi, Mu Li, and Alex Smola. 2025. Emergenttts-eval: Evaluating tts models on complex prosodic, expressiveness, and linguistic challenges using model-as-a-judge. Preprint, arXiv:2505.23009. — [arXiv:2505.23009](https://arxiv.org/abs/2505.23009) find
  26. Aleksandr Meister, Matvei Novikov, Nikolay Karpov, Evelina Bakhturina, Vitaly Lavrukhin, and Boris Ginsburg. 2023. Librispeech-pc: Benchmark for evaluation of punctuation and capitalization capabilities of end-to-end asr models. Preprint, arXiv:2310.02943. — [arXiv:2310.02943](https://arxiv.org/abs/2310.02943) find
  27. Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. 2023. Finite scalar quantization: Vq-vae made simple. Preprint, arXiv:2309.15505. — [arXiv:2309.15505](https://arxiv.org/abs/2309.15505) find
  28. Gaspard Michel, Elena Epure, Romain Hennequin, and Christophe Cerisara. 2024. Distinguishing fictional voices: a study of authorship verification models for quotation attribution. In Proceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Hum… find
  29. Gaspard Michel, Elena V. Epure, Romain Hennequin, and Christophe Cerisara. 2025. Evaluating LLMs for quotation attribution in literary texts: A case study of LLaMa3. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human L… find
  30. Tu Anh Nguyen, Wei-Ning Hsu, Antony d’Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarani, Tal Remez, Jade Copet, Gabriel Synnaeve, Michael Hassid, and 1 others. 2023. Expresso: A benchmark and analysis of discrete expressive speech resynthesis. arXiv preprint arXiv:2308.05725. — [arXiv:2308.05725](https://arxiv.org/abs/2308.05725) find
  31. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An asr corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206–5210. find
  32. Ankita Pasad, Bowen Shi, and Karen Livescu. 2023. Comparative layer-wise analysis of self-supervised speech models. Preprint, arXiv:2211.03929. — [arXiv:2211.03929](https://arxiv.org/abs/2211.03929) find
  33. William Peebles and Saining Xie. 2023. Scalable diffusion models with transformers. Preprint, arXiv:2212.09748.
  34. Charuta Pethe, Bach Pham, Felix D Childress, Yunting Yin, and Steven Skiena. 2025. Prosody analysis of audiobooks. In 2025 19th International Conference on Semantic Computing (ICSC), page 217–221. IEEE. find
  35. Andrew Piper and Sunyam Bagga. 2024. Using large language models for understanding narrative discourse. In Proceedings of the 6th Workshop on Narrative Understanding, pages 37–46, Miami, Florida, USA. Association for Computational Linguistics. find
  36. Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. Mls: A large-scale multilingual dataset for speech research. arXiv preprint arXiv:2012.03411.
  37. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR.
  38. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. Preprint, arXiv:2204.02152. find
  39. Sandeep Soni, Amanpreet Sihra, Elizabeth Evans, Matthew Wilkens, and David Bamman. 2023. Grounding characters and places in narrative text. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11723–11736, Toronto, Canada. Associat… find
  40. Dominik Stammbach, Maria Antoniak, and Elliott Ash. 2022. Heroes, villains, and victims, and GPT-3: Automated extraction of character roles without training data. In Proceedings of the 4th Workshop of Narrative Understanding (WNU2022), pages 47–56, Seattle, United States. Association for Computation… find
  41. Kobie Van Krieken, Hans Hoeken, and José Sanders. 2017. Evoking and measuring identification with narrative characters–a linguistic cues framework. Frontiers in psychology, 8:1190. find
  42. Krishnapriya Vishnubhotla, Adam Hammond, Graeme Hirst, and Saif Mohammad. 2024. The emotion dynamics of literary novels. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2557–2574, Bangkok, Thailand. Association for Computational Linguistics. find
  43. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, and 1 others. 2023. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111.
  44. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, and 1 others. 2025. Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens. arXiv preprint arXiv:2503.01710. — [arXiv:2503.01710](https://arxiv.org/abs/2503.01710) find
  45. Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024. Maskgct: Zero-shot text-to-speech with masked generative codec transformer. arXiv preprint arXiv:2409.00750. — [arXiv:2409.00750](https://arxiv.org/abs/2409.00750) find
  46. Alissa P Wolters, Young-Suk Grace Kim, and John William Szura. 2022. Is reading prosody related to reading comprehension? a meta-analysis. Scientific Studies of Reading, 26(1):1–20. find
  47. Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. 2023. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. arXiv preprint arXiv:2304.09116. — [arXiv:2304.09116](https://arxiv.org/abs/2304.09116) find
  48. Zhen Ye, Xinfa Zhu, Chi-Min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, Hongzhan Lin, Jianyi Chen, Xingjian Du, Liumeng Xue, Yunlin Chen, Zhifei Li, Lei Xie, Qiuqiang Kong, Yike Guo, and Wei Xue. 2025. Llasa: Scaling train-time and inference-time compute for lla… — [arXiv:2502.04128](https://arxiv.org/abs/2502.04128) find
  49. Hubert Siuzdak. 2024. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. Preprint, arXiv:2306.00814. — [arXiv:2306.00814](https://arxiv.org/abs/2306.00814) find
  50. Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. Libritts: A corpus derived from librispeech for text-to-speech. CoRR, abs/1904.02882.
  51. Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. 2024. Speechtokenizer: Unified speech tokenizer for speech large language models. Preprint, arXiv:2308.16692.
  52. Guanlong Zhao, Sinem Sonsaat, Alif Silpachai, Ivana Lucic, Evgeny Chukharev-Hudilainen, John Levis, and Ricardo Gutierrez-Osuna. 2018. L2-arctic: A non-native english speech corpus. In Interspeech 2018, pages 2783–2787. find
  53. Kun Zhou, Berrak Sisman, Rui Liu, and Haizhou Li. 2021. Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 920–924. IEEE. find
  54. Siyi Zhou, Yiquan Zhou, Yi He, Xun Zhou, Jinchao Wang, Wei Deng, and Jingchen Shu. 2025. Indextts2: A breakthrough in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech. Preprint, arXiv:2506.21619. — [arXiv:2506.21619](https://arxiv.org/abs/2506.21619) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/michel-2026-computational-narrative-understanding-expressive-text/