Elamur Research⌕
☰
home / papers / 2026 / The Prosody of Emojis

The Prosody of Emojis

Paper · 2026 · Giulio Zhou, Tsz Kin Lam, Alexandra Birch et al.

Summary

The paper asks whether emojis in written text carry prosodic intent, that is, whether they change how people say a sentence aloud and whether listeners can work back from the voice to the emoji. Zhou, Lam, Haddow and Birch ran four crowdsourced experiments. Participants added contrasting emojis to fixed sentences, British English speakers read the variants without being told the study was about emojis, and listeners judged recordings of the same speaker and of different speakers. Because the wording stayed identical across conditions, any acoustic differences can be attributed to the emoji.

Bayesian multilevel models showed three things:

  • Speakers reliably adjust their delivery depending on the emoji.
  • Listeners identify the intended emoji well above chance, and they do better when different speakers converge on similar contours.
  • Emojis that are further apart in meaning produce larger prosodic differences, with intensity and timing as the main acoustic carriers.

The work offers the first direct empirical link between emoji semantics and spoken prosody. It also releases a parallel speech resource relevant to expressive TTS, screen readers and multimodal LLMs that currently treat emojis as plain tokens.

Key points

  • Data pipeline:
    • 230 base utterances taken from user-generated text yielded 1,595 emoji-augmented stimuli (Experiment 1).
    • 129 British English speakers produced 3,153 valid recordings, with emoji-free control versions included (Experiment 2).
  • Perception tests:
    • 107 listeners rated 645 same-speaker pairs, covering Emoji–Emoji and Emoji–NoEmoji contrasts (Experiment 3).
    • 10 listeners rated 49 cross-speaker pairs against distractors drawn from 60 face emojis (Experiment 4).
  • Analysis: Bayesian multilevel modelling with speaker-level random intercepts. Prosodic distance was measured with DTW on pitch and intensity contours.
  • Resource: the dataset is published on Hugging Face as GiulioZh/ProsodyEmoji. Participants were recruited via Prolific under University of Edinburgh ethics approval, and Whisper was used to screen the recordings.
  • Limitations:
    • The speech is elicited reading rather than spontaneous conversation.
    • Only British English speakers took part.
    • Experiment 4 had a small listener pool.
    • DTW-based distances are acknowledged as imperfect proxies for human perception.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom381 works · 80,747 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 worksHuman-Computer Interaction · 160 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 519 works

Cite

Cite as Giulio Zhou, Tsz Kin Lam, Alexandra Birch and Barry Haddow. 2026. The Prosody of Emojis. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1459

@inproceedings{zhou2026prosody,
  title = {{The Prosody of Emojis}},
  author = {Giulio Zhou, Tsz Kin Lam, Alexandra Birch and Barry Haddow},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1459},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1459}
}

References

44 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ehab Saleh Alnuzaili, Muhammad Waqar Amin, Sami Saad Alghamdi, Nazir Ahmed Malik, Abdulbasit A. Alhaj, and Asad Ali. 2024. Emojis as graphic equivalents of prosodic features in natural speech: evidence from computer-mediated discourse of whatsapp and facebook. Cogent Arts & Humanities, 11(1):2391646… find
  2. Hamza Alshenqeeti. 2016. Are emojis creating a new or old visual language for new generations? a socio-semiotic study. Advances in language and Literary Studies, 7(6). find
  3. Dane Archer and Robin M Akert. 1977. Words and everything else: Verbal and nonverbal cues in social interpretation. Journal of personality and social psychology, 35(6):443. find
  4. Dale J Barr, Roger Levy, Christoph Scheepers, and Harry J Tily. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language, 68(3):255–278. find
  5. Isabelle Boutet, Megan LeBlanc, Justin A Chamber-land, and Charles A Collin. 2021. Emojis influence emotional communication, social attributions, and information processing. Computers in Human Behavior, 119:106722. find
  6. Paul-Christian Bürkner. 2021. Bayesian item response modeling in R with brms and Stan. Journal of Statistical Software, 100(5):1–54. find
  7. Ricardo JGB Campello, Davoud Moulavi, and Jörg Sander. 2013. Density-based clustering based on hierarchical density estimates. In Pacific-Asia conference on knowledge discovery and data mining, pages 160–172. Springer. find
  8. Jie Chi, Maureen de Seyssel, and Natalie Schluter. 2025. The role of prosody in spoken question answering. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 8468–8479, Albuquerque, New Mexico. Association for Computational Linguistics. find
  9. Jennifer Cole. 2015. Prosody in context: A review. Language, Cognition and Neuroscience, 30(1-2):1– 31. \#\# Acknowledgments David Crystal. 2001. Language and the Internet. Cambridge University Press. find
  10. This work was supported in part by the Centre for Doctoral Training in Natural Language Processing, funded by UKRI (grant EP/S022481/1). We extend our sincere thanks to Catherine Lai and Artemis Deligianni (University of Edinburgh) for their valu-Justyna Czkestochowska, Kristina Gligorić, Maxime Pe… find
  11. Caluã de Lacerda Pataca and Paula Dornhofer Paro Costa. 2020. Speech modulated typography: towards an affective representation model. In Proceedings of the 25th International Conference on Intelligent User Interfaces, pages 139–143. find
  12. John E Drury, Shari R Baum, Hope Valeriote, and Karsten Steinhauer. 2016. Punctuation and implicit prosody in silent reading: An erp study investigating english garden-path sentences. Frontiers in psychology, 7:1375. find
  13. LouAnn Gerken and Karla McGregor. 1998. An overview of prosody and its role in normal and disordered child language. American Journal of Speech-Language Pathology, 7(2):38–48. find
  14. Tüge T Gülşen. 2016. You tell me in emojis. In Computational and cognitive approaches to narratology, pages 354–375. IGI Global. find
  15. Nele Hellbernd and Daniela Sammler. 2016. Prosody conveys speaker’s intentions: Acoustic cues for speech act perception. Journal of Memory and Language, 88:70–86. find
  16. Julia Hirschberg. 2002. Communication and prosody: Functional aspects of prosody. Speech Communication, 36(1-2):31–43. find
  17. Thomas Holtgraves and Caleb Robinson. 2020. Emoji can facilitate recognition of conveyed indirect meaning. PloS one, 15(4):e0232361. find
  18. Jiaxiong Hu, Qianyao Xu, Limin Paul Fu, and Yingqing Xu. 2019. Emojilization: An automated method for speech to emoji-labeled text. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–6. find
  19. Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of classification, 2:193–218. find
  20. Yannick Jadoul, Bill Thompson, and Bart de Boer. 2018. Introducing Parselmouth: A Python interface to Praat. Journal of Phonetics, 71:1–15. find
  21. Allan James. 2017. Prosody and paralanguage in speech and the social media: The vocal and graphic realisation of affective meaning. Linguistica, 57(1):137– 149. find
  22. Harold Jeffreys. 1998. The theory of probability. OuP Oxford. find
  23. Elsi Kaiser. 2021. Focus marking with emoji: On the relation between information structure and expressive meaning. In Colloque de syntaxe et sémantique á Paris (CSSP), Paris, France. find
  24. Joseph Kane, Michael N Johnstone, and Patryk Szewczyk. 2024. Voice synthesis improvement by machine learning of natural prosody. Sensors, 24(5):1624. find
  25. Ilse Lehiste and Norman J Lass. 1976. Suprasegmental features of speech. Contemporary issues in experimental phonetics, 225:239. find
  26. Li Li and Yue Yang. 2018. Pragmatic functions of emoji in internet-based communication—a corpus-based study. Asian-Pacific Journal of Second and Foreign Language Education, 3:1–12. find
  27. Leland McInnes, John Healy, Nathaniel Saul, and Lukas Grossberger. 2018. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861. find
  28. Hannah Miller, Daniel Kluver, Jacob Thebault-Spieker, Loren Terveen, and Brent Hecht. 2017. Understanding emoji ambiguity in context: The role of text in emoji-related miscommunication. In Eleventh international AAAI conference on web and social media. find
  29. Max Morrison, Caedon Hsieh, Nathan Pruyne, and Bryan Pardo. 2023. Cross-domain neural pitch and periodicity estimation. In arXiv preprint arXiv:2301.12258. find
  30. Noa Na’aman, Hannah Provenza, and Orion Montoya. 2017. Varying linguistic purposes of emoji in (Twitter) context. In Proceedings of ACL 2017, Student Research Workshop, pages 136–141, Vancouver, Canada. Association for Computational Linguistics. find
  31. R Core Team. 2025. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. find
  32. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  33. Seamless Communication et al. 2023. Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187.
  34. RJ Skerry-Ryan, Eric Battenberg, Ying Xiao, Yuxuan Wang, Daisy Stanton, Joel Shor, Ron Weiss, Rob Clark, and Rif A Saurous. 2018. Towards end-to-end prosody transfer for expressive speech synthesis with tacotron. In international conference on machine learning, pages 4693–4702. PMLR. find
  35. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. Advances in neural information processing systems, 33:16857– 16867.
  36. Ioannis Tsiamas, Matthias Sperber, Andrew Finch, and Sarthak Garg. 2024. Speech is more than words: Do speech-to-text translation systems leverage prosody? In Proceedings of the Ninth Conference on Machine Translation, pages 1235–1257, Miami, Florida, USA. Association for Computational Linguistics. find
  37. Paige Tuttösí, Shivam Mehta, Zachary Syvenky, Bermet Burkanova, Gustav Eje Henter, and Angelica Lim. 2025. Emojivoice: Towards long-term controllable expressivity in robot speech. arXiv preprint arXiv:2506.15085. — [arXiv:2506.15085](https://arxiv.org/abs/2506.15085) find
  38. Nguyen Xuan Vinh, Julien Epps, and James Bailey. 2010. Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. Journal of Machine Learning Research, 11(95):2837–2854. find
  39. Michael Wagner. 2016. How to be kind with prosody. in Speech prosody, 1:250–1253. find
  40. Sarenne Wallbridge, Christoph Minixhofer, Catherine Lai, and Peter Bell. 2025. Prosodic structure beyond lexical content: a study in self-supervised learning. In proceedings of Interspeech 2025. find
  41. Nigel Ward and Gina-Anne Levow. 2021. Prosody: Models, methods, and applications. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Tutorial Abstracts, pages 26–28, Online. Associatio… find
  42. Lukas Wolf, Tiago Pimentel, Evelina Fedorenko, Ryan Cotterell, Alex Warstadt, Ethan Wilcox, and Tamar Regev. 2023. Quantifying the redundancy between prosody and text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9765–9784, Singapore. Association f… find
  43. Giulio Zhou, Sydelle De Souza, Ella Markham, Oghenetekevwe Kwakpovwe, and Sumin Zhao. 2024a. Semantics and sentiment: Cross-lingual variations in emoji use. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18698–18712, Miami, Florida, USA. Association… find
  44. Giulio Zhou, Tsz Kin Lam, Alexandra Birch, and Barry Haddow. 2024b. Prosody in cascade and direct speech-to-text translation: a case study on Korean wh-phrases. In Findings of the Association for Computational Linguistics: EACL 2024, pages 674–683, St. Julian’s, Malta. Association for Computational… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2026-prosody-emojis/