Elamur Research⌕
☰
home / papers / 2026 / StressTest: Can YOUR Speech LM Handle the Stress?

StressTest: Can YOUR Speech LM Handle the Stress?

Paper · 2026 · Iddo Yosha, Gallil Maimon and Yossi Adi

Summary

Speech-aware language models (SLMs) process audio directly, but no one has systematically checked whether they grasp sentence stress. Stress is the emphasis on particular words that can change what a speaker means even when the wording stays the same. The authors built StressTest, a benchmark of English sentences that a single professional actor recorded with two or more emphasis patterns, each tied to a different intended meaning. It covers two tasks:

  • Sentence Stress Reasoning (SSR): inferring the speaker's intent from the audio, a new task.
  • Sentence Stress Detection (SSD): finding the emphasized words, an established task.

They added StressPresso, a re-annotated subset of Expresso, as a complementary test set. Leading SLMs performed poorly. To address this, the authors designed a synthetic data generation and filtering pipeline that produces Stress-17k, and fine-tuned on it to create StresSLM. StresSLM transfers to real recordings and clearly beats existing SLMs on both tasks, with little loss on the original tasks. The work shows that prosodic meaning is a real gap in current speech models and offers a practical way to close it.

Key points

  • Data: StressTest sentences were written and cross-checked by fluent annotators and grouped into four stress types: contrastive, emphatic, new-information and focus. StressPresso uses only neutral-emotion Expresso samples from four speakers (two male, two female), with intended meanings annotated after recording.
  • Evaluation: SSR accuracy is scored with gpt-4o as an LLM-as-a-judge. An open-ended SSR variant rates free-form answers on a 1–5 scale.
  • Reference baselines: A text-only gpt-4o given the transcription and the gold stress labels reaches 86.2 on StressTest and 83.6 on StressPresso. A cascade of WhiStress → gpt-4o reaches 83.4 and 79.7.
  • Stress detection: StresSLM trained only on detection-oriented tasks reaches an F1 of 90.5 on StressTest, compared with 88.3 for WhiStress.
  • Ablations:
    • Fine-tuning the speech encoder helps both tasks.
    • Staged training (full data, then the filtered subset) balances SSR and SSD.
    • Removing the elaborated-explanation task hurts SSR.
    • The work covers English only.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hebrew University of JerusalemJerusalem · Israel65 works · 9,387 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

Cite

Cite as Iddo Yosha, Gallil Maimon and Yossi Adi. 2026. StressTest: Can YOUR Speech LM Handle the Stress?. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.64

@inproceedings{yosha2026stresstest,
  title = {{StressTest: Can YOUR Speech LM Handle the Stress?}},
  author = {Iddo Yosha, Gallil Maimon and Yossi Adi},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.64},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.64}
}

References

53 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Siddhant Arora, Kai-Wei Chang, Chung-Ming Chien, Yifan Peng, Haibin Wu, Yossi Adi, Emmanuel Dupoux, Hung-Yi Lee, Karen Livescu, and Shinji Watanabe. 2025. On the landscape of spoken language models: A comprehensive survey. Preprint, arXiv:2504.08528. — [arXiv:2504.08528](https://arxiv.org/abs/2504.08528) find
  2. Jinze Bai and Shuai Bai et al. 2023. Qwen technical report. Preprint, arXiv:2309.16609.
  3. Steven E. Boer. 1979. Meaning and contrastive stress. The Philosophical Review, 88(2):263–298. find
  4. Dwight Bolinger. 1972. Accent is predictable (if you’re a mind-reader). Language, 48(3):633–644. find
  5. Dwight L. Bolinger. 1958. Stress and information. American Speech, 33(1):5–20. find
  6. Dwight L. Bolinger. 1961. Contrastive accent and contrastive stress. Language, 37(1):83–96. find
  7. Noam Chomsky and Morris Halle. 1968. The sound pattern of english. find
  8. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. Qwen2-audio technical report. Preprint, arXiv:2407.10759.
  9. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, and Asaf Aharoni. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context,…
  10. Maureen de Seyssel, Antony D’Avirro, Adina Williams, and Emmanuel Dupoux. 2024. Emphassess : a prosodic benchmark on assessing emphasis transfer in speech-to-speech models. Preprint, arXiv:2312.14069. find
  11. Fred I. Dretske. 1972. Contrastive statements. The Philosophical Review, 81(4):411–437. find
  12. Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. 2025. Llama-omni: Seamless speech interaction with large language models. Preprint, arXiv:2409.06666. find
  13. Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. 2025. Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities. Preprint, arXiv:2503.03983. — [arXiv:2503.03983](https://arxiv.org/abs/2503.03983) find
  14. Arushi Goel, Sreyan Ghosh, Jaehyeon Kim, Sonal Kumar, Zhifeng Kong, Sang gil Lee, Chao-Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, and Bryan Catanzaro. 2025. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models. Preprint, arXiv:2507.08128. — [arXiv:2507.08128](https://arxiv.org/abs/2507.08128) find
  15. Aaron Grattafiori and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  16. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2025. A survey on llm-as-a-judge. Preprint, arXiv:2411.15594. find
  17. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.
  18. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  19. Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, and 1 others. 2024. Wavchat: A survey of spoken dialogue models. arXiv preprint arXiv:2411.13577. find
  20. J. Kahn, M. Riviere, W. Zheng, E. Kharitonov, Q. Xu, P.E. Mazare, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux. 2020. Libri-light: A benchmark for asr with limited or no supervision. In ICASSP 2020 - 2020 IEEE International… find
  21. Damjan Kalajdzievski. 2023. A rank stabilization scaling factor for fine-tuning with lora. Preprint, arXiv:2312.03732. — [arXiv:2312.03732](https://arxiv.org/abs/2312.03732) find
  22. D. Robert Ladd. 2008. Intonational Phonology, 2 edition. Cambridge Studies in Linguistics. Cambridge University Press. find
  23. Gary Lee, Ho-Young Lee, Jieun Song, Byeongchang Kim, Sechun Kang, Jinsik Lee, and Hyosung Hwang. 2016. Automatic sentence stress feedback for non-native english learners. Computer Speech & Language, 41. find
  24. Binghuai Lin, Liyuan Wang, Xiaoli Feng, and Jinsong Zhang. 2020. Joint detection of sentence stress and phrase boundary for prosody. In Interspeech. find
  25. Guan-Ting Lin and Hung-yi Lee. 2024. Can LLMs understand the implication of emphasized sentences in dialogue? In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13391–13401, Miami, Florida, USA. Association for Computational Linguistics. find
  26. Samuel Lipping, Parthasaarathy Sudarsanam, Konstantinos Drossos, and Tuomas Virtanen. 2022. Clotho-aqa: A crowdsourced dataset for audio question answering. Preprint, arXiv:2204.09634. — [arXiv:2204.09634](https://arxiv.org/abs/2204.09634) find
  27. Chengqian Ma, Wei Tao, and Yiwen Guo. 2025. C3: A bilingual benchmark for spoken dialogue models exploring challenges in complex conversations. Preprint, arXiv:2507.22968. find
  28. Gallil Maimon, Michael Hassid, Amit Roth, and Yossi Adi. 2025a. Scaling analysis of interleaved speech-text language models. arXiv preprint arXiv:2504.02398. — [arXiv:2504.02398](https://arxiv.org/abs/2504.02398) find
  29. Gallil Maimon, Amit Roth, and Yossi Adi. 2025b. Salmon: A suite for acoustic language model evaluation. Preprint, arXiv:2409.07437. — [arXiv:2409.07437](https://arxiv.org/abs/2409.07437) find
  30. Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, and 57 others. 2025. Phi-4-mini technical…
  31. Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2025. Large language models: A survey. Preprint, arXiv:2402.06196.
  32. Taniya Mishra, Vivek Rangarajan Sridhar, and Alistair Conkie. 2012. Word prominence detection using robust yet simple prosodic features. In Interspeech 2012, pages 1864–1867. find
  33. Max Morrison, Pranav Pawar, Nathan Pruyne, Jennifer Cole, and Bryan Pardo. 2023. Crowdsourced and automatic speech prominence estimation. Preprint, arXiv:2310.08464. — [arXiv:2310.08464](https://arxiv.org/abs/2310.08464) find
  34. João Moura, Brandon Hancock, and CrewAI Developers. 2023. CrewAI: Fast and Flexible Multi-Agent Automation Framework. https://www.crewai.com. ↗
  35. Tu Anh Nguyen, Wei-Ning Hsu, Antony D’Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarani, Tal Remez, Jade Copet, Gabriel Synnaeve, Michael Hassid, Felix Kreuk, Yossi Adi, and Emmanuel Dupoux. 2023. Expresso: A benchmark and analysis of discrete expressive speech resynthesis. Preprint, arXiv:2308.05725… — [arXiv:2308.05725](https://arxiv.org/abs/2308.05725) find
  36. OpenAI. 2023. Text-to-Speech API. //platform.openai.com/docs/guides/ text-to-speech. find
  37. Brooke Stephenson, Laurent Besacier, Laurent Girin, and Thomas Hueber. 2022. Bert, can he predict contrastive focus? predicting and controlling prominence in neural tts using a language model. Preprint, arXiv:2207.01718.
  38. Aleksander Szwedek. 1986. A Linguistic Analysis of Sentence Stress. [Gunter Narr Verlag, Tubingen]. find
  39. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2024. Salmonn: Towards generic hearing abilities for large language models. Preprint, arXiv:2310.13289.
  40. Vincent Van Heuven. 2018. Acoustic Correlates and Perceptual Cues of Word and Sentence Stress: Theories, Methods and Data, pages 15–59. find
  41. Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, and Nancy F. Chen. 2025. Audiobench: A universal benchmark for audio large language models. Preprint, arXiv:2406.16020.
  42. Changhan Wang, Anne Wu, and Juan Pino. 2020. Covost 2 and massively multilingual speech-to-text translation. Preprint, arXiv:2007.10310. find
  43. Shu wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung yi Lee.… find
  44. Deirdre Wilson and Tim Wharton. 2006. Relevance and prosody. Journal of Pragmatics, 38(10):1559–1579. Special Issue: Prosody and Pragmatics. find
  45. Zhifei Xie and Changqiao Wu. 2024. Mini-omni: Language models can hear, talk while thinking in streaming. Preprint, arXiv:2408.16725. https: find
  46. Ankita Pasad, Ju-Chieh Chou, and Karen Livescu. 2022. Layer-wise analysis of a self-supervised speech representation model. Preprint, arXiv:2107.04734. — [arXiv:2107.04734](https://arxiv.org/abs/2107.04734) find
  47. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. Meld: A multimodal multi-party dataset for emotion recognition in conversations. Preprint, arXiv:1810.02508.
  48. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. Preprint, arXiv:2212.04356.
  49. Rosaria Silipo and Steven Greenberg. 2000. Prosodic stress revisited: Reassessing the role of fundamental frequency. find
  50. Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, and 19 others. 2025. Qwen3-omni technical report. Preprint, arXiv:2509.17765.
  51. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 43 others. 2024a. Qwen2 technical report. Preprint, arXiv:24… find
  52. Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, and Jingren Zhou. 2024b. Air-bench: Benchmarking large audio-language models via generative comprehension. Preprint, arXiv:2402.07729.
  53. Iddo Yosha, Dorin Shteyman, and Yossi Adi. 2025. Whistress: Enriching transcriptions with sentence stress detection. Preprint, arXiv:2505.19103. — [arXiv:2505.19103](https://arxiv.org/abs/2505.19103) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yosha-2026-stresstest-can-your-speech-lm/