Elamur Research⌕
☰
home / papers / 2026 / Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care

Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care

Paper · 2026 · Vassilis Lyberatos, Edmund Dervakos, Eleni Adamidi et al.

Summary

Mental health screening still relies largely on interviews and self-reports, which are slow and prone to bias. Many automated speech-based tools are opaque, so clinicians struggle to trust them. The authors propose an interpretable analysis pipeline that pairs perceptually meaningful speech and language descriptors with transparent modelling. They do not aim to produce diagnostic labels.

Acoustic measures come from Parselmouth/Praat: pitch, intensity, jitter, shimmer, HNR, pauses, rhythm and related cues. Linguistic measures cover lexical diversity, syntactic complexity, discourse graphs, coherence and sentiment. Neural models add emotion and sarcasm estimates. The features feed statistical tests and XGBoost classifiers, which are explained with SHAP and LIME. The framework is applied to binary detection of stress, depression, anxiety and ADHD-related status on four benchmark corpora and one real-world clinical dataset. Ablations show which feature groups contribute most.

Across datasets, symptom severity is consistently linked to:

  • voice perturbations such as shimmer and jitter,
  • lexical-syntactic patterns,
  • emotional tone.

The work offers clinicians understandable evidence to support assessment and generate hypotheses, but it is not a replacement for clinical judgement.

Key points

  • Datasets: StressID, DAIC-WOZ, Androids and EATD cover lab stress elicitation, clinical-style interviews and multilingual depression corpora. A fifth dataset comes from real-world digital assessments. Labels come from questionnaire cutoffs such as PHQ-8/9, GAD-7 and ASRS.
  • Tooling:
    • Parselmouth (Praat) for acoustic measures,
    • a HuBERT emotion model fine-tuned on IEMOCAP,
    • spaCy and Stanza for parsing,
    • VADER for sentiment,
    • Sentence-BERT (paraphrase-MiniLM-L6-v2) for coherence.
  • Sarcasm detector: a custom multimodal model combining frozen BERT and Wav2Vec2 encoders, trained on MUStARD. It reaches roughly 70% accuracy and supplies sarcasm probabilities as extra features.
  • PHQ-9 depression findings: emotion cues from both audio and text, plus the content–function word ratio, were among the most influential features.
  • Limitations noted by the authors:
    • confounds such as noise and fatigue,
    • domain shift across devices and languages,
    • imperfect questionnaire-based ground truth,
    • possible bias in neural auxiliary features.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Psychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksMedicine · 241 worksHealth Professions · 69 worksHealth Information Management · 50 worksPhysiology · 29 works

Cite

Cite as Vassilis Lyberatos, Edmund Dervakos, Eleni Adamidi, Athanasios Voulodimos and Giorgos Stamou. 2026. Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care. Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.clpsych-1.15

@inproceedings{lyberatos2026exploration,
  title = {{Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care}},
  author = {Vassilis Lyberatos, Edmund Dervakos, Eleni Adamidi, Athanasios Voulodimos and Giorgos Stamou},
  booktitle = {Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.clpsych-1.15},
  url = {https://doi.org/10.18653/v1/2026.clpsych-1.15}
}

References

58 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tuka Al Hanai, Mohammad M Ghassemi, and James R Glass. 2018. Detecting depression with audio/text sequence modeling of interviews. In Interspeech, pages 1716–1720. find
  2. Nujud Aloshban, Anna Esposito, and Alessandro Vinciarelli. 2022. What you say or how you say it? depression detection through joint modeling of linguistic and acoustic aspects of speech. Cognitive Computation, 14(5):1585–1598. find
  3. Hava Chaptoukaev, Valeriya Strizhkova, Michele Panariello, Bianca Dalpaos, Aglind Reka, Valeria Manera, Susanne Thümmler, Esma ISMAILOVA, Nicholas W., francois bremond, Massimiliano Todisco, Maria A Zuluaga, and Laura M. Ferrari. 2023. Stressid: a multimodal dataset for stress identification. In Adv… find
  4. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785– 794. find
  5. Michael T Compton, Benson S Ku, Michael A Covington, Celia Metzger, and Anya Hogoboom. 2023. Lexical diversity and other linguistic measures in schizophrenia: associations with negative symptoms and neurocognitive performance. The Journal of nervous and mental disease, 211(8):613–620. find
  6. Cheryl M Corcoran, Facundo Carrillo, Diego Fernández-Slezak, Gillinder Bedi, Casimir Klim, Daniel C Javitt, Carrie E Bearden, and Guillermo A Cecchi. 2018. Prediction of psychosis across protocols and risk cohorts using automated language analysis. World Psychiatry, 17(1):67–75. find
  7. Nicholas Cummins, Alice Baird, and Bjoern W Schuller. 2018. Speech analysis for health: Current state-ofthe-art and the increasing impact of deep learning. Methods, 151:41–54. find
  8. Nicholas Cummins, Stefan Scherer, Jarek Krajewski, Sebastian Schnieder, Julien Epps, and Thomas F Quatieri. 2015. A review of depression and suicide risk assessment using speech analysis. Speech communication, 71:10–49. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  10. Alberto Dionigi, Mirko Duradoni, and Laura Vagnoli. 2023. Understanding the association between humor and emotional distress: The role of light and dark humor in predicting depression, anxiety, and stress. Europe’s Journal of Psychology, 19(4):358. find
  11. Jon Donnelly, Luke Moffett, Alina Jade Barnett, Hari Trivedi, Fides Schwartz, Joseph Lo, and Cynthia Rudin. 2024. Asymmirai: interpretable mammography-based deep learning model for 1– 5-year breast cancer risk prediction. Radiology, 310(3):e232780. find
  12. Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
  13. Paul E Engelhardt, Fernanda Ferreira, and Joel T Nigg. 2011. Language production strategies and disfluencies in multi-clause network descriptions: a study of adult attention-deficit/hyperactivity disorder. Neuropsychology, 25(4):442. find
  14. Eric Ettore, Philipp Müller, Jonas Hinze, Michel Benoit, Bruno Giordana, Danilo Postin, Amandine Lecomte, Hali Lindsay, P. Robert, and Alexandra König. 2022. find
  15. Digital phenotyping for differential diagnosis of major depressive episode: A narrative review (preprint). JMIR Mental Health, 10. find
  16. Jonathan Gratch, Ron Artstein, Gale Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, David Traum, Skip Rizzo, and Louis-Philippe Morency. 2014. The distress analysis interview corpus of human and computer interviews. In Proceedings of th… find
  17. James J Gross and Hooria Jazaieri. 2014. Emotion, emotion regulation, and psychopathology: An affective science perspective. Clinical psychological science, 2(4):387–401. find
  18. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):1– 42. find
  19. Andreas Holzinger, Georg Langs, Helmut Denk, Kurt Zatloukal, and Heimo Müller. 2019. Causability and explainability of artificial intelligence in medicine. Wiley interdisciplinary reviews: data mining and knowledge discovery, 9(4):e1312. find
  20. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, Adriane Boyd, and 1 others. 2020. spacy: Industrial-strength natural language processing in python. find
  21. Clayton Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media, volume 8, pages 216–225. find
  22. Florian Hönig, Anton Batliner, Elmar Nöth, Sebastian Schnieder, and Jarek Krajewski. 2014. Automatic modelling of depressed speech: relevant features and relevance of gender. In Interspeech 2014, pages 1248–1252. find
  23. Yannick Jadoul, Bill Thompson, and Bart de Boer. 2018. Introducing Parselmouth: A Python interface to Praat. Journal of Phonetics, 71:1–15. find
  24. Yishan Jiao, Visar Berisha, and Julie Liss. 2017. Interpretable phonological features for clinical applications. In 2017 ieee international conference on acoustics, speech and signal processing (icassp), pages 5045–5049. IEEE. find
  25. Mark Jones, Flora Anagnostou, and Jo Verhoeven. 2011. The vocal expression of emotion: An acoustic analysis of anxiety. In ICPhS, pages 982–985. find
  26. Daniel Korzekwa, Roberto Barra-Chicote, Bozena Kostek, Thomas Drugman, and Mateusz Lajszczak. 2019. Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech. In Interspeech 2019, pages 3890–3894. find
  27. Roman Kotov, Robert Krueger, David Watson, Thomas Achenbach, Robert Althoff, R. Bagby, Timothy Brown, William Carpenter, Avshalom Caspi, Lee Clark, Nicholas Eaton, Miriam Forbes, Kelsie Forbush, David Goldberg, Deborah Hasin, Steven Hyman, Masha Ivanova, Donald Lynam, Kristian Markon, and Mark Zimme… find
  28. Emma CL Leschly, Oliver Roesler, Michael Neumann, Jackson Liscombe, Abhishek Hosamath, Lakshmi Arbatti, Line H Clemmensen, Melanie Ganz, and Vikram Ramanarayanan. 2025. An exploration of interpretable deep learning models for the assessment of mild cognitive impairment. In Proc. find
  29. Interspeech 2025, pages 271–275. find
  30. Daniel M Low, Kate H Bentley, and Satrajit S Ghosh. 2020. Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope investigative otolaryngology, 5(1):96–116. find
  31. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
  32. Felix Menne, Felix Dörr, Julia Schräder, Johannes Tröger, Ute Habel, Alexandra König, and Lisa Wagels. 2024. The voice of depression: speech features as biomarkers for major depressive disorder. BMC psychiatry, 24(1):794. find
  33. Natalia B. Mota, Nathaly A. P. Vasconcelos, Nilza Lemos, Ana C. Pieretti, Osame Kinouchi, Guillermo A. Cecchi, Mauro Copelli, and Sidarta Ribeiro. 2012. Speech graphs provide a quantitative measure of thought disorder in psychosis. PLOS ONE, 7(4):e34928. find
  34. Michael Neumann, Hardik Kothare, Beverly Insel, An-zalee Khan, Danyah Nadim, Jean-Pierre Lindenmayer, and Vikram Ramanarayanan. 2025. Multimodal speech, language and orofacial analysis for remote assessment of positive, negative and cognitive symptoms in schizophrenia. In Proc. find
  35. Interspeech 2025, pages 5703–5707. find
  36. James W Pennebaker, Matthias R Mehl, and Kate G Niederhoffer. 2003. Psychological aspects of natural language use: Our words, our selves. Annual review of psychology, 54(1):547–577. find
  37. Benjamin Pope, Thomas Blass, Aron W Siegman, and Jack Raher. 1970. Anxiety and depression in speech. Journal of Consulting and Clinical Psychology, 35(1p1):128. find
  38. Gowtham Premananth, Philip Resnik, Sonia Bansal, Deanna L. Kelly, and Carol Espy-Wilson. 2025. Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation. In Interspeech 2025, pages 3065–3069. find
  39. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, pages 101–108.
  40. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 3982–3992.
  41. Rachid Riad, Martin Denais, Marc de Gennes, Adrien Lesage, Vincent Oustric, Xuan Nga Cao, Stéphane Mouchabac, and Alexis Bourla. 2024. Automated speech analysis for risk detection of depression, anxiety, insomnia, and fatigue: algorithm development and validation study. Journal of Medical Internet R… find
  42. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135– 1144.
  43. Ying Shen, Huiyu Yang, and Lin Lin. 2022. Automatic depression detection: An emotional audio-textual corpus and a gru/bilstm-based model. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6247–6251. IEEE. find
  44. Si-Ioi Ng, Lingfeng Xu, Ingo Siegert, Nicholas Cummins, Nina R Benway, Julie Liss, and Visar Berisha. 2024. A tutorial on clinical speech ai development: From data collection to model validation. arXiv preprint arXiv:2410.21640. — [arXiv:2410.21640](https://arxiv.org/abs/2410.21640) find
  45. Anke R Sonnenschein, Stefan G Hofmann, Tobias Ziegelmayer, and Wolfgang Lutz. 2018. Linguistic analysis of patients with mood and anxiety disorders during cognitive behavioral therapy. Cognitive behaviour therapy, 47(4):315–327. find
  46. Stavros Ntalampiras. 2025. Interpretable probabilistic identification of depression in speech. Sensors, 25(4):1270. find
  47. Jee Eun Sung, Sujin Choi, Bora Eom, Jae Keun Yoo, and Jee Hyang Jeong. 2020. Syntactic complexity as a linguistic marker to differentiate mild cognitive impairment from normal aging. Journal of Speech, Language, and Hearing Research, 63(5):1416–1429. find
  48. James W Pennebaker and Laura A King. 1999. Linguistic styles: language use as an individual difference. Journal of personality and social psychology, 77(6):1296. find
  49. Fuxiang Tao, Anna Esposito, and Alessandro Vinciarelli. 2023. The Androids Corpus: A New Publicly Available Benchmark for Speech Based Depression Detection. In Interspeech 2023, pages 4149–4153. find
  50. Mashrura Tasnim, Malikeh Ehghaghi, Brian Diep, and Jekaterina Novikova. 2022. DEPAC: a corpus for depression and anxiety detection from speech. In Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, pages 1–16, Seattle, USA. Association for Computational Linguist… find
  51. Bazen Gashaw Teferra, Sophie Borwein, Danielle D DeSouza, William Simpson, Ludovic Rheault, and Jonathan Rose. 2022. Acoustic and linguistic features of impromptu speech and their association with anxiety: validation study. JMIR mental health, 9(7):e36828. find
  52. Ming Tu, Visar Berisha, and Julie Liss. 2017. Interpretable objective assessment of dysarthric speech based on deep neural networks. In Interspeech, pages 1849–1853. find
  53. Wouter van Ballegooijen, Heleen Riper, Pim Cuijpers, Patricia van Oppen, and Johannes H Smit. 2016. Validation of online psychometric instruments for common mental health disorders: a systematic review. BMC psychiatry, 16(1):45. find
  54. Rohit Voleti, Julie M Liss, and Visar Berisha. 2019. A review of automated speech and language features for assessment of cognitive and thought disorders. IEEE journal of selected topics in signal processing, 14(2):282–298. find
  55. Theo Vos, {Stephen S} Lim, Cristiana Abbafati, {Kaja M} Abbas, Mohammad Abbasi, Mitra Abbasifard, Mohsen Abbasi-Kangevari, Hedayat Abbastabar, Foad Abd-Allah, Ahmed Abdelalim, Mohammad Abdollahi, Ibrahim Abdollahpour, Hassan Abolhassani, Victor Aboyans, {Elissa M} Abrams, {Lucas Guimarães} Abreu, {M… find
  56. Lingfeng Xu, Julie Liss, and Visar Berisha. 2023. Dysarthria detection based on a deep learning model with a clinically-interpretable layer. JASA Express Letters, 3(1). find
  57. Shu-Wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung yi Lee.… find
  58. SUPERB: Speech Processing Universal PERformance Benchmark. In Interspeech 2021, pages 1194–1198. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lyberatos-2026-exploration-perceptual-speech-features-clinical/