Elamur Research⌕
☰
home / papers / 2026 / When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews

When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews

Summary

Text-based depression detection from clinical interviews often reports strong scores, but it is rarely clear what the models actually rely on. This paper asks whether a classifier can identify depressed participants using only the interviewer's side of the conversation. The authors study three semi-structured interview corpora: ANDROIDS (Italian), DAIC-WOZ and E-DAIC (English). Interviewer and participant transcripts for ANDROIDS and E-DAIC were produced with WhisperX. Two architectures, Longformer and the ω-GCN, were each trained in two variants: participant-only (P) and interviewer-only (I). In many settings, I-models equal or beat P-models. The gap is largest on ANDROIDS, where I-Longformer reaches about 0.98 macro-F1. Interpretability analyses show that I-models exploit script artifacts, namely recurring prompts and their positions in the interview, and concentrate on narrow segments. P-models instead draw on evidence spread across the whole interview. The authors call this "prompt-induced bias" and show it holds across datasets and architectures. The practical lesson is that including interviewer prompts can inflate reported results, so evaluations should check which speaker and which part of the interview drive predictions.

Key points

  • Corpora:
    • DAIC-WOZ: 189 interviews (107/35/47 split), led by the virtual agent "Ellie," who draws on a fixed set of 191 prompts.
    • E-DAIC: 163/56/56 split, fully automatic interviewer; only participant transcripts are released, and some mislabeled subjects were corrected.
    • ANDROIDS: 116 Italian speakers (64 depressed, 52 controls); only the Interview Task is used.
  • Transcription: WhisperX with Whisper large-v3 was used for both speakers in ANDROIDS and E-DAIC. Its WER on the E-DAIC participant side is 15.3%.
  • Dev macro-F1 for GCN, P → I:
    • ANDROIDS: 0.93 → 0.97
    • DAIC-WOZ: 0.85 → 0.88
    • E-DAIC: 0.70 → 0.74
  • Dev macro-F1 for Longformer, P → I:
    • ANDROIDS: 0.79 → 0.98
    • DAIC-WOZ: 0.71 → 0.73
    • E-DAIC: 0.67 → 0.65, the one case where the I-model is slightly worse.
  • Relation to prior work: The study extends Burdisso et al. (2024), which examined only DAIC-WOZ. It adds more corpora, held-out test evaluation and ASR transcripts. Limitations include possible ASR noise and a text-only scope, with multimodal analysis left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/34hw23mzd8c7

@inproceedings{watawana2026when,
  title = {{When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews}},
  author = {Watawana, Hasindri and Burdisso, Sergio and Moreno-Galván, Diego A. and Sánchez-Vega, Fernando and López-Monroy, A. Pastor and Motlicek, Petr and Villatoro-Tello, Esaú},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/34hw23mzd8c7},
  url = {https://doi.org/10.63317/34hw23mzd8c7}
}

References

25 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Navneet Agarwal and Gaël Dias. 2024. Analysing Relevance of Discourse Structure for Improved Mental Health Estimation. In 9th Workshop on Computational Linguistics and Clinical Psychology (CLPSYCH) associated to 18th Conference of the European Chapter of the Association for Computational Linguistics… find
  2. Abdelrahman A. Ali, Aya E. Fouda, Radwa J. Hanafy, and Mohammed E. Fouda. 2025. Leveraging audio and text modalities in mental health: A study of llms performance. find
  3. American Psychiatric Association. 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). American Psychiatric Publishing. find
  4. Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisserman. 2023. Whisperx: Time-accurate speech transcription of long-form audio. find
  5. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. find
  6. Annapia Borraccino. 2025. Modeling depressive patterns in italian discourse: Insights from natural language processing. Master’s thesis, The Graduate Center, City University of New York (CUNY), New York, NY. find
  7. Sergio Burdisso, Ernesto Reyes-Ramírez, Esaú Villatoro-Tello, Fernando Sánchez-Vega, Pastor López-Monroy, and Petr Motlicek. 2024. find
  8. Daic-woz: On the validity of using the therapist’s prompts in automatic depression detection from clinical interviews. arXiv preprint arXiv:2404.14463. find
  9. Sergio Burdisso, Esaú Villatoro-Tello, Srikanth Madikeri, and Petr Motlicek. 2023. Node-weighted Graph Convolutional Network for Depression Detection in Transcribed Clinical Interviews. In Proc. find
  10. INTERSPEECH 2023, pages 3617–3621. find
  11. Loukas Ilias and Dimitris Askounis. 2024. A Cross-Attention Layer coupled with Multimodal Fusion Methods for Recognizing Depression from Spontaneous Speech. In Interspeech 2024, pages 912–916. find
  12. Kurt Kroenke, Tara W Strine, Robert L Spitzer, Janet BW Williams, Joyce T Berry, and Ali H Mokdad. 2009. The phq-8 as a measure of current depression in the general population. Journal of affective disorders, 114(1-3):163–173. find
  13. Nikolaos Malandrakis and Shrikanth S Narayanan. 2015. Therapy language analysis using automatically generated psycholinguistic norms. In Proc. find
  14. Kirill Milintsevich, Kairit Sirts, and Gaël Dias. 2023. Towards automatic text-based estimation of depression through symptom prediction. Brain informatics, 10(1):4. find
  15. James W Pennebaker, Matthias R Mehl, and Kate G Niederhoffer. 2003. Psychological aspects of natural language use: Our words, our selves. Annual review of psychology, 54(1):547– 577. find
  16. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. find our card
  17. Ying Shen, Huiyu Yang, and Lin Lin. 2022. Automatic depression detection: An emotional audio-textual corpus and a gru/bilstm-based model. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6247–6251. IEEE. find
  18. Allison M Tackman, David A Sbarra, Angela L Carey, M Brent Donnellan, Andrea B Horn, Nicholas S Holtzman, To’Meisha S Edwards, James W Pennebaker, and Matthias R Mehl. 2019. Depression, negative emotionality, and self-referential language: A multi-lab, multi-measure, and multi-language-task research… find
  19. Esaú Villatoro-Tello, Gabriela Ramírez-de-la Rosa, Daniel Gática-Pérez, Mathew Magimai.-Doss, and Héctor Jiménez-Salazar. 2021a. Approximating the mental lexicon from clinical interviews as a support tool for depression detection. In Proc. ICMI’21, page 557–566. find
  20. Esaú Villatoro-Tello, S. Pavankumar Dubagunta, Gabriela Ramírez-de-la-Rosa Julian Fritsch, Petr Motlicek, and Mathew Magimai-Doss. 2021b. Late Fusion of the Available Lexicon and Raw Waveform-Based Acoustic Modeling for Depression and Dementia Recognition. In Proc. find
  21. Interspeech 2021, pages 1927–1931. find
  22. Chen Zhuang, Deng Jiawen, Zhou Jinfeng, Wu Jincenzi, Qian Tieyun, and Minlie Huang. 2024. Depression detection in clinical interviews with LLM-empowered structural element graph. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hum… find
  23. Language Resource References David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, et al. 2014. Simsensei kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 internationa… find
  24. Jonathan Gratch, Ron Artstein, Gale M Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, et al. 2014. The distress analysis interview corpus of human and computer interviews. In Lrec, volume 14, pages 3123–3128. Reykjavik. find
  25. Fuxiang Tao, Anna Esposito, and Alessandro Vinciarelli. 2023. The androids corpus: A new publicly available benchmark for speech based depression detection. In Interspeech 2023, pages 4149–4153. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/watawana-2026-when-consistency-becomes-bias-interviewer/