Elamur Research⌕
☰
home / papers / 2026 / Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs

Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs

Paper · 2026 · Christopher Song and Abdullah P. Rashed Ahmed

Summary

Studies on the bilingual TAUKADIAL benchmark keep finding that speech-based detection of mild cognitive impairment (MCI) works better for Chinese than for English, but nobody had explained why. The authors propose a "tonal salience" hypothesis: in Mandarin, pitch distinguishes word meaning on every syllable and demands fine motor-cognitive control, so cognitive decline should leave clearer acoustic traces than in non-tonal English. They tested this with Gemini 2.5 Pro, prompted without any fine-tuning, on TAUKADIAL-24 under three setups: zero-shot on raw audio, zero-shot on transcripts, and in-context learning (ICL) with audio examples. They then reran the audio setups after flattening F0 with PSOLA, which removes tone but keeps other acoustic cues. Pitch flattening hurt Chinese results clearly, while English results stayed flat or slightly improved. This points to lexical tone as the source of the cross-lingual gap. The authors argue that screening tools should take language typology into account: raw audio suits tonal languages, while text may be enough, and cheaper, for non-tonal ones.

Key points

  • Data: TAUKADIAL-24 (Interspeech 2024 challenge, hosted by DementiaBank) contains picture-description recordings.
    • 507 samples in total.
    • English: 246 samples (93 NC / 153 MCI).
    • Chinese: 261 samples (129 NC / 132 MCI).
  • Effect of pitch flattening on Chinese (UAR): −4.78 in zero-shot and −5.92 in ICL.
  • Effect of pitch flattening on English (UAR): +0.11 in zero-shot and +2.98 in ICL.
  • ICL vs. zero-shot: ICL with randomly chosen exemplars outperformed zero-shot prompting, including earlier zero-shot AudioLLM work with Qwen2-Audio. The model was run at default temperature 1, and gemini-3.0-pro was also tried on the ICL audio setup.
  • Stated limitations:
    • The pitch-flattening effect was shown on only one main model.
    • TAUKADIAL is the only multilingual MCI speech corpus available.
    • The task is framed as binary classification only.
    • The model's decisions are a black box.
    • Future work should test other languages such as Cantonese, Vietnamese, Thai and Japanese.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Brown UniversityProvidence · United States76 works · 25,952 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksMedicine · 241 worksPhysiology · 29 worksPsychiatry and Mental health · 9 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Christopher Song and Abdullah P. Rashed Ahmed. 2026. Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.81

@inproceedings{song2026tonal,
  title = {{Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs}},
  author = {Christopher Song and Abdullah P. Rashed Ahmed},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.81},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.81}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. [2] Alzheimer’s Association. 2023. 2023 Alzheimer’s disease facts and figures. Alzheimer’s & Dementia, 19(4). find
  2. [4] Tom B. Brown, Benjamin Mann, Nick Ryder, and 1 others. 2020. Language models are few-shot learners. In NeurIPS.
  3. [14] Alyssa M. Lanzi, Anna K. Saylor, Davida Fromm, Hui Liu, Brian MacWhinney, and Matthew Cohen. 2023. DementiaBank: Theoretical rationale, protocol, and illustrative analyses. American Journal of Speech-Language Pathology. find
  4. [15] Siyu Liang and Gina-Anne Levow. 2025. Tone in perspective: A computational typological analysis of tone function in ASR. In Proceedings of the 7th Workshop on Research in Computational Linguistic Typology and Multilingual NLP, pages 82–92, Vienna, Austria. Association for Computational Linguist… find
  5. [16] Junfeng Lu, Yuanning Li, Zehao Zhao, Yan Liu, Yanming Zhu, Ying Mao, Jinsong Wu, and Edward F Chang. 2023. Neural control of lexical tone production in human laryngeal motor cortex. Nature Communications, 14(1):6150. find
  6. [28] H. Valbret, E. Moulines, and J. P. Tubach. 1992. Voice transformation using PSOLA technique. Speech Communication, 11(2):175–187. find
  7. [17] S. Luz, S. De La Fuente Garcia, F. Haider, and 1 others. 2024. Connected speech-based cognitive assessment in Chinese and English. In Interspeech. find
  8. [29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  9. [18] M. Martinc and S. Pollak. 2020. Tackling the ADReSS challenge: a multimodal approach to the automated recognition of Alzheimer’s dementia. In Interspeech. find
  10. [30] W3Techs. 2026. Usage statistics of content languages for websites. https://w3techs.com/ technologies/overview/content_language. Accessed: 8 May 2026. ↗
  11. [19] Francisco Martínez-Sánchez, Juan José García Meilán, Julia García-Sevilla, Juan Carro, and José M Arana. 2012. Temporal parameters of spontaneous speech in Alzheimer’s disease. International Journal of Speech-Language Pathology, 14(5):428–436. find
  12. [31] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 43 others. 2024. Qwen2 technical report. Preprint, arXi… find
  13. [20] Milap A. Nowrangi, Paul B. Rosenberg, and Jeannie-Marie S. Leoutsakos. 2016. Subtle changes in daily functioning predict conversion from normal to mild cognitive impairment or dementia. International Psychogeriatrics, 28(12):2009–2018. find
  14. [32] J. Yuan, Y. Bian, X. Cai, and 1 others. 2020. Disfluencies and fine-tuning pre-trained language models for detection of Alzheimer’s disease. In Interspeech. find
  15. [21] Kristin Qi, Jiali Cheng, Youxiang Zhu, Hadi Amiri, and Xiaohui Liang. 2025. Unveil multi-picture descriptions for multilingual mild cognitive impairment detection via contrastive learning. arXiv preprint arXiv:2505.17067. — [arXiv:2505.17067](https://arxiv.org/abs/2505.17067) find
  16. [22] Alec Radford, Jong Wook Kim, Tao Xu, and 1 others. 2023. Robust speech recognition via large-scale weak supervision. In ICML.
  17. [23] Paul R Rosenbaum and Donald B Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41–55. find
  18. [24] Mostafa Shahin, Beena Ahmed, and Julien Epps. 2025. Zero-shot cognitive impairment detection from speech using AudioLLM. In Interspeech. find
  19. [25] Ravi Shankar, Anjali Bundele, and Amartya Mukhopadhyay. 2025. A systematic review of natural language processing techniques for early detection of cognitive impairment. Mayo Clinic Proceedings: Digital Health, 3(2):100205. find
  20. [26] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambro… find
  21. [27] Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2026-tonal-salience-cognitive-decline-context/