Elamur Research⌕
☰
home / papers / 2026 / Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration

Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration

Paper · 2026 · Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang et al.

Summary

Multilingual ASR systems such as Whisper learn from web data in which one language may be transcribed in several writing systems. As a result, they can switch unpredictably between, for example, Latin and Cyrillic for Serbian, or Traditional and Simplified characters for Mandarin. The authors show that script is encoded as a linear direction in the decoder activations of Whisper.

To build a script vector, they prompt the decoder to produce the source and the target script. They keep only outputs that are close to the ground truth, using a normalized Levenshtein threshold. For each layer, the vector is the difference between the two sets of mean-pooled activations. At inference, these vectors are added to every decoder layer with a single scaling factor tuned on validation data, so no training is required.

The method reduces script confusion and outperforms prompt-based biasing across all Whisper sizes. It also produces transliterations for unusual pairings, such as Italian in Cyrillic or Japanese in Latin. This gives cheap post-hoc control over output script, which matters because script choice carries sociopolitical meaning.

Key points

  • Two settings were tested:
    • Script confusion for Serbian (Latin/Cyrillic) and Mandarin (Traditional/Simplified), with gold references generated by cyrtranslit and OpenCC.
    • Zero-shot X→Latin and X→Cyrillic transliteration.
  • Baselines were raw decoding ("no-prompt") and target-script text prompting ("prompt"). Steering performs competitively for every Whisper model size.
  • Data needs are minimal: a usable vector can come from as few as one good-quality example pair.
  • Layer analysis shows script is linearly represented in all decoder layers, which supports steering all layers jointly.
  • Output quality: induced romanizations (e.g., Russian, Hindi, Greek, Japanese, Korean with Whisper large-v2) tend to follow actual pronunciation rather than deterministic spelling rules. Code is available at github.com/mainlp/transliteration.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓University of California, BerkeleyBerkeley · United States175 works · 227,440 citations🎓The University of Texas at AustinAustin · United States109 works · 56,942 citations🎓Ludwig-Maximilians-Universität MünchenMunich · Germany107 works · 56,478 citations🎓Chinese University of Hong Kong, ShenzhenShenzhen · China47 works · 2,596 citations🎓Munich Center for Machine LearningMunich · Germany57 works · 1,094 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang, Ming-Hao Hsu, Florian Eichin, Zhizheng Wu, Alane Suhr, Michael A. Hedderich, David Harwath, David R. Mortensen and Barbara Plank. 2026. Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.464

@inproceedings{shim2026linear,
  title = {{Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration}},
  author = {Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang, Ming-Hao Hsu, Florian Eichin, Zhizheng Wu, Alane Suhr, Michael A. Hedderich, David Harwath, David R. Mortensen and Barbara Plank},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.464},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.464}
}

References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Georges Labrèche. 2025. Cyrtranslit. A Python package for bi-directional transliteration of Cyrillic script to Latin script and vice versa. Supports transliteration for Belarusian, Bulgarian, Greek, Montenegrin, Macedonian, Mongolian, Russian, Serbian, Tajik, and Ukrainian. find
  2. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. find
  3. Sangmin Lee, Woojin Chung, and Hong-Goo Kang. 2025. Lama-ut: Language agnostic multilingual asr through orthography unification and language-specific transliteration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24393–24401. find
  4. Samuel Marks and Max Tegmark. 2024. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling.
  5. Chin-Jou Li, Kalvin Chang, Shikhar Bharadwaj, Eunjung Yeo, Kwanghee Choi, Jian Zhu, David Mortensen, and Shinji Watanabe. 2025. Powsm: A phonetic open whisper-style speech foundation model. Preprint, arXiv:2510.24992. find
  6. Thomas Marshall, Adam Scherlis, and Nora Belrose. 2024. Refusal in llms is an affine function. arXiv preprint arXiv:2411.09003. — [arXiv:2411.09003](https://arxiv.org/abs/2411.09003) find
  7. Haizhou Li, Khe Chai Sim, Jin-Shea Kuo, and Minghui Dong. 2007. Semantic transliteration of personal names. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 120–127, Prague, Czech Republic. Association for Computational Linguistics. find
  8. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. Inference-time intervention: Eliciting truthful answers from a language model. Preprint, arXiv:2306.03341.
  9. Tsung-En Lin, Kuan-Yi Lee, and Hung-Yi Lee. 2025a. Adaptive vector steering: A training-free, layer-wise intervention for hallucination mitigation in large audio and multimodal models. arXiv preprint arXiv:2510.12851. — [arXiv:2510.12851](https://arxiv.org/abs/2510.12851) find
  10. Weilin Lin, Jianze Li, Hui Xiong, and Li Liu. 2025b. Sarsteer: Safeguarding large audio language models via safe-ablated refusal steering. arXiv preprint arXiv:2510.17633. — [arXiv:2510.17633](https://arxiv.org/abs/2510.17633) find
  11. Hexin Liu, Haihua Xu, Leibny Paola Garcia, Andy WH Khong, Yi He, and Sanjeev Khudanpur. 2023. Reducing language confusion for code-switching speech recognition with token-level language diarization. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)… find
  12. Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Ayyoob ImaniGooghari, Orgest Xhelili, Haotian Ye, Chunlan Ma, François Yvon, and Hinrich Schütze. 2025. How transliterations improve crosslingual alignment. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2417–…
  13. Rao Ma, Mengjie Qian, Yassir Fathullah, Siyuan Tang, Mark Gales, and Kate Knill. 2025. Cross-lingual transfer learning for speech translation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (V… find
  14. Kelly Marchisio, Wei-Yin Ko, Alexandre Berard, Théo Dehaze, and Sebastian Ruder. 2024. Understanding and mitigating language confusion in LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6653– 6677, Miami, Florida, USA. Association for Computatio…
  15. Samuel E Martin. 2003. A reference grammar of Japanese. University of Hawaii Press. find
  16. Hiroshi Miura and 1 others. 2024. pykakasi: Kana kanji simple inversion library. find
  17. Alexander Modell, Patrick Rubin-Delanchy, and Nick Whiteley. 2025. The origins of representation manifolds in large language models. arXiv preprint arXiv:2505.18235. — [arXiv:2505.18235](https://arxiv.org/abs/2505.18235) find
  18. Hellina Hailu Nigatu and Hanan Aldarmaki. 2025. Exploring transliteration-based zero-shot transfer for amharic asr. In Proceedings of the Sixth Workshop on African Natural Language Processing (AfricaNLP 2025), pages 64–73.
  19. ASR Omnilingual, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, and 1 others. 2025. Omnilingual asr: Open-source multilingual speech recognition for 1600+ languages. arXiv preprint arXiv:2511.09690. — [arXiv:2511.09690](https://arxiv.org/abs/2511.09690) find
  20. Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
  21. Yifan Peng, Shakeel Muhammad, Yui Sudo, William Chen, Jinchuan Tian, Chyi-Jiunn Lin, and Shinji Watanabe. 2025a. Owsm v4: Improving open whisper-style speech models via data scaling and cleaning. Preprint, arXiv:2506.00338. — [arXiv:2506.00338](https://arxiv.org/abs/2506.00338) find
  22. Yifan Peng, Muhammad Shakeel, Yui Sudo, William Chen, Jinchuan Tian, Chyi-Jiunn Lin, and Shinji Watanabe. 2025b. OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning. In Interspeech 2025, pages 2225–2229. find
  23. Sukannya Purkayastha, Sebastian Ruder, Jonas Pfeiffer, Iryna Gurevych, and Ivan Vulić. 2023. Romanization-based large-scale adaptation of multilingual language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7996–8005, Singapore. Association for Computational… find
  24. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR.
  25. Shauli Ravfogel, Gilad Yehudai, Tal Linzen, Joan Bruna, and Alberto Bietti. 2025. Emergence of linear truth encodings in language models. Advances in Neural Information Processing Systems, 38. find
  26. Brian Roark, Lawrence Wolf-Sonkin, Christo Kirov, Sabrina J Mielke, Cibu Johny, Isin Demirsahin, and Keith Hall. 2020. Processing south asian languages written in the latin script: the dakshina dataset. arXiv preprint arXiv:2007.01176. 2023-2023 IEEE International Conference On Acoustics, Speech And… find
  27. Han Zhu, Gaofeng Cheng, Qingwei Zhao, and Pengyuan Zhang. 2024. Transliterated zero-shot domain adaptation for automatic speech recognition. arXiv preprint arXiv:2412.11185. — [arXiv:2412.11185](https://arxiv.org/abs/2412.11185) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shim-2026-linear-script-representations-speech-foundation/